Hi fellow wiki editors!

To help newly registered users get more familiar with the wiki (and maybe older users too) there is now a {{Welcome to the wiki}} template. Have a look at it and feel free to add it to new users discussion pages (and perhaps your own).

I have tried to keep the template short, but meaningful. /Johan G

Difference between revisions of "Compositor"

From FlightGear wiki
Jump to: navigation, search
m (Status: https://sourceforge.net/p/flightgear/mailman/message/36391516/)
(Status)
Line 56: Line 56:
  
 
== Status ==
 
== Status ==
 +
'''08/2018'''
 +
 
Fernando has been working on and off on multi-pass rendering support for
 
Fernando has been working on and off on multi-pass rendering support for
 
FlightGear since late 2017. It went through several iterations and
 
FlightGear since late 2017. It went through several iterations and

Revision as of 07:45, 28 October 2018

Compositor Subsystem
Canvas-view-element-prototype-by-icecode gl.png
Started in 01/2018
Description Dynamic rendering pipeline configured via the property tree
Maintainer(s) none
Contributor(s) Icecode
Status experimental as of 02/2018

A Compositor is a rendering pipeline configured by the Property Tree. Every configurable element of the pipeline wraps an OpenGL/OSG object and exposes its parameters to the property tree. This way, the rendering pipeline becomes dynamically-reconfigurable at runtime.

Related efforts: Howto:Canvas View Camera Element

Background

First discussed in 03/2012 during the early Rembrandt days, Zan came up with patches demonstrating how to create an XML-configurable rendering pipeline.

Back then, this work was considered to look pretty promising [1] and at the time plans were discussed to unify this with the ongoing Rembrandt implementation (no longer maintained).

Adopting Zan's approach would have meant that efforts like Rembrandt could have been implemented without requiring C++ space modifications.

Rembrandt's developer (FredB) suggested to extend the format to avoid duplicating the stages when you have more than one viewport, i.e. specifying a pipeline as a template, with conditions like in effects, and have the current camera layout refer the pipeline that would be duplicated, resized and positioned for each declared viewport [2]


Zan's original patches can still be found in his newcameras branches which allow the user to define the rendering pipeline in preferences.xml:

At that point, it didn't have everything Rembrandt's pipeline needs, but most likely could be easily enhanced to support those things.

Basically the original version added support for multiple camera passes, texture targets, texture formats, passing textures from one pass to another etc, while preserving the standard rendering line if user wants that. [3]

In 02/2018, Icecode GL re-implemented a more generic version of Zan's idea using Canvas concepts, which means the rendering pipeline is not just xml-configurable, but using listeners to be dynamically-reconfigurable at runtime via properties.

Status

08/2018

Fernando has been working on and off on multi-pass rendering support for FlightGear since late 2017. It went through several iterations and design changes, but he think it's finally on the right track. It's heavily based on the Ogre3D Compositor and inspired by many data-driven rendering pipelines. Its features include:

  • Completely independent of other parts of the simulator, i.e. it's part of SimGear and can be used in a standalone fashion if needed, ala Canvas.
  • Although independent, its aim is to be fully compatible with the current rendering framework in FG. This includes the Effects system, CameraGroup, Rembrandt and ALS (and obviously the Canvas).
  • Its functionality overlaps Rembrandt: what can be done with Rembrandt can be done with the Compositor, but not vice versa.
  • Fully configurable via a XML interface without compromising performance (ala Effects, using PropertyList files).
  • Optional at compile time to aid merge request efforts.
  • Flexible, expandable and compatible with modern graphics.
  • It doesn't increase the hardware requirements, it expands the hardware range FG can run on. People with integrated GPUs (Intel HD etc) can run a Compositor with a single pass that renders directly to the screen like before, while people with more powerful cards can run a Compositor that implements deferred rendering, for example.

Unlike Rembrandt, the Compositor makes use of scene graph cameras instead of viewer level cameras.

This allows CameraGroup to manage windows, near/far cameras and other slaves without interfering with what is being rendered (post-processing, shadows...).

The Compositor is in an usable state right now: it works but there are no effects or pipelines developed for it. There are also some bugs and features that don't work as expected because of some hardcoded assumptions in the FlightGear Viewer code. Still, I think it's time to announce it so people much more knowledgeable than me can point me in the right direction to get this upstream and warn me about possible issues and worries. :)

A screenshot showing post-processed gaussian blur and a blue filter: https://i.imgur.com/Zzxre8a.png

The next major (and hopefully final) iteration of the Compositor is mostly done now, just some more cleanup is needed. I've tried to keep the changes contained in CameraGroup.cxx/hxx and in new files in SimGear to ease the merging process.

Use Cases

Elements

Buffers

A buffer represents a texture or, more generically, a region of GPU memory. Textures can be of any type allowed by OpenGL: 1D, 2D, rectangle, 2Darray, 3D or cubemap.

A typical property tree structure describing a buffer may be as follows:

      <buffer>
            <name>buffer-name</name>
	    <type>2d</type>
	    <width>512</width>
	    <height>512</height>
	    <scale-factor>1.0</scale-factor>
	    <internal-format>rgba8</internal-format>
	    <source-format>rgba</source-format>
            <source-type>ubyte</source-type>
      </buffer>

Passes

A pass wraps around an osg::Camera. As of February 2018, there are two types of passes supported:

  • scene. Renders a scene. The scene can be an already loaded scene graph node (terrain, aircraft etc.) or a path to a custom 3D model.
  • quad. Renders a fullscreen quad with an optional effect applied. Useful for screen space shaders (like SSAO, Screen Space Reflections or bloom) and deferred rendering.

Passes can render to a buffer (Render to Texture), to several buffers (Multiple Render Targets) or directly to the OSG context. This allows chaining of multiple passes, sharing buffers between them. They can also receive other buffers as input, so effects can access them as normal textures.

Example XML for a quad type pass:

      <pass-quad>
          <name>pass-name</name>

          <effect>Effects/test</effect>
    
          <input-buffer>
              <buffer>some-buffer</buffer>
              <unit>0</unit>
          </input-buffer>

          <output-buffer>
              <buffer>color-buffer</buffer>
              <component>color</component>
          </output-buffer>
          <output-buffer>
              <buffer>depth-buffer</buffer>
              <component>depth</component>
          </output-buffer>
      </pass-quad>

Canvas integration

Post processed image (right) after applying gaussian blur and a blue filter to a scene pass (left).

Apart from serving as a debugging tool for visualizing the contents of a buffer, integrating the Compositor with Canvas allows aircraft developers to access RTT capabilities. Compositor buffers can be accessed within Canvas via a new custom Canvas Image protocol buffer://. For example, the path buffer://test-compositor/test-buffer displays the buffer test-buffer declared in test-compositor.

var (width,height) = (612,612);
var title = 'Compositor&Canvas test';
var window = canvas.Window.new([width,height],"dialog")
 .set('title',title);
var myCanvas = window.createCanvas().set("background", canvas.style.getColor("bg_color"));
var root = myCanvas.createGroup();
var path = "buffer://test-compositor/test-buffer";
var child = root.createChild("image")
    .setTranslation(50, 50)
    .setSize(512, 512)
    .setFile(path);

Known Issues

  • Canvas (sc::Image) doesn't update its texture unless .update() is invoked in Nasal. Is this a feature?
  • The scene graph and the viewer are still managed by FGRenderer. Maybe hijacking the init sequence is a good idea?
  • light sources ?

Related

References
  1. Mathias Fröhlich  (Mar 7th, 2012).  Re: [Flightgear-devel] [Rembrandt] the plan .
  2. Frederic Bouvier  (Mar 7th, 2012).  Re: [Flightgear-devel] [Rembrandt] the plan .
  3. Lauri Peltonen  (Mar 7th, 2012).  [Flightgear-devel] [Rembrandt] the plan .