Hi fellow wiki editors!

To help newly registered users get more familiar with the wiki (and maybe older users too) there is now a {{Welcome to the wiki}} template. Have a look at it and feel free to add it to new users discussion pages (and perhaps your own).

I have tried to keep the template short, but meaningful. /Johan G

Difference between revisions of "Compositor"

From FlightGear wiki
Jump to: navigation, search
(Status)
Line 110: Line 110:
 
== Canvas integration ==
 
== Canvas integration ==
  
[[File:Canvas&Compositor test.png|thumb|A fullscreen quad with a shader that colors everything magenta is rendered to a buffer, which is displayed by a Canvas Image.]]
+
[[File:Blur_and_blue_filter_applied_to_a_buffer.png|thumb|Post processed image (right) after applying gaussian blur and a blue filter to a scene pass (left).]]
  
 
Apart from serving as a debugging tool for visualizing the contents of a buffer, integrating the Compositor with [[Canvas]] allows aircraft developers to access RTT capabilities. Compositor buffers can be accessed within Canvas via a new custom Canvas Image protocol '''buffer://'''. For example, the path <code>buffer://test-compositor/test-buffer</code> displays the buffer test-buffer declared in test-compositor.
 
Apart from serving as a debugging tool for visualizing the contents of a buffer, integrating the Compositor with [[Canvas]] allows aircraft developers to access RTT capabilities. Compositor buffers can be accessed within Canvas via a new custom Canvas Image protocol '''buffer://'''. For example, the path <code>buffer://test-compositor/test-buffer</code> displays the buffer test-buffer declared in test-compositor.

Revision as of 16:53, 27 February 2018

Compositor Subsystem
Canvas-view-element-prototype-by-icecode gl.png
Started in 01/2018
Description Dynamic rendering pipeline configured via the property tree
Maintainer(s) none
Contributor(s) Icecode
Status experimental as of 02/2018

A Compositor is a rendering pipeline configured by the Property Tree. Every configurable element of the pipeline wraps an OpenGL/OSG object and exposes its parameters to the property tree. This way, the rendering pipeline becomes dynamically-reconfigurable at runtime.

Related efforts: Howto:Canvas View Camera Element

Background

First discussed in 03/2012 during the early Rembrandt days, Zan came up with patches demonstrating how to create an XML-configurable rendering pipeline.

Back then, this work was considered to look pretty promising [1] and at the time plans were discussed to unify this with the ongoing Rembrandt implementation (no longer maintained).

Adopting Zan's approach would have meant that efforts like Rembrandt could have been implemented without requiring C++ space modifications.

Rembrandt's developer (FredB) suggested to extend the format to avoid duplicating the stages when you have more than one viewport, i.e. specifying a pipeline as a template, with conditions like in effects, and have the current camera layout refer the pipeline that would be duplicated, resized and positioned for each declared viewport [2]


Zan's original patches can still be found in his newcameras branches which allow the user to define the rendering pipeline in preferences.xml:

At that point, it didn't have everything Rembrandt's pipeline needs, but most likely could be easily enhanced to support those things.

Basically the original version added support for multiple camera passes, texture targets, texture formats, passing textures from one pass to another etc, while preserving the standard rendering line if user wants that. [3]

In 02/2018, Icecode GL re-implemented a more generic version of Zan's idea using Canvas concepts, which means the rendering pipeline is not just xml-configurable, but using listeners to be dynamically-reconfigurable at runtime.

Status

  • 02/2018 Passes now support both input and output textures.

Use Cases

Elements

Buffers

A buffer represents a texture or, more generically, a region of GPU memory. Textures can be of any type allowed by OpenGL: 1D, 2D, rectangle, 2Darray, 3D or cubemap.

A typical property tree structure describing a buffer may be as follows:

      <buffer>
            <name>buffer-name</name>
	    <type>2d</type>
	    <width>512</width>
	    <height>512</height>
	    <scale-factor>1.0</scale-factor>
	    <internal-format>rgba8</internal-format>
	    <source-format>rgba</source-format>
            <source-type>ubyte</source-type>
      </buffer>

Passes

A pass wraps around an osg::Camera. As of February 2018, there are two types of passes supported:

  • scene. Renders a scene. The scene can be an already loaded scene graph node (terrain, aircraft etc.) or a path to a custom 3D model.
  • quad. Renders a fullscreen quad with an optional effect applied. Useful for screen space shaders (like SSAO, Screen Space Reflections or bloom) and deferred rendering.

Passes can render to a buffer (Render to Texture), to several buffers (Multiple Render Targets) or directly to the OSG context. This allows chaining of multiple passes, sharing buffers between them.

Example XML for a quad type pass:

      <pass>
          <type>quad</type>
          <name>pass-name</name>

          <effect>Effects/test.eff</effect>
    
          <output-buffer>
              <buffer>buffer-name</buffer>
              <component>color</component>
          </output-buffer>
      </pass>

Canvas integration

Post processed image (right) after applying gaussian blur and a blue filter to a scene pass (left).

Apart from serving as a debugging tool for visualizing the contents of a buffer, integrating the Compositor with Canvas allows aircraft developers to access RTT capabilities. Compositor buffers can be accessed within Canvas via a new custom Canvas Image protocol buffer://. For example, the path buffer://test-compositor/test-buffer displays the buffer test-buffer declared in test-compositor.

var (width,height) = (612,612);
var title = 'Compositor&Canvas test';
var window = canvas.Window.new([width,height],"dialog")
 .set('title',title);
var myCanvas = window.createCanvas().set("background", canvas.style.getColor("bg_color"));
var root = myCanvas.createGroup();
var path = "buffer://test-compositor/test-buffer";
var child = root.createChild("image")
    .setTranslation(50, 50)
    .setSize(512, 512)
    .setFile(path);

Known Issues

  • Canvas (sc::Image) doesn't update its texture unless .update() is invoked in Nasal. Is this a feature?
  • The scene graph and the viewer are still managed by FGRenderer. Maybe hijacking the init sequence is a good idea?
  • Effects need that buffers are created before reading the .eff file. Maybe some kind of listener is needed ala Rembrandt?
  • light sources ?

Related

References
  1. Mathias Fröhlich  (Mar 7th, 2012).  Re: [Flightgear-devel] [Rembrandt] the plan .
  2. Frederic Bouvier  (Mar 7th, 2012).  Re: [Flightgear-devel] [Rembrandt] the plan .
  3. Lauri Peltonen  (Mar 7th, 2012).  [Flightgear-devel] [Rembrandt] the plan .