Hi fellow wiki editors!

To help newly registered users get more familiar with the wiki (and maybe older users too) there is now a {{Welcome to the wiki}} template. Have a look at it and feel free to add it to new users discussion pages (and perhaps your own).

I have tried to keep the template short, but meaningful. /Johan G

Difference between revisions of "Compositor"

From FlightGear wiki
Jump to: navigation, search
m (Use Cases: https://sourceforge.net/p/flightgear/mailman/message/36339959/)
 
(85 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
{{Template:Non-stable|version=2019.2|build_flags=-DENABLE_COMPOSITOR|progress=100}}
 +
 +
 +
 +
{{forum|47|Effects & Shaders}}
 +
 
{{infobox subsystem
 
{{infobox subsystem
|image      = Canvas-view-element-prototype-by-icecode gl.png
+
|image      = ALS Compositor pipeline.jpg
 
|name        = Compositor Subsystem
 
|name        = Compositor Subsystem
|started    = 01/2018
+
|started    = 01/2018 (Available since FlightGear 2019.2)
|description = Dynamic rendering pipeline configured via the property tree
+
|description = Dynamic rendering pipeline configured via the property tree and XML
|status      = experimental as of 02/2018
+
|status      = Stable
|maintainers = none
+
|developers  = Fernando García Liñán
|developers  = Icecode
+
|folders =  
 +
* {{flightgear file|src/Viewer}}
 +
* {{simgear file|simgear/scene/viewer}}
 +
* {{fgdata file|Compositor}}
 
}}
 
}}
  
A '''Compositor''' is a rendering pipeline configured by the [[Property Tree]]. Every configurable element of the pipeline wraps an OpenGL/OSG object and exposes its parameters to the property tree. This way, the rendering pipeline becomes dynamically-reconfigurable at runtime.
+
The '''Compositor''' aims to bring multi-pass rendering to FlightGear. It encapsulates a rendering pipeline and exposes its parameters to a [[Property Tree]] interface. At startup, FlightGear reads the pipeline definition file for each physical viewport defined on the [[Howto:Configure camera view windows|CameraGroup settings]]. If no Compositor file is specified for a physical camera, the one given by the <code>--compositor=</code> startup command will be used. If such startup option is not used either, FlightGear will look for a valid Compositor file in $FG_ROOT/Compositor/default.xml
  
Related efforts: [[Howto:Canvas View Camera Element]]
+
The Compositor introduces a new dedicated fgdata directory for new/custom rendering pipelines: {{Fgdata file|Compositor}}.
  
 
== Background ==
 
== Background ==
First discussed in 03/2012 during the early [[Rembrandt]] days, Zan came up with patches demonstrating how to create an XML-configurable rendering pipeline.
+
{{See also|Supporting multiple renderers|Howto:Canvas View Camera Element}}
 +
 
 +
First discussed in 03/2012 during the early [[Rembrandt]] days, Zan (Lauri Peltonen) came up with a set of patches demonstrating how to create an XML-configurable rendering pipeline.
  
 
Back then, this work was considered to look pretty promising <ref>{{cite web
 
Back then, this work was considered to look pretty promising <ref>{{cite web
Line 25: Line 36:
 
   }}</ref> and at the time plans were discussed to unify this with the ongoing Rembrandt implementation (no longer maintained).
 
   }}</ref> and at the time plans were discussed to unify this with the ongoing Rembrandt implementation (no longer maintained).
  
Adopting Zan's approach would have meant that efforts like Rembrandt could have been implemented without requiring C++ space modifications.
+
Adopting Zan's approach would have meant that efforts like [[Rembrandt]] (deferred rendering) could have been implemented without requiring C++ space modifications, i.e. purely in [[Base package]] space.
  
 
Rembrandt's developer (FredB) suggested to extend the format to avoid duplicating the stages when you have more than one viewport, i.e.  specifying a pipeline as a template, with conditions like in effects, and have the current camera layout refer the pipeline that would be duplicated, resized and positioned for each declared viewport <ref>{{cite web
 
Rembrandt's developer (FredB) suggested to extend the format to avoid duplicating the stages when you have more than one viewport, i.e.  specifying a pipeline as a template, with conditions like in effects, and have the current camera layout refer the pipeline that would be duplicated, resized and positioned for each declared viewport <ref>{{cite web
Line 35: Line 46:
 
   |script_version = 0.36  
 
   |script_version = 0.36  
 
   }}</ref>
 
   }}</ref>
 
  
 
Zan's original patches can still be found in his newcameras branches which allow the user to define the rendering pipeline in preferences.xml:
 
Zan's original patches can still be found in his newcameras branches which allow the user to define the rendering pipeline in preferences.xml:
Line 44: Line 54:
 
At that point, it didn't have everything Rembrandt's pipeline needs, but most likely could be easily enhanced to support those things.  
 
At that point, it didn't have everything Rembrandt's pipeline needs, but most likely could be easily enhanced to support those things.  
  
Basically the original version added support for multiple camera passes, texture targets, texture formats, passing textures from one pass to another etc, while preserving the standard rendering line if user wants that. <ref>{{cite web
+
Basically, the original version added support for multiple camera passes, texture targets, texture formats, passing textures from one pass to another etc, while preserving the standard rendering line if user wants that. <ref>{{cite web
 
   |url    =  https://sourceforge.net/p/flightgear/mailman/message/28944733/  
 
   |url    =  https://sourceforge.net/p/flightgear/mailman/message/28944733/  
 
   |title  =  <nowiki> [Flightgear-devel] [Rembrandt] the plan </nowiki>  
 
   |title  =  <nowiki> [Flightgear-devel] [Rembrandt] the plan </nowiki>  
Line 53: Line 63:
 
   }}</ref>
 
   }}</ref>
  
In 02/2018, Icecode GL re-implemented a more generic version of Zan's idea using Canvas concepts, which means the rendering pipeline is not just xml-configurable, but using listeners to be dynamically-reconfigurable at runtime via properties.
+
Since the early days of Zan's groundwork, providing the (hooks) infrastructure to enable base package developers to prototype, test and develop distinct rendering pipelines without requiring C++ space modifications has been a long-standing idea, especially after the [[Canvas]] system became available in early 2012, which demonstrated how RTT-rendering buffers (FBOs) could be set up, created and manipulated procedurally (i.e. at run-time) using XML, the property tree and [[Nasal]] scripting. <ref>https://forum.flightgear.org/search.php?st=0&sk=t&sd=d&sr=posts&keywords=zan++rembrandt+canvas</ref>
 +
 
 +
The new '''Compositor''' is an improved re-implementation of Zan's original work using not just XML, but also [[Property Tree|properties]] and a handful of [[Canvas]] concepts.
  
 
== Status ==
 
== Status ==
'''08/2018'''
+
'''06/2019'''
  
Fernando has been working on and off on multi-pass rendering support for
+
The Compositor core is complete and stable. Effects and shaders still need to be adapted to the Compositor.
FlightGear since late 2017. It went through several iterations and
+
 
design changes, but he think it's finally on the right track. It's
+
== Features ==
heavily based on the Ogre3D Compositor and inspired by many
+
data-driven rendering pipelines. Its features include:
+
  
 
* Completely independent of other parts of the simulator, i.e. it's part of SimGear and can be used in a standalone fashion if needed, ala Canvas.
 
* Completely independent of other parts of the simulator, i.e. it's part of SimGear and can be used in a standalone fashion if needed, ala Canvas.
 
* Although independent, its aim is to be fully compatible with the current rendering framework in FG. This includes the Effects system, CameraGroup, Rembrandt and ALS (and obviously the Canvas).
 
* Although independent, its aim is to be fully compatible with the current rendering framework in FG. This includes the Effects system, CameraGroup, Rembrandt and ALS (and obviously the Canvas).
 
* Its functionality overlaps Rembrandt: what can be done with Rembrandt can be done with the Compositor, but not vice versa.
 
* Its functionality overlaps Rembrandt: what can be done with Rembrandt can be done with the Compositor, but not vice versa.
* Fully configurable via a XML interface without compromising performance (ala Effects, using PropertyList files).
+
* Fully configurable via an XML interface without compromising performance (ala Effects, using PropertyList files).
* Optional at compile time to aid merge request efforts.
+
 
* Flexible, expandable and compatible with modern graphics.
 
* Flexible, expandable and compatible with modern graphics.
 
* It doesn't increase the hardware requirements, it expands the hardware range FG can run on. People with integrated GPUs (Intel HD etc) can run a Compositor with a single pass that renders directly to the screen like before, while people with more powerful cards can run a Compositor that implements deferred rendering, for example.
 
* It doesn't increase the hardware requirements, it expands the hardware range FG can run on. People with integrated GPUs (Intel HD etc) can run a Compositor with a single pass that renders directly to the screen like before, while people with more powerful cards can run a Compositor that implements deferred rendering, for example.
 +
* Static branching support. Every pipeline element can be enabled/disabled at startup via a <condition> block.
  
Unlike Rembrandt, the Compositor makes use of scene graph cameras
+
== Elements ==
instead of viewer level cameras.
+
  
This allows CameraGroup to manage windows, near/far cameras and other
+
=== Buffers ===
slaves without interfering with what is being rendered
+
(post-processing, shadows...).
+
  
The Compositor is in an usable state right now: it works but there are
+
A buffer represents a texture or, more generically, a region of GPU memory. It can have the following parameters:
no effects or pipelines developed for it. There are also some bugs and
+
features that don't work as expected because of some hardcoded
+
assumptions in the FlightGear Viewer code. Still, I think it's time to
+
announce it so people much more knowledgeable than me can point me in
+
the right direction to get this upstream and warn me about possible
+
issues and worries. :)
+
  
A screenshot showing post-processed gaussian blur and a blue filter:
+
; name
https://i.imgur.com/Zzxre8a.png
+
: Passes will be able to address the buffer by this name.
 +
; type
 +
: Any texture type allowed by OpenGL: 1d, 2d, 2d-array, 2d-multisample, 3d, rect or cubemap.
 +
; width
 +
: Texture width. It's possible to write 'screen' to use the physical viewport width.
 +
; screen-width-scale
 +
: If 'screen' was used, this controls the width scaling factor.
 +
; height
 +
: Texture height. It's possible to write 'screen' to use the physical viewport height.
 +
; screen-height-scale
 +
: If 'screen' was used, this controls the height scaling factor.
 +
; depth
 +
: Texture depth.
 +
; internal-format, source-format and source-type
 +
: They specify the texture format. They correspond to the arguments ''internalformat'', ''format'' and ''type'' respectively of the OpenGL function ''glTexImage2D''.
 +
; min-filter and mag-filter (Optional)
 +
: They change the minification and magnification filtering respectively. Possible values are: linear, linear-mipmap-linear, linear-mipmap-nearest, nearest, nearest-mipmap-linear and nearest-mipmap-nearest. The default value for both filters is linear.
 +
; wrap-s, wrap-t and wrap-r (Optional)
 +
: They change the wrap mode for each coordinate. Possible values are: clamp, clamp-to-edge, clamp-to-border, repeat and mirror. The default value for every coordinate is clamp-to-border.
 +
; condition
 +
: A valid boolean condition to enable the buffer at startup (doesn't work at runtime).
  
The next major (and hopefully final) iteration of the Compositor is
+
A typical [[PropertyList XML File|property tree structure]] describing a buffer may be as follows:
mostly done now, just some more cleanup is needed. I've tried to keep
+
the changes contained in CameraGroup.cxx/hxx and in new files in
+
SimGear to ease the merging process.
+
  
== Use Cases ==
+
<syntaxhighlight lang="xml">
* [[Procedural Texturing]]
+
<buffer>
* [[Shuttle ADI ball]]
+
  <name>color</name>
 +
  <type>2d</type>
 +
  <width>screen</width>
 +
  <height>screen</height>
 +
  <screen-width-scale>1.5</screen-width-scale>
 +
  <screen-height-scale>1.5</screen-height-scale>
 +
  <internal-format>rgba8</internal-format>
 +
  <source-format>rgba</source-format>
 +
  <source-type>ubyte</source-type>
 +
  <min-filter>nearest-mipmap-nearest</min-filter>
 +
  <mag-filter>nearest-mipmap-nearest</mag-filter>
 +
  <wrap-s>repeat</wrap-s>
 +
  <wrap-t>repeat</wrap-t>
 +
</buffer>
 +
</syntaxhighlight>
  
== FAQs ==
+
=== Passes ===
I started the Compositor to hopefully unify rendering in FG in the most
+
non-radical way. Instead of changing a large part of the codebase like
+
Rembrandt, the Compositor aims to bring the same funcionality and slowly
+
deprecate parts of the FG Viewer code.
+
  
=== Shadows ===
+
A pass wraps around an [http://public.vrac.iastate.edu/vancegroup/docs/OpenSceneGraphReferenceDocs-3.0/a00089.html osg::Camera]. Passes all have some common parameters:
It includes shadows indeed. The Compositor stacks several passes of
+
different types on top of each other, and exchanges buffers between
+
them. Nothing prevents someone from creating a new pass type that
+
implements a shadow map pre-pass. I've already tried myself
+
successfully, but the problem is telling the pass which light should be
+
used, which brings me to spotlights.
+
  
=== Light Sources ===
+
; clear-color, clear-accum, clear-depth and clear-stencil
Allowing aircraft/scenery developers to include "real" lights is a whole
+
: Default values are black, black, 1.0 and 0 respectively.
different aspect of the scene graph that shouldn't be managed by the
+
; clear-color-bit, clear-accum-bit, clear-depth-bit and clear-stencil-bit
Compositor. Some kind of system that exposes OpenGL lights should be
+
: Default values are true, false, true, false, respectively.
implemented and linked to the Compositor. Some decisions have to be made
+
; effect-scheme
though. In a forward renderer lights aren't as cheap as in a deferred
+
: The pass will try to use the specified effect scheme to draw every object.
renderer, so if the aircraft developer says there has to be a light
+
; condition
there, someone using a forward renderer might see a performance drop or
+
: A valid boolean condition to enable the pass at startup (doesn't work at runtime).
not see them at all.
+
  
== Elements ==
+
Passes can render to a buffer (Render to Texture), to several buffers (Multiple Render Targets) or directly to the framebuffer. This is accomplished by the <code><attachment/></code> tag. Possible parameters of an attachment are:
  
=== Buffers ===
+
; buffer
 +
: The name of the buffer to output to.
 +
; component
 +
: FBO attachment point. Possible values are color0 to color15, depth, stencil and depth-stencil.
 +
; level (Optional)
 +
: Controls the mip map level of the texture that is attached. Default value is 0.
 +
; face (Optional)
 +
: Controls the face of texture cube map or z level of 3d texture. Default value is 0.
 +
; mipmap-generation (Optional)
 +
: Controls whether mipmap generation should be done for texture. Default value is false.
 +
; multisample-samples (Optional)
 +
: MSAA samples. Default value is 0.
 +
; multisample-color-samples (Optional)
 +
: MSAA color samples. Default value is 0.
 +
; condition
 +
: A valid boolean condition to enable the attachment at startup (doesn't work at runtime).
  
A buffer represents a texture or, more generically, a region of GPU memory. Textures can be of any type allowed by OpenGL: 1D, 2D, rectangle, 2Darray, 3D or cubemap.
+
Passes can also receive buffers as input and use them in their shaders. This is accomplished by the <code><binding/></code> tag, which has the following parameters:
  
A typical [[PropertyList XML File|property tree structure]] describing a buffer may be as follows:
+
; buffer
 +
: The name of the buffer to bind.
 +
; unit
 +
: The texture unit to place the texture on. Effects will be able to access the buffer on this texture unit.
 +
; condition
 +
: A valid boolean condition to enable the binding at startup (doesn't work at runtime).
 +
 
 +
There are specific pass types, each with their own set of custom parameters:
 +
 
 +
==== scene ====
 +
Renders the scene from the point of view given by the CameraGroup.
 +
; cull-mask
 +
: A 32 bit number that specifies the cull mask to be used. See <tt>simgear/scene/util/RenderConstants.hxx</tt> to know which bits enable what.
 +
; z-near and z-far
 +
: They change the depth range to be used. If both of them are zero, the default Z range in the CameraGroup is used.
 +
; clustered-forward
 +
: Enables the use of clustered forward rendering for this pass.
 +
; cubemap-face
 +
: Ignores the given view and projection matrices and uses a custom one that renders the scene as if it was seen from inside a cubemap looking towards the specified face.
 +
 
 +
==== quad ====
 +
Renders a fullscreen quad with an optional [[Effects|effect]] applied. Useful for screen space shaders (like SSAO, Screen Space Reflections or bloom) and deferred rendering.
 +
; geometry
 +
: Specifies the x, y, width and height of the fullscreen quad inside the viewport using normalized coordinates.
 +
; effect
 +
: The quad will use this effect.
 +
 
 +
==== shadow-map ====
 +
Renders the scene from a light's point of view.
 +
; light-num
 +
: The OpenGL light number to use for this shadow map.
 +
; near-m and far-m
 +
: They specify the range of the shadow map.
 +
 
 +
Example XML for a ''scene'' type pass:
  
 
<syntaxhighlight lang="xml">
 
<syntaxhighlight lang="xml">
      <buffer>
+
<pass>
            <name>buffer-name</name>
+
  <name>forward-lighting</name>
    <type>2d</type>
+
  <type>scene</type>
    <width>512</width>
+
  <clear-color type="vec4d">0 0 0 0</clear-color>
    <height>512</height>
+
 
    <scale-factor>1.0</scale-factor>
+
  <clustered-forward/>
    <internal-format>rgba8</internal-format>
+
 
    <source-format>rgba</source-format>
+
  <binding>
            <source-type>ubyte</source-type>
+
    <buffer>shadowmap</buffer>
      </buffer>
+
    <unit>10</unit>
 +
  </binding>
 +
  <attachment>
 +
    <buffer>color</buffer>
 +
    <component>color0</component>
 +
    <multisample-samples>4</multisample-samples>
 +
    <multisample-color-samples>4</multisample-color-samples>
 +
  </attachment>
 +
  <attachment>
 +
    <buffer>depth</buffer>
 +
    <component>depth</component>
 +
  </attachment>
 +
</pass>
 
</syntaxhighlight>
 
</syntaxhighlight>
  
=== Passes ===
+
== Pipelines ==
  
A pass wraps around an [http://public.vrac.iastate.edu/vancegroup/docs/OpenSceneGraphReferenceDocs-3.0/a00089.html osg::Camera]. As of February 2018, there are two types of passes supported:
+
=== Low-Spec pipeline ===
  
* '''scene'''. Renders a scene. The scene can be an already loaded scene graph node (terrain, aircraft etc.) or a path to a custom 3D model.
+
A fixed function forward rendering pipeline mainly targeted to low spec systems. It imitates the classic forward pipeline used before multi-pass rendering was introduced by using two near/far cameras rendering directly to the screen.
* '''quad'''. Renders a fullscreen quad with an optional [[Effects|effect]] applied. Useful for screen space shaders (like SSAO, Screen Space Reflections or bloom) and deferred rendering.
+
  
Passes can render to a buffer (Render to Texture), to several buffers (Multiple Render Targets) or directly to the OSG context. This allows chaining of multiple passes, sharing buffers between them. They can also receive other buffers as input, so effects can access them as normal textures.
+
[[File:Low-spec Compositor pipeline.png|thumb|Screenshot showing OSG stats of the Compositor-based low-spec rendering pipeline.]]
  
Example XML for a ''quad'' type pass:
+
=== ALS ===
  
<syntaxhighlight lang="xml">
+
The ALS pipeline tries to bring multipass rendering to the current ALS framework, effectively combining the best from ALS and Project Rembrandt.
      <pass-quad>
+
          <name>pass-name</name>
+
  
          <effect>Effects/test</effect>
+
==== Cascaded shadow mapping ====
   
+
          <input-buffer>
+
              <buffer>some-buffer</buffer>
+
              <unit>0</unit>
+
          </input-buffer>
+
  
          <output-buffer>
+
The main issue with shadow mapping in FlightGear is the complexity of the scene graph. Culling times can become huge if we don't carefully select which parts of the scene graph we want to render in the shadow maps. Some possible optimizations:
              <buffer>color-buffer</buffer>
+
* Study the minimum shadow map distance we can get without noticeable light leaking. Select an appropiate amount of cascades (more cascades = more passes over all geometry, and in general we want to keep the amount of forward passes to a minimum). We should have at least three cascades: the first just for cockpit/internal shadows, the second for the whole aircraft and the third for the rest of the scenery geometry. A fourth can be added if the transition between the second and the third is too harsh.
              <component>color</component>
+
* Improve the culling masks (simgear/scene/util/RenderConstants.hxx). The CASTSHADOW_BIT flag is present in almost every object in the scene graph. Turning this flag off for trees, random buildings and other geometry intensive objects improves framerates by a very considerable amount. Should the user be able to select which objects cast shadows?
          </output-buffer>
+
* Should the terrain cast shadows? The terrain is rarely steep enough to cast shadows. Apart from that, the terrain in FlightGear messes with automatic near/far computations for the shadow passes since the geometry is not tessellated enough. Also, the terrain LOD is not good enough to have decent cull times at far cascades.
          <output-buffer>
+
* Adding a "internal only" shadow flag for aircraft developers. This allows farther shadow cascades to cull complex objects that are only visible in the nearest cascades. (Very important optimization for aircrafts with complex cockpit geometry).
              <buffer>depth-buffer</buffer>
+
              <component>depth</component>
+
          </output-buffer>
+
      </pass-quad>
+
</syntaxhighlight>
+
  
== Canvas integration ==
+
==== Post-processing ====
  
[[File:Blur_and_blue_filter_applied_to_a_buffer.png|thumb|Post processed image (right) after applying gaussian blur and a blue filter to a scene pass (left).]]
+
Gamma correction, night vision and other ALS filters should happen in a quad pass. The current filter_combined() should be left for post-processing that requires as much precision as possible - e.g. dithering to prevent banding). HDR is not a planned feature for now so ALS will be using rgba8 buffers for most of its features.
  
Apart from serving as a debugging tool for visualizing the contents of a buffer, integrating the Compositor with [[Canvas]] allows aircraft developers to access RTT capabilities. Compositor buffers can be accessed within Canvas via a new custom Canvas Image protocol '''buffer://'''. For example, the path <code>buffer://test-compositor/test-buffer</code> displays the buffer test-buffer declared in test-compositor.
+
==== Real-time dynamic reflections ====
  
<syntaxhighlight lang="nasal">
+
Rendering dynamically to a cubemap is possible. As with shadow mapping, minimizing the object count and number of forward passes is vital to get good performance in FlightGear. Rendering to six cubemap faces requires six forward passes, but we can render to a dual paraboloid map instead, reducing this number to two.
var (width,height) = (612,612);
+
 
var title = 'Compositor&Canvas test';
+
==== Transparency ====
var window = canvas.Window.new([width,height],"dialog")
+
 
.set('title',title);
+
When shadows (and multipass rendering in general) come into play, transparent objects have to be treated differently, even when we are dealing with a forward renderer. In OSG there are two ways to separate transparent surfaces:
var myCanvas = window.createCanvas().set("background", canvas.style.getColor("bg_color"));
+
 
var root = myCanvas.createGroup();
+
* Using RenderBins. After a single scene cull traversal, surfaces which belong to a special RenderBin type (DepthSortedBin) are removed or moved to another camera. This is how Rembrandt does it and it is the most backwards compatible approach since RenderBins can be changed directly inside Effects.
var path = "buffer://test-compositor/test-buffer";
+
* Using cull masks. Two separate traversals are done: one for opaque objects and another for translucent objects. This requires offering aircraft developers another way of tagging a surface as transparent. A trivial approach would be to add a new <animation> type called 'transparent', but that wouldn't be backwards compatible. Maybe we can add some kind of system where we can change cull masks inside Effects? Would that be too hacky or out of place?
var child = root.createChild("image")
+
 
    .setTranslation(50, 50)
+
== TODO ==
    .setSize(512, 512)
+
 
    .setFile(path);
+
* Bring back distortion correction.
</syntaxhighlight>
+
* Add spotlight support to clustered shading.
 +
* Some kind of versioning system to be able to make breaking changes in the future if/when the compositor is updated in any significant way, without people having to manually update their configs.
 +
* Bring back Canvas integration so aircraft devs have access to the rendering pipeline. This allows to render exterior views in cockpit displays etc.
  
 
== Known Issues ==
 
== Known Issues ==
  
* Canvas (sc::Image) doesn't update its texture unless .update() is invoked in Nasal. Is this a feature?
+
* Setting a buffer scale factor different from 1.0 and rendering to it might not scale the splash screen correctly.
* The scene graph and the viewer are still managed by FGRenderer. Maybe hijacking the init sequence is a good idea?
+
* light sources ?
+
  
 
== Related ==
 
== Related ==
 +
=== Articles ===
 +
* [[Uniform Buffer Objects]]
 +
* [[CompositeViewer Support]]
 +
* [[FlightGear CIGI Support (Common Image Generator Interface)]]
 +
=== Discussions ===
 +
* [https://forum.flightgear.org/viewtopic.php?f=47&t=35095 Clustered Forward Rendering] (12/2018)
 +
* [https://forum.flightgear.org/viewtopic.php?f=47&t=33045 Getting started with RTT]
 +
* [https://forum.flightgear.org/viewtopic.php?f=47&t=32846 Canvas::View Development]
 +
* [https://forum.flightgear.org/viewtopic.php?f=71&t=23929 Gear View in Cockpit]
 +
* [https://forum.flightgear.org/viewtopic.php?f=37&t=20057 Rear-View mirror]
 +
* [https://forum.flightgear.org/viewtopic.php?f=30&t=18905 Progress on synthetic terrain]
 +
* [https://forum.flightgear.org/viewtopic.php?f=71&t=17184 Instruments with heightmaps]
 +
 +
 
{{Appendix}}
 
{{Appendix}}
 +
 +
[[Category:Core development projects]]

Latest revision as of 16:19, 13 June 2019

This article describes content/features that may not yet be available in the latest stable version of FlightGear (2018.3).
You may need to install some extra components, use the latest development (Git) version or even rebuild FlightGear from source, possibly from a custom topic branch using special build settings: -DENABLE_COMPOSITOR.

This feature is scheduled for FlightGear 2019.2. 100}% completed

If you'd like to learn more about getting your own ideas into FlightGear, check out Implementing new features for FlightGear.


Compositor Subsystem
ALS Compositor pipeline.jpg
Started in 01/2018 (Available since FlightGear 2019.2)
Description Dynamic rendering pipeline configured via the property tree and XML
Contributor(s) Fernando García Liñán
Status Stable
Folders

The Compositor aims to bring multi-pass rendering to FlightGear. It encapsulates a rendering pipeline and exposes its parameters to a Property Tree interface. At startup, FlightGear reads the pipeline definition file for each physical viewport defined on the CameraGroup settings. If no Compositor file is specified for a physical camera, the one given by the --compositor= startup command will be used. If such startup option is not used either, FlightGear will look for a valid Compositor file in $FG_ROOT/Compositor/default.xml

The Compositor introduces a new dedicated fgdata directory for new/custom rendering pipelines: fgdata/Compositor.

Background

First discussed in 03/2012 during the early Rembrandt days, Zan (Lauri Peltonen) came up with a set of patches demonstrating how to create an XML-configurable rendering pipeline.

Back then, this work was considered to look pretty promising [1] and at the time plans were discussed to unify this with the ongoing Rembrandt implementation (no longer maintained).

Adopting Zan's approach would have meant that efforts like Rembrandt (deferred rendering) could have been implemented without requiring C++ space modifications, i.e. purely in Base package space.

Rembrandt's developer (FredB) suggested to extend the format to avoid duplicating the stages when you have more than one viewport, i.e. specifying a pipeline as a template, with conditions like in effects, and have the current camera layout refer the pipeline that would be duplicated, resized and positioned for each declared viewport [2]

Zan's original patches can still be found in his newcameras branches which allow the user to define the rendering pipeline in preferences.xml:

At that point, it didn't have everything Rembrandt's pipeline needs, but most likely could be easily enhanced to support those things.

Basically, the original version added support for multiple camera passes, texture targets, texture formats, passing textures from one pass to another etc, while preserving the standard rendering line if user wants that. [3]

Since the early days of Zan's groundwork, providing the (hooks) infrastructure to enable base package developers to prototype, test and develop distinct rendering pipelines without requiring C++ space modifications has been a long-standing idea, especially after the Canvas system became available in early 2012, which demonstrated how RTT-rendering buffers (FBOs) could be set up, created and manipulated procedurally (i.e. at run-time) using XML, the property tree and Nasal scripting. [4]

The new Compositor is an improved re-implementation of Zan's original work using not just XML, but also properties and a handful of Canvas concepts.

Status

06/2019

The Compositor core is complete and stable. Effects and shaders still need to be adapted to the Compositor.

Features

  • Completely independent of other parts of the simulator, i.e. it's part of SimGear and can be used in a standalone fashion if needed, ala Canvas.
  • Although independent, its aim is to be fully compatible with the current rendering framework in FG. This includes the Effects system, CameraGroup, Rembrandt and ALS (and obviously the Canvas).
  • Its functionality overlaps Rembrandt: what can be done with Rembrandt can be done with the Compositor, but not vice versa.
  • Fully configurable via an XML interface without compromising performance (ala Effects, using PropertyList files).
  • Flexible, expandable and compatible with modern graphics.
  • It doesn't increase the hardware requirements, it expands the hardware range FG can run on. People with integrated GPUs (Intel HD etc) can run a Compositor with a single pass that renders directly to the screen like before, while people with more powerful cards can run a Compositor that implements deferred rendering, for example.
  • Static branching support. Every pipeline element can be enabled/disabled at startup via a <condition> block.

Elements

Buffers

A buffer represents a texture or, more generically, a region of GPU memory. It can have the following parameters:

name
Passes will be able to address the buffer by this name.
type
Any texture type allowed by OpenGL: 1d, 2d, 2d-array, 2d-multisample, 3d, rect or cubemap.
width
Texture width. It's possible to write 'screen' to use the physical viewport width.
screen-width-scale
If 'screen' was used, this controls the width scaling factor.
height
Texture height. It's possible to write 'screen' to use the physical viewport height.
screen-height-scale
If 'screen' was used, this controls the height scaling factor.
depth
Texture depth.
internal-format, source-format and source-type
They specify the texture format. They correspond to the arguments internalformat, format and type respectively of the OpenGL function glTexImage2D.
min-filter and mag-filter (Optional)
They change the minification and magnification filtering respectively. Possible values are: linear, linear-mipmap-linear, linear-mipmap-nearest, nearest, nearest-mipmap-linear and nearest-mipmap-nearest. The default value for both filters is linear.
wrap-s, wrap-t and wrap-r (Optional)
They change the wrap mode for each coordinate. Possible values are: clamp, clamp-to-edge, clamp-to-border, repeat and mirror. The default value for every coordinate is clamp-to-border.
condition
A valid boolean condition to enable the buffer at startup (doesn't work at runtime).

A typical property tree structure describing a buffer may be as follows:

<buffer>
  <name>color</name>
  <type>2d</type>
  <width>screen</width>
  <height>screen</height>
  <screen-width-scale>1.5</screen-width-scale>
  <screen-height-scale>1.5</screen-height-scale>
  <internal-format>rgba8</internal-format>
  <source-format>rgba</source-format>
  <source-type>ubyte</source-type>
  <min-filter>nearest-mipmap-nearest</min-filter>
  <mag-filter>nearest-mipmap-nearest</mag-filter> 
  <wrap-s>repeat</wrap-s>
  <wrap-t>repeat</wrap-t>
</buffer>

Passes

A pass wraps around an osg::Camera. Passes all have some common parameters:

clear-color, clear-accum, clear-depth and clear-stencil
Default values are black, black, 1.0 and 0 respectively.
clear-color-bit, clear-accum-bit, clear-depth-bit and clear-stencil-bit
Default values are true, false, true, false, respectively.
effect-scheme
The pass will try to use the specified effect scheme to draw every object.
condition
A valid boolean condition to enable the pass at startup (doesn't work at runtime).

Passes can render to a buffer (Render to Texture), to several buffers (Multiple Render Targets) or directly to the framebuffer. This is accomplished by the <attachment/> tag. Possible parameters of an attachment are:

buffer
The name of the buffer to output to.
component
FBO attachment point. Possible values are color0 to color15, depth, stencil and depth-stencil.
level (Optional)
Controls the mip map level of the texture that is attached. Default value is 0.
face (Optional)
Controls the face of texture cube map or z level of 3d texture. Default value is 0.
mipmap-generation (Optional)
Controls whether mipmap generation should be done for texture. Default value is false.
multisample-samples (Optional)
MSAA samples. Default value is 0.
multisample-color-samples (Optional)
MSAA color samples. Default value is 0.
condition
A valid boolean condition to enable the attachment at startup (doesn't work at runtime).

Passes can also receive buffers as input and use them in their shaders. This is accomplished by the <binding/> tag, which has the following parameters:

buffer
The name of the buffer to bind.
unit
The texture unit to place the texture on. Effects will be able to access the buffer on this texture unit.
condition
A valid boolean condition to enable the binding at startup (doesn't work at runtime).

There are specific pass types, each with their own set of custom parameters:

scene

Renders the scene from the point of view given by the CameraGroup.

cull-mask
A 32 bit number that specifies the cull mask to be used. See simgear/scene/util/RenderConstants.hxx to know which bits enable what.
z-near and z-far
They change the depth range to be used. If both of them are zero, the default Z range in the CameraGroup is used.
clustered-forward
Enables the use of clustered forward rendering for this pass.
cubemap-face
Ignores the given view and projection matrices and uses a custom one that renders the scene as if it was seen from inside a cubemap looking towards the specified face.

quad

Renders a fullscreen quad with an optional effect applied. Useful for screen space shaders (like SSAO, Screen Space Reflections or bloom) and deferred rendering.

geometry
Specifies the x, y, width and height of the fullscreen quad inside the viewport using normalized coordinates.
effect
The quad will use this effect.

shadow-map

Renders the scene from a light's point of view.

light-num
The OpenGL light number to use for this shadow map.
near-m and far-m
They specify the range of the shadow map.

Example XML for a scene type pass:

<pass>
  <name>forward-lighting</name>
  <type>scene</type>
  <clear-color type="vec4d">0 0 0 0</clear-color>

  <clustered-forward/>

  <binding>
    <buffer>shadowmap</buffer>
    <unit>10</unit>
  </binding>
  <attachment>
    <buffer>color</buffer>
    <component>color0</component>
    <multisample-samples>4</multisample-samples>
    <multisample-color-samples>4</multisample-color-samples>
  </attachment>
  <attachment>
    <buffer>depth</buffer>
    <component>depth</component>
  </attachment>
</pass>

Pipelines

Low-Spec pipeline

A fixed function forward rendering pipeline mainly targeted to low spec systems. It imitates the classic forward pipeline used before multi-pass rendering was introduced by using two near/far cameras rendering directly to the screen.

Screenshot showing OSG stats of the Compositor-based low-spec rendering pipeline.

ALS

The ALS pipeline tries to bring multipass rendering to the current ALS framework, effectively combining the best from ALS and Project Rembrandt.

Cascaded shadow mapping

The main issue with shadow mapping in FlightGear is the complexity of the scene graph. Culling times can become huge if we don't carefully select which parts of the scene graph we want to render in the shadow maps. Some possible optimizations:

  • Study the minimum shadow map distance we can get without noticeable light leaking. Select an appropiate amount of cascades (more cascades = more passes over all geometry, and in general we want to keep the amount of forward passes to a minimum). We should have at least three cascades: the first just for cockpit/internal shadows, the second for the whole aircraft and the third for the rest of the scenery geometry. A fourth can be added if the transition between the second and the third is too harsh.
  • Improve the culling masks (simgear/scene/util/RenderConstants.hxx). The CASTSHADOW_BIT flag is present in almost every object in the scene graph. Turning this flag off for trees, random buildings and other geometry intensive objects improves framerates by a very considerable amount. Should the user be able to select which objects cast shadows?
  • Should the terrain cast shadows? The terrain is rarely steep enough to cast shadows. Apart from that, the terrain in FlightGear messes with automatic near/far computations for the shadow passes since the geometry is not tessellated enough. Also, the terrain LOD is not good enough to have decent cull times at far cascades.
  • Adding a "internal only" shadow flag for aircraft developers. This allows farther shadow cascades to cull complex objects that are only visible in the nearest cascades. (Very important optimization for aircrafts with complex cockpit geometry).

Post-processing

Gamma correction, night vision and other ALS filters should happen in a quad pass. The current filter_combined() should be left for post-processing that requires as much precision as possible - e.g. dithering to prevent banding). HDR is not a planned feature for now so ALS will be using rgba8 buffers for most of its features.

Real-time dynamic reflections

Rendering dynamically to a cubemap is possible. As with shadow mapping, minimizing the object count and number of forward passes is vital to get good performance in FlightGear. Rendering to six cubemap faces requires six forward passes, but we can render to a dual paraboloid map instead, reducing this number to two.

Transparency

When shadows (and multipass rendering in general) come into play, transparent objects have to be treated differently, even when we are dealing with a forward renderer. In OSG there are two ways to separate transparent surfaces:

  • Using RenderBins. After a single scene cull traversal, surfaces which belong to a special RenderBin type (DepthSortedBin) are removed or moved to another camera. This is how Rembrandt does it and it is the most backwards compatible approach since RenderBins can be changed directly inside Effects.
  • Using cull masks. Two separate traversals are done: one for opaque objects and another for translucent objects. This requires offering aircraft developers another way of tagging a surface as transparent. A trivial approach would be to add a new <animation> type called 'transparent', but that wouldn't be backwards compatible. Maybe we can add some kind of system where we can change cull masks inside Effects? Would that be too hacky or out of place?

TODO

  • Bring back distortion correction.
  • Add spotlight support to clustered shading.
  • Some kind of versioning system to be able to make breaking changes in the future if/when the compositor is updated in any significant way, without people having to manually update their configs.
  • Bring back Canvas integration so aircraft devs have access to the rendering pipeline. This allows to render exterior views in cockpit displays etc.

Known Issues

  • Setting a buffer scale factor different from 1.0 and rendering to it might not scale the splash screen correctly.

Related

Articles

Discussions


References