Difference between revisions of "Compositor"

From FlightGear wiki
Jump to: navigation, search
 
(189 intermediate revisions by 9 users not shown)
Line 1: Line 1:
 
+
{{forum|47|Effects & Shaders}}
{{Template:Non-stable|version=2019.3|progress=60}}
+
 
+
  
 
{{infobox subsystem
 
{{infobox subsystem
|image      = Canvas-view-element-prototype-by-icecode gl.png
+
|image      = ALS Compositor pipeline.jpg
|name        = Compositor Subsystem
+
|name        = Compositor Framework
|started    = 01/2018
+
|started    = 01/2018 (Available since FlightGear 2019.2)
|description = Dynamic rendering pipeline configured via the property tree
+
|description = Dynamic rendering pipeline configured via the [[Property tree]] and [[PropertyList XML File|XML]]
|status      = * experimental as of 02/2018
+
|status      = Stable (merged and actively maintained)
* put up for review/discussion as of 08/2018
+
|developers  = Fernando García Liñán <ref>https://sourceforge.net/u/fgarlin/profile/</ref>
|maintainers = none
+
|changelog = https://sourceforge.net/u/fgarlin/profile/feed.rss
|developers  = Icecode
+
|folders =  
|topic-sg = https://sourceforge.net/u/fgarlin/simgear/ci/new-compositor/tree/
+
* {{flightgear file|src/Viewer}}
|topic-fg = https://sourceforge.net/u/fgarlin/flightgear-src/ci/new-compositor/tree/
+
* {{simgear file|simgear/scene/viewer}}
|topic-fgdata = https://sourceforge.net/u/fgarlin/flightgear/ci/new-compositor/tree/
+
* {{fgdata file|Compositor}}
 
}}
 
}}
  
A '''Compositor''' is a rendering pipeline configured by the [[Property Tree]]. Every configurable element of the pipeline wraps an OpenGL/OSG object and exposes its parameters to the property tree. This way, the rendering pipeline becomes dynamically-reconfigurable at runtime.
 
  
Related efforts: [[Howto:Canvas View Camera Element]]
+
 
 +
The '''Compositor''' aims to bring multi-pass rendering to FlightGear. It encapsulates a rendering pipeline and exposes its parameters to a [[Property Tree]] interface. At startup, FlightGear reads the pipeline definition file for each physical viewport defined on the [[Howto:Configure camera view windows|CameraGroup settings]]. If no Compositor file is specified for a physical camera, the one given by the <code>--compositor=</code> startup command will be used. If such startup option is not used either, FlightGear will look for a valid Compositor file in $FG_ROOT/Compositor/default.xml
 +
 
 +
The Compositor introduces a new dedicated fgdata directory for new/custom rendering pipelines: {{Fgdata file|Compositor}}.
 +
 
 +
The framework is now included in the latest stable release ([[Changelog_2020.1|FlightGear v2020.1]]) as a separate binary. Developers are encouraged to test the Compositor with their personal projects and report any breakages. Also keep in mind that some features might not work/look as intended - see [[#Known Issues|Known Issues]].
  
 
== Background ==
 
== Background ==
{{See also|Supporting multiple renderers}}
+
{{See also|Supporting multiple renderers|Howto:Canvas View Camera Element}}
  
First discussed in 03/2012 during the early [[Rembrandt]] days, Zan came up with patches demonstrating how to create an XML-configurable rendering pipeline.
+
First discussed in 03/2012 during the early [[Rembrandt]] days, Zan (Lauri Peltonen) came up with a set of patches demonstrating how to create an XML-configurable rendering pipeline.
  
 
Back then, this work was considered to look pretty promising <ref>{{cite web
 
Back then, this work was considered to look pretty promising <ref>{{cite web
Line 35: Line 37:
 
   }}</ref> and at the time plans were discussed to unify this with the ongoing Rembrandt implementation (no longer maintained).
 
   }}</ref> and at the time plans were discussed to unify this with the ongoing Rembrandt implementation (no longer maintained).
  
Adopting Zan's approach would have meant that efforts like Rembrandt could have been implemented without requiring C++ space modifications.
+
Adopting Zan's approach would have meant that efforts like [[Rembrandt]] (deferred rendering) could have been implemented without requiring C++ space modifications, i.e. purely in [[Base package]] space.
  
 
Rembrandt's developer (FredB) suggested to extend the format to avoid duplicating the stages when you have more than one viewport, i.e.  specifying a pipeline as a template, with conditions like in effects, and have the current camera layout refer the pipeline that would be duplicated, resized and positioned for each declared viewport <ref>{{cite web
 
Rembrandt's developer (FredB) suggested to extend the format to avoid duplicating the stages when you have more than one viewport, i.e.  specifying a pipeline as a template, with conditions like in effects, and have the current camera layout refer the pipeline that would be duplicated, resized and positioned for each declared viewport <ref>{{cite web
Line 46: Line 48:
 
   }}</ref>
 
   }}</ref>
  
 
+
Zan's original patches can still be found in his newcameras branches which allow the user to define the rendering pipeline in preferences.xml: {{gitorious source|proj=fg|repo=zans-flightgear|branch=newcameras|text=FlightGear}}, {{gitorious source|proj=fg|repo=zans-simgear|branch=newcameras|text=SimGear}}.
Zan's original patches can still be found in his newcameras branches which allow the user to define the rendering pipeline in preferences.xml:
+
 
+
* [https://gitorious.org/fg/zans-flightgear?p=fg:zans-flightgear.git;a=shortlog;h=refs/heads/newcameras FlightGear]
+
* [https://gitorious.org/fg/zans-simgear?p=fg:zans-simgear.git;a=log;h=refs/heads/newcameras SimGear]
+
  
 
At that point, it didn't have everything Rembrandt's pipeline needs, but most likely could be easily enhanced to support those things.  
 
At that point, it didn't have everything Rembrandt's pipeline needs, but most likely could be easily enhanced to support those things.  
  
Basically the original version added support for multiple camera passes, texture targets, texture formats, passing textures from one pass to another etc, while preserving the standard rendering line if user wants that. <ref>{{cite web
+
Basically, the original version added support for multiple camera passes, texture targets, texture formats, passing textures from one pass to another etc, while preserving the standard rendering line if user wants that. <ref>{{cite web
 
   |url    =  https://sourceforge.net/p/flightgear/mailman/message/28944733/  
 
   |url    =  https://sourceforge.net/p/flightgear/mailman/message/28944733/  
 
   |title  =  <nowiki> [Flightgear-devel] [Rembrandt] the plan </nowiki>  
 
   |title  =  <nowiki> [Flightgear-devel] [Rembrandt] the plan </nowiki>  
Line 63: Line 61:
 
   }}</ref>
 
   }}</ref>
  
In 02/2018, Icecode GL re-implemented a more generic version of Zan's idea using Canvas concepts, which means the rendering pipeline is not just xml-configurable, but using listeners to be dynamically-reconfigurable at runtime via properties.
+
Since the early days of Zan's groundwork, providing the (hooks) infrastructure to enable base package developers to prototype, test and develop distinct rendering pipelines without requiring C++ space modifications has been a long-standing idea, especially after the [[Canvas]] system became available in early 2012, which demonstrated how RTT-rendering buffers (FBOs) could be set up, created and manipulated procedurally (i.e. at run-time) using XML, the property tree and [[Nasal]] scripting. <ref>{{forum link|type=search|title=Zan's Rembrandt and Canvas work|keywords=zan+rembrandt+canvas}}</ref>
  
The corresponding set of patches (topic branches) were put up for review in 08/2018 to discuss the underlying approach and hopefully get this merged in 2019.
+
The new '''Compositor''' is an improved re-implementation of Zan's original work using not just XML, but also [[Property Tree|properties]] and a handful of [[Canvas]] concepts.
  
== Status ==
+
== Features ==
'''08/2018'''
+
  
Fernando has been working on and off on multi-pass rendering support for
+
* Completely independent of other parts of the simulator, i.e. it's part of [[SimGear]] and can be used in a standalone fashion if needed, ala Canvas.
FlightGear since late 2017. It went through several iterations and
+
* Although independent, its aim is to be fully compatible with the current rendering framework in FG. This includes the [[Effects]] system, [[Howto:Configure camera view windows|CameraGroup]], [[Rembrandt]] and [[ALS]] (and obviously the [[Canvas]]).
design changes, but he think it's finally on the right track. It's
+
heavily based on the Ogre3D Compositor and inspired by many
+
data-driven rendering pipelines. Its features include:
+
 
+
* Completely independent of other parts of the simulator, i.e. it's part of SimGear and can be used in a standalone fashion if needed, ala Canvas.
+
* Although independent, its aim is to be fully compatible with the current rendering framework in FG. This includes the Effects system, CameraGroup, Rembrandt and ALS (and obviously the Canvas).
+
 
* Its functionality overlaps Rembrandt: what can be done with Rembrandt can be done with the Compositor, but not vice versa.
 
* Its functionality overlaps Rembrandt: what can be done with Rembrandt can be done with the Compositor, but not vice versa.
* Fully configurable via a XML interface without compromising performance (ala Effects, using PropertyList files).
+
* Fully configurable via an XML interface without compromising performance (ala Effects, using [[PropertyList XML File|PropertyList files]]).
* Optional at compile time to aid merge request efforts.
+
 
* Flexible, expandable and compatible with modern graphics.
 
* Flexible, expandable and compatible with modern graphics.
 
* It doesn't increase the hardware requirements, it expands the hardware range FG can run on. People with integrated GPUs (Intel HD etc) can run a Compositor with a single pass that renders directly to the screen like before, while people with more powerful cards can run a Compositor that implements deferred rendering, for example.
 
* It doesn't increase the hardware requirements, it expands the hardware range FG can run on. People with integrated GPUs (Intel HD etc) can run a Compositor with a single pass that renders directly to the screen like before, while people with more powerful cards can run a Compositor that implements deferred rendering, for example.
 +
* Static branching support. Every pipeline element can be enabled/disabled at startup via a [[Conditions|<condition> block]].
  
Unlike Rembrandt, the Compositor makes use of scene graph cameras
+
== How to enable the Compositor ==
instead of viewer level cameras.
+
  
This allows CameraGroup to manage windows, near/far cameras and other
+
Currently, the Compositor can only be enabled at compile time via the <code>-DENABLE_COMPOSITOR=ON</code> CMake flag in FlightGear.
slaves without interfering with what is being rendered
+
SimGear doesn't require any extra parameters.  
(post-processing, shadows...).
+
  
The Compositor is in an usable state right now: it works but there are
+
On Linux, one can use <code>download_and_compile.sh --compositor</code> (see [[Scripted Compilation on Linux Debian/Ubuntu]]).
no effects or pipelines developed for it. There are also some bugs and
+
features that don't work as expected because of some hardcoded
+
assumptions in the FlightGear Viewer code. Still, I think it's time to
+
announce it so people much more knowledgeable than me can point me in
+
the right direction to get this upstream and warn me about possible
+
issues and worries. :)
+
  
[[File:Compositor-08-2018.png|thumb|A screenshot showing post-processed gaussian blur and a blue filter:[https://i.imgur.com/Zzxre8a.png]]]
+
Once you have a binary with the Compositor enabled and you run it, you will be presented with the default rendering pipeline. At the time of writing, this is the low spec rendering pipeline. If you want to try the [[#ALS|ALS pipeline]], start FlightGear with the command line argument: <code>--compositor=Compositor/als</code>
  
The next major (and hopefully final) iteration of the Compositor is
+
If you want to enable shadows on all objects in the ALS pipeline use these options as a startup parameters (in QT GUI or in the commandline) <code>--prop:bool:/sim/rendering/als/shadows/enabled=true</code> and <code>--prop:int:/sim/rendering/als/shadows/sun-atlas-size=2048</code>. If you feel like the shadows are too low-quality (specially in the cockpit), increase the shadow resolution to 4096 or 8192 instead of 2048.
mostly done now, just some more cleanup is needed. I've tried to keep
+
the changes contained in CameraGroup.cxx/hxx and in new files in
+
SimGear to ease the merging process.
+
  
== Use Cases ==
+
If you are having trouble with lights using an integrated GPU and the Mesa drivers under Linux, try starting FlightGear with the environment variable <tt>MESA_GL_VERSION_OVERRIDE="3.1COMPAT"</tt>
* tail cameras
+
* mirrors
+
* [[Procedural Texturing]]
+
* [[Shuttle ADI ball]]
+
  
== FAQs ==
+
== Notes for aircraft developers ==
{{See also|Unifying the 2D rendering backend via canvas}}
+
  
I started the Compositor to hopefully unify rendering in FG in the most
+
=== Lights ===
non-radical way. Instead of changing a large part of the codebase like
+
Rembrandt, the Compositor aims to bring the same funcionality and slowly
+
deprecate parts of the FG Viewer code.
+
  
Indeed, this should help bring more interesting effects to the table.
+
The Compositor introduces a new way of defining lights that is renderer agnostic, so every rendering pipeline will be able to access the lights that have been implemented like this. As of 2019/11, the only pipeline that supports dynamic lights is the ALS pipeline. The resulting light volumes can be visualized for debugging purposes by setting the property <tt>/sim/debug/show-light-volumes</tt> to true.
I'm specially looking forward to porting ALS to a multi-pass
+
 
environment through PBR.
+
{|cellpadding=10|
 +
|valign=top style="width: 20%;"|
 +
<syntaxhighlight lang="xml">
 +
<light>
 +
  <name>my-spotlight</name>
 +
  <type>spot</type>
 +
  <position>
 +
    <x-m>-7.7476</x-m>
 +
    <y-m>0</y-m>
 +
    <z-m>-1.7990</z-m>
 +
  </position>
 +
  <direction>
 +
    <x>-1.0</x>
 +
    <y>0</y>
 +
    <z>-0.013</z>
 +
  </direction>
 +
  <ambient>
 +
    <r>0.03</r>
 +
    <g>0.03</g>
 +
    <b>0.03</b>
 +
    <a>1</a>
 +
  </ambient>
 +
  <diffuse>
 +
    <r>0.95</r>
 +
    <g>0.9</g>
 +
    <b>0.9</b>
 +
    <a>1</a>
 +
  </diffuse>
 +
  <specular>
 +
    <r>0.95</r>
 +
    <g>0.9</g>
 +
    <b>0.9</b>
 +
    <a>1</a>
 +
  </specular>
 +
  <attenuation>
 +
    <c>1.0</c>
 +
    <l>0.09</l>
 +
    <q>0.032</q>
 +
  </attenuation>
 +
  <spot-exponent>5</spot-exponent>
 +
  <spot-cutoff>40</spot-cutoff>
 +
  <range-m>50</range-m>
 +
</light>
 +
</syntaxhighlight>
 +
|valign=top style="width: 80%;"|
 +
* <tt>'''name'''</tt>. An {{tag|animation}} will be able to reference the light by this name. Most animations will work as expected (rotate, translate, spin etc).
 +
* <tt>'''type'''</tt>. <tt>spot</tt> or <tt>point</tt>.
 +
* <tt>'''position'''</tt>. The position of the light source in model space and in meters.
 +
* <tt>'''direction'''</tt>. Only available in <tt>spot</tt> lights. It indicates the direction of the spotlight. This parameter can be specified in three different ways:
 +
{| class="wikitable" style="border: 1px solid darkgray;"
 +
! scope="col" style="width:33%;" |Direction vector
 +
! scope="col" style="width:33%;" |Look-at point
 +
! scope="col" style="width:33%;" |Rotation angles
 +
|-
 +
| style="padding: 10px" | A vector in model space that specifies the direction. Doesn't have to be normalized.
 +
<syntaxhighlight lang="xml">
 +
<x>-1.0</x>
 +
<y>0</y>
 +
<z>-0.013</z>
 +
</syntaxhighlight>
 +
| style="padding: 10px" | The spotlight will calculate its direction by looking at this position from the light position. The point is in model space and in meters.
 +
<syntaxhighlight lang="xml">
 +
<lookat-x-m>-8.031</lookat-x-m>
 +
<lookat-y-m>0</lookat-y-m>
 +
<lookat-z-m>-2</lookat-z-m>
 +
</syntaxhighlight>
 +
| style="padding: 10px" | A three angle rotation in degrees that rotates the spotlight around the three axes. A 0 degree angle in all axes makes the spotlight point downwards (negative Z).
 +
<syntaxhighlight lang="xml">
 +
<pitch-deg>90</pitch-deg>
 +
<roll-deg>0</roll-deg>
 +
<heading-deg>0</heading-deg>
 +
</syntaxhighlight>
 +
|}
 +
* <tt>'''ambient'''</tt>, <tt>'''diffuse'''</tt> and <tt>'''specular'''</tt>. Four-component vectors that specify the light color.
 +
* <tt>'''attenuation'''</tt>. Three-component vector where <code><c></code> specifies the constant factor, <code><l></code> specifies the linear factor and <code><nowiki><q></nowiki></code> specifies the quadratic factor. These factors are plugged into the OpenGL light attenuation formula [[File:Spotlight_attenuation.png]] where d is the distance of the fragment to the light source. See this [http://wiki.ogre3d.org/tiki-index.php?page=-Point+Light+Attenuation table] for a list of attenuation values based on the range of the light.
 +
* <tt>'''range-m'''</tt>. Maximum range from the light source position in meters. This value will be used by the renderers to determine if a fragment is illuminated by this source. Every fragment outside this range isn't guaranteed to be affected by the light, even if the attenuation factor isn't 0 in that particular fragment.
 +
* <tt>'''cutoff'''</tt>. Only available in <tt>spot</tt> lights. It specifies the maximum spread angle of a light source. Only values in the range 0 90 are accepted. If the angle between the direction of the light and the direction from the light to the fragment being lighted is greater than the spot cutoff angle, it won't be lit.
 +
* <tt>'''exponent'''</tt>. Only available in <tt>spot</tt> lights. Higher spot exponents result in a more focused light source, regardless of the spot cutoff angle.
 +
* <tt>'''debug-color'''</tt> ('''Optional'''). Sets the color of the debug light volume. By default it's red.
 +
|}
  
 
=== Shadows ===
 
=== Shadows ===
It includes shadows indeed. The Compositor stacks several passes of
 
different types on top of each other, and exchanges buffers between
 
them. Nothing prevents someone from creating a new pass type that
 
implements a shadow map pre-pass. I've already tried myself
 
successfully, but the problem is telling the pass which light should be
 
used, which brings me to spotlights.
 
  
=== Light Sources ===
+
The shadow mapping algorithm can be customized entirely by the rendering pipeline. This means that each one will have its own requirements when it comes to shadows. Here are some general recommendations:
Allowing aircraft/scenery developers to include "real" lights is a whole
+
different aspect of the scene graph that shouldn't be managed by the  
+
Compositor. Some kind of system that exposes OpenGL lights should be
+
implemented and linked to the Compositor. Some decisions have to be made
+
though. In a forward renderer lights aren't as cheap as in a deferred
+
renderer, so if the aircraft developer says there has to be a light
+
there, someone using a forward renderer might see a performance drop or
+
not see them at all.
+
  
 +
* Use the <code><noshadow></code> animation to disable shadows on objects that don't need them. An example would be billboarded lights or really small cockpit elements that don't need shadows and would cause degraded performance.
 +
* Try to mark as many cockpit objects as possible as <tt>interior</tt>.
 +
<syntaxhighlight lang="xml">
 +
<model>
 +
  <name>interior</name>
 +
  <usage>interior</usage>
 +
  <path>Aircraft/JA37/Models/ja37-interior.xml</path> <!-- All the objects that should only be seen when inside the cockpit are in this file -->
 +
</model>
 +
</syntaxhighlight>
 +
* Unlike in Rembrandt, polygons facing the Sun are the ones used to generate the shadow map, so single sided surfaces and non-closed objects should be rendered correctly.
  
=== Show me the code ===
+
== Porting and developing Effects/Shaders ==
I've submitted all I have to my FG fork:
+
  
 +
Effects can now have different implementations depending on the Compositor pipeline being used. For example, a grass Effect implemented in the ALS pipeline might have much more detail than the one in the low-spec pipeline. Still, they both implement the "look" of grass, so they share the same Effect file (grass.eff).
  
https://sourceforge.net/u/fgarlin/simgear/ci/new-compositor/tree/
+
The Compositor chooses which implementation of an Effect to render based on the <tt><scheme></tt> of the techniques.
 +
<syntaxhighlight lang="xml">
 +
<technique n="15">
 +
  <scheme>als-lighting</scheme>
 +
  [...]
 +
</technique>
 +
</syntaxhighlight>
  
https://sourceforge.net/u/fgarlin/flightgear-src/ci/new-compositor/tree/
+
In this case the technique will be chosen if the Compositor pipeline <tt><scene></tt> pass uses the <tt>als-lighting</tt> Effect scheme. Consequently, porting an Effect to a pipeline will require knowing which Effect schemes it uses and writing a technique for each one. The only exception to this is the Default (Low-Spec) pipeline, which uses techniques with no scheme.
  
https://sourceforge.net/u/fgarlin/flightgear/ci/new-compositor/tree/
+
Generally porting legacy Effects will require these steps:
 +
* Remove <tt>/sim/rendering/shaders/skydome</tt> from the ALS techniques and add the <tt>als-lighting</tt> scheme.
 +
* Remove Rembrandt techniques.
 +
* Use technique numbers from 0 to 9 for the Default pipeline techniques and 10 to 19 for the ALS pipeline.
 +
* Move the shaders to their correct directory. Now the <tt>$FG_ROOT/Shaders</tt> directory is subdivided into different folders for each pipeline. For example, shaders related to the ALS pipeline are located in <tt>$FG_ROOT/Shaders/ALS</tt>.
  
I haven't had the time to update the repos to the latest changes in the official repos, but it shouldn't matter for now.
+
To add features specific to a particular pipeline (like shadows in the ALS pipeline), see each pipeline's documentation below.
  
=== Merging ===
+
=== ALS ===
I'd love to have this merged as soon as possible, but maybe next month is a
+
bit too rushed. I don't even know if the approach I'm using will be the
+
definitive one or the small implications the Compositor might have on the
+
rest of the viewer related code. I'm not in a rush though, I can wait for
+
the next release if I can't make a decent enough merge request in June/July.
+
  
== Elements ==
+
==== Adding shadows ====
  
=== Buffers ===
+
==== Rendering correct depth ====
  
A buffer represents a texture or, more generically, a region of GPU memory. Textures can be of any type allowed by OpenGL: 1D, 2D, rectangle, 2Darray, 3D or cubemap.
+
The ALS pipeline uses a logarithmic depth buffer. This is accomplished by writing to <tt>gl_FragDepth</tt> in the fragment shader or by modifying <tt>gl_Position.z</tt> in the vertex shader. The second option is faster but doesn't work for polygons that intersect the near plane of the camera (e.g. terrain), so you should use it by default unless visual bugs appear.
  
A typical [[PropertyList XML File|property tree structure]] describing a buffer may be as follows:
+
* '''1st option'''
 +
Add the following to the vertex shader:
 +
<syntaxhighlight lang="glsl">
 +
uniform float fg_Fcoef;
 +
[...]
 +
gl_Position = ...
 +
gl_Position.z = (log2(max(1e-6, 1.0 + gl_Position.w)) * fg_Fcoef - 1.0) * gl_Position.w;
 +
</syntaxhighlight>
  
<syntaxhighlight lang="xml">
+
* '''2nd option'''
      <buffer>
+
Add the following to the vertex shader:
            <name>buffer-name</name>
+
<syntaxhighlight lang="glsl">
    <type>2d</type>
+
varying float flogz;
    <width>512</width>
+
[...]
    <height>512</height>
+
gl_Position = ...
    <scale-factor>1.0</scale-factor>
+
flogz = 1.0 + gl_Position.w;
    <internal-format>rgba8</internal-format>
+
    <source-format>rgba</source-format>
+
            <source-type>ubyte</source-type>
+
      </buffer>
+
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
And the following to the fragment shader:
 +
<syntaxhighlight lang="glsl">
 +
uniform float fg_Fcoef;
 +
varying float flogz;
 +
[...]
 +
gl_FragColor = ...
 +
gl_FragDepth = log2(flogz) * fg_Fcoef * 0.5;
 +
</syntaxhighlight>
 +
 +
== Pipeline technical notes ==
 +
 +
=== Default (Low-Spec) ===
 +
 +
A fixed function forward rendering pipeline mainly targeted to low spec systems. It imitates the classic forward pipeline used before multi-pass rendering was introduced by using two near/far cameras rendering directly to the screen.
 +
 +
[[File:Low-spec Compositor pipeline.png|thumb|Screenshot showing OSG stats of the Compositor-based low-spec rendering pipeline.]]
 +
 +
=== ALS ===
 +
{{See also|ALS technical notes}}
 +
 +
The ALS pipeline tries to bring multipass rendering to the current ALS framework, effectively combining the best from ALS and Project Rembrandt.
 +
 +
==== Cascaded shadow mapping ====
 +
 +
The main issue with shadow mapping in FlightGear is the complexity of the scene graph. Culling times can become huge if we don't carefully select which parts of the scene graph we want to render in the shadow maps. Some possible optimizations:
 +
* Study the minimum shadow map distance we can get without noticeable light leaking. Select an appropiate amount of cascades (more cascades = more passes over all geometry, and in general we want to keep the amount of forward passes to a minimum). We should have at least three cascades: the first just for cockpit/internal shadows, the second for the whole aircraft and the third for the rest of the scenery geometry. A fourth can be added if the transition between the second and the third is too harsh.
 +
* Improve the culling masks (simgear/scene/util/RenderConstants.hxx). The CASTSHADOW_BIT flag is present in almost every object in the scene graph. Turning this flag off for trees, random buildings and other geometry intensive objects improves framerates by a very considerable amount. Should the user be able to select which objects cast shadows?
 +
* Should the terrain cast shadows? The terrain is rarely steep enough to cast shadows. Apart from that, the terrain in FlightGear messes with automatic near/far computations for the shadow passes since the geometry is not tessellated enough. Also, the terrain LOD is not good enough to have decent cull times at far cascades.
 +
* Adding a "internal only" shadow flag for aircraft developers. This allows farther shadow cascades to cull complex objects that are only visible in the nearest cascades. (Very important optimization for aircraft with complex cockpit geometry).
 +
* Vegetation shadows will be done by the "legacy" method currently in use. Shadow mapping on vegetation is much more expensive in terms of performance and the current algorithm does the job well enough. <ref>https://forum.flightgear.org/viewtopic.php?f=47&p=357606</ref>
 +
 +
==== Post-processing ====
 +
 +
Gamma correction, night vision and other ALS filters should happen in a quad pass. The current filter_combined() should be left for post-processing that requires as much precision as possible - e.g. dithering to prevent banding). HDR is not a planned feature for now so ALS will be using rgba8 buffers for most of its features.
 +
 +
==== Real-time dynamic reflections ====
 +
 +
Rendering dynamically to a cubemap is possible. As with shadow mapping, minimizing the object count and number of forward passes is vital to get good performance in FlightGear. Rendering to six cubemap faces requires six forward passes, but we can render to a dual paraboloid map instead, reducing this number to two.
 +
 +
==== Transparency ====
 +
 +
When shadows (and multipass rendering in general) come into play, transparent objects have to be treated differently, even when we are dealing with a forward renderer. In OSG there are two ways to separate transparent surfaces:
 +
 +
* Using RenderBins. After a single scene cull traversal, surfaces which belong to a special RenderBin type (DepthSortedBin) are removed or moved to another camera. This is how Rembrandt does it and it is the most backwards compatible approach since RenderBins can be changed directly inside Effects.
 +
* Using cull masks. Two separate traversals are done: one for opaque objects and another for translucent objects. This requires offering aircraft developers another way of tagging a surface as transparent. A trivial approach would be to add a new <animation> type called 'transparent', but that wouldn't be backwards compatible. Maybe we can add some kind of system where we can change cull masks inside Effects? Would that be too hacky or out of place?
 +
 +
== Creating a custom rendering pipeline ==
 +
 +
Since the Compositor is completely data-driven, new rendering pipelines can be created by writing a custom XML pipeline definition. This section tries to document most of the available parameters, but the best and most up-to-date resource is the Compositor parsing code in SimGear ({{simgear file|simgear/scene/viewer}}). See existing pipelines in {{fgdata file|Compositor}} for practical examples on how to use these parameters.
 +
 +
=== Buffers ===
 +
 +
A buffer represents a texture or, more generically, a region of GPU memory.
 +
 +
{| class="wikitable" style="text-align: center; font-size: 85%; width: auto; table-layout: fixed;
 +
! scope="col" | Parameter Name
 +
! scope="col" | Optional
 +
! scope="col" | Value
 +
! scope="col" | Default Value
 +
! scope="col" | Description
 +
|-
 +
! scope="row"| <tt>name</tt>
 +
| {{No}}
 +
| string
 +
|
 +
| Passes will be able to address the buffer by this name
 +
|-
 +
! scope="row"| <tt>type</tt>
 +
| {{No}}
 +
| <tt>1d, 2d, 2d-array, 2d-multisample, 3d, rect, cubemap</tt>
 +
|
 +
| Any texture type allowed by OpenGL
 +
|-
 +
! scope="row"| <tt>width</tt>
 +
| {{No}}
 +
| Any unsigned integer or <tt>screen</tt> to use the physical viewport width. The <code><property></code> tag can also be used to use a property value
 +
|
 +
| Texture width
 +
|-
 +
! scope="row"| <tt>screen-width-scale</tt>
 +
| {{Yes}}
 +
| float
 +
| <tt>1.0</tt>
 +
| If <tt>screen</tt> was used, this controls the width scaling factor
 +
|-
 +
! scope="row"| <tt>height</tt>
 +
| {{No}}
 +
| Any unsigned integer or <tt>screen</tt> to use the physical viewport height. The <code><property></code> tag can also be used to use a property value
 +
|
 +
| Texture height
 +
|-
 +
! scope="row"| <tt>screen-height-scale</tt>
 +
| {{Yes}}
 +
| float
 +
| <tt>1.0</tt>
 +
| If <tt>screen</tt> was used, this controls the height scaling factor
 +
|-
 +
! scope="row"| <tt>depth</tt>
 +
| {{No}}
 +
| Any unsigned integer. The <code><property></code> tag can also be used to use a property value
 +
|
 +
| Texture depth
 +
|-
 +
! scope="row"| <tt>format</tt>
 +
| {{Yes}}
 +
| See {{simgear file|simgear/scene/viewer/CompositorBuffer.cxx}} for the latest available values
 +
| <tt>rgba8</tt>
 +
| Specifies the texture format. It corresponds to the ''internalformat'', ''format'' and ''type'' arguments of the OpenGL function ''glTexImage2D''
 +
|-
 +
! scope="row"| <tt>min-filter, mag-filter</tt>
 +
| {{Yes}}
 +
| <tt>linear, linear-mipmap-linear, linear-mipmap-nearest, nearest, nearest-mipmap-linear, nearest-mipmap-nearest</tt>
 +
| <tt>linear</tt>
 +
| Change the minification and magnification filtering respectively
 +
|-
 +
! scope="row"| <tt>wrap-s, wrap-t, wrap-r</tt>
 +
| {{Yes}}
 +
| <tt>clamp, clamp-to-edge, clamp-to-border, repeat, mirror</tt>
 +
| <tt>clamp-to-border</tt>
 +
| They change the wrap mode for each coordinate
 +
|-
 +
! scope="row"| <tt>anisotropy</tt>
 +
| {{Yes}}
 +
| float
 +
| <tt>1.0</tt>
 +
|
 +
|-
 +
! scope="row"| <tt>border-color</tt>
 +
| {{Yes}}
 +
| vec4
 +
| <tt>(0.0f, 0.0f, 0.0f, 0.0f)</tt>
 +
|
 +
|-
 +
! scope="row"| <tt>shadow-comparison</tt>
 +
| {{Yes}}
 +
| bool
 +
| <tt>true</tt>
 +
|
 +
|-
 +
! scope="row"| <tt>shadow-texture-mode</tt>
 +
| {{Yes}}
 +
| <tt>luminance, intensity, alpha</tt>
 +
| <tt>luminance</tt>
 +
|
 +
|-
 +
! scope="row"| <tt>shadow-compare-func</tt>
 +
| {{Yes}}
 +
| <tt>never, less, equal, lequal, greater, notequal, gequal, always</tt>
 +
| <tt>lequal</tt>
 +
|
 +
|}
  
 
=== Passes ===
 
=== Passes ===
  
A pass wraps around an [http://public.vrac.iastate.edu/vancegroup/docs/OpenSceneGraphReferenceDocs-3.0/a00089.html osg::Camera]. As of February 2018, there are two types of passes supported:
+
A pass wraps around an [http://public.vrac.iastate.edu/vancegroup/docs/OpenSceneGraphReferenceDocs-3.0/a00089.html osg::Camera]. Passes all have some common parameters:
  
* '''scene'''. Renders a scene. The scene can be an already loaded scene graph node (terrain, aircraft etc.) or a path to a custom 3D model.
+
{| class="wikitable" style="text-align: center; font-size: 85%; width: auto; table-layout: fixed;
* '''quad'''. Renders a fullscreen quad with an optional [[Effects|effect]] applied. Useful for screen space shaders (like SSAO, Screen Space Reflections or bloom) and deferred rendering.
+
! scope="col" | Parameter Name
 +
! scope="col" | Optional
 +
! scope="col" | Value
 +
! scope="col" | Default Value
 +
! scope="col" | Description
 +
|-
 +
! scope="row"| <tt>clear-color, clear-accum, clear-depth and clear-stencil</tt>
 +
| {{Yes}}
 +
| vec4
 +
| black, black, <tt>1.0</tt>, <tt>0</tt> respectively
 +
| Pass clear colors
 +
|-
 +
! scope="row"| <tt>clear-mask</tt>
 +
| {{Yes}}
 +
| <tt>color, stencil, depth, accum</tt>
 +
| <tt>color depth</tt>
 +
| Pass clear mask
 +
|-
 +
! scope="row"| <tt>effect-scheme</tt>
 +
| {{Yes}}
 +
| Valid effect scheme name
 +
| None
 +
| The pass will try to use the specified effect scheme to draw every object.
 +
|}
  
Passes can render to a buffer (Render to Texture), to several buffers (Multiple Render Targets) or directly to the OSG context. This allows chaining of multiple passes, sharing buffers between them. They can also receive other buffers as input, so effects can access them as normal textures.
+
Passes can render to a buffer (Render to Texture), to several buffers (Multiple Render Targets) or directly to the framebuffer. This is accomplished by the <code><attachment></code> tag. Possible parameters of an attachment are:
  
Example XML for a ''quad'' type pass:
+
{| class="wikitable" style="text-align: center; font-size: 85%; width: auto; table-layout: fixed;
 +
! scope="col" | Parameter Name
 +
! scope="col" | Optional
 +
! scope="col" | Value
 +
! scope="col" | Default Value
 +
! scope="col" | Description
 +
|-
 +
! scope="row"| <tt>buffer</tt>
 +
| {{No}}
 +
| Valid buffer name
 +
|
 +
| The name of the buffer to output to
 +
|-
 +
! scope="row"| <tt>component</tt>
 +
| {{No}}
 +
| <tt>color, color0</tt> to <tt>color15, depth, stencil, depth-stencil</tt>
 +
|
 +
| FBO attachment point
 +
|-
 +
! scope="row"| <tt>level</tt>
 +
| {{Yes}}
 +
| int
 +
| <tt>0</tt>
 +
| Mipmap level of the texture that is attached
 +
|-
 +
! scope="row"| <tt>face</tt>
 +
| {{Yes}}
 +
| int
 +
| <tt>0</tt>
 +
| Face of cube map texture or z-level of 3d texture
 +
|-
 +
! scope="row"| <tt>mipmap-generation</tt>
 +
| {{Yes}}
 +
| bool
 +
| <tt>false</tt>
 +
| Whether mipmap generation should be done for texture
 +
|-
 +
! scope="row"| <tt>multisample-samples</tt>
 +
| {{Yes}}
 +
| int
 +
| <tt>0</tt>
 +
| Multisample anti-aliasing (MSAA) samples
 +
|-
 +
! scope="row"| <tt>multisample-color-samples</tt>
 +
| {{Yes}}
 +
| int
 +
| <tt>0</tt>
 +
| Multisample anti-aliasing (MSAA) color samples
 +
|}
  
<syntaxhighlight lang="xml">
+
Passes can also receive buffers as input and use them in their shaders. This is accomplished by the <code><binding></code> tag, which has the following parameters:
      <pass-quad>
+
          <name>pass-name</name>
+
  
          <effect>Effects/test</effect>
+
{| class="wikitable" style="text-align: center; font-size: 85%; width: auto; table-layout: fixed;
   
+
! scope="col" | Parameter Name
          <input-buffer>
+
! scope="col" | Optional
              <buffer>some-buffer</buffer>
+
! scope="col" | Value
              <unit>0</unit>
+
! scope="col" | Default Value
          </input-buffer>
+
! scope="col" | Description
 +
|-
 +
! scope="row"| <tt>buffer</tt>
 +
| {{No}}
 +
| Valid buffer name
 +
|
 +
| The name of the buffer to bind
 +
|-
 +
! scope="row"| <tt>unit</tt>
 +
| {{No}}
 +
| int
 +
|
 +
| The texture unit to place the texture on. Effects will be able to access the buffer on this texture unit
 +
|}
  
          <output-buffer>
+
There are specific pass types, each with their own set of custom parameters.
              <buffer>color-buffer</buffer>
+
              <component>color</component>
+
          </output-buffer>
+
          <output-buffer>
+
              <buffer>depth-buffer</buffer>
+
              <component>depth</component>
+
          </output-buffer>
+
      </pass-quad>
+
</syntaxhighlight>
+
  
== Canvas integration ==
+
==== scene ====
 +
Renders the scene from the point of view given by the CameraGroup.
 +
{| class="wikitable" style="text-align: center; font-size: 85%; width: auto; table-layout: fixed;
 +
! scope="col" | Parameter Name
 +
! scope="col" | Optional
 +
! scope="col" | Value
 +
! scope="col" | Default Value
 +
! scope="col" | Description
 +
|-
 +
! scope="row"| <tt>cull-mask</tt>
 +
| {{Yes}}
 +
| A 32 bit number. See {{simgear file|simgear/scene/util/RenderConstants.hxx}} to know which bits enable what
 +
| <tt>0xffffffff</tt>
 +
| Specifies the cull mask to be used in the underlying <tt>osg::Camera</tt>
 +
|-
 +
! scope="row"| <tt>z-near, z-far</tt>
 +
| {{Yes}}
 +
| float
 +
| 0.0 uses the value given by the CameraGroup
 +
| Sets a custom near and far values. Useful for implementing depth partition and limiting the depth range on cubemap passes
 +
|-
 +
! scope="row"| <tt>cubemap-face</tt>
 +
| {{Yes}}
 +
| int
 +
| <tt>-1</tt> (don't use cubemap)
 +
| Ignores the given view and projection matrices and uses a custom one that renders the scene as if it was seen from inside a cubemap looking towards the specified face
 +
|-
 +
! scope="row"| <tt>use-shadow-pass</tt>
 +
| {{Yes}}
 +
| string
 +
| Empty
 +
| Name of a shadow mapping pass. Exposes shadow mapping related uniforms to the shaders of the current pass
 +
|}
  
[[File:Blur_and_blue_filter_applied_to_a_buffer.png|thumb|Post processed image (right) after applying gaussian blur and a blue filter to a scene pass (left).]]
+
Scene passes can also use the tag <code><clustered-shading></code> to enable clustered shading (lights). The following parameters are available:
  
Apart from serving as a debugging tool for visualizing the contents of a buffer, integrating the Compositor with [[Canvas]] allows aircraft developers to access RTT capabilities. Compositor buffers can be accessed within Canvas via a new custom Canvas Image protocol '''buffer://'''. For example, the path <code>buffer://test-compositor/test-buffer</code> displays the buffer test-buffer declared in test-compositor.
+
{| class="wikitable" style="text-align: center; font-size: 85%; width: auto; table-layout: fixed;
 +
! scope="col" | Parameter Name
 +
! scope="col" | Optional
 +
! scope="col" | Value
 +
! scope="col" | Default Value
 +
! scope="col" | Description
 +
|-
 +
! scope="row"| <tt>tile-size</tt>
 +
| {{Yes}}
 +
| int
 +
| 128
 +
| Size of each clustered shading tile
 +
|-
 +
! scope="row"| <tt>num-threads</tt>
 +
| {{Yes}}
 +
| int
 +
| 1
 +
| Number of threads to use during the light culling process. Keep in mind that a high thread count when there aren't many lights will worsen performance due to the thread creation overhead
 +
|-
 +
! scope="row"| <tt>depth-slices</tt>
 +
| {{Yes}}
 +
| int
 +
| 1
 +
| Number of slices to partition the view frustum in the Z axis. Higher numbers will cull lights more aggressively, increasing performance if there are many lights further out that don't contribute much to the overall scene's lighting
 +
|}
  
<syntaxhighlight lang="nasal">
+
==== quad ====
var (width,height) = (612,612);
+
Renders a fullscreen quad with an optional [[Effects|effect]] applied. Useful for screen space shaders (like SSAO, Screen Space Reflections or bloom) and deferred rendering.
var title = 'Compositor&Canvas test';
+
{| class="wikitable" style="text-align: center; font-size: 85%; width: auto; table-layout: fixed;
var window = canvas.Window.new([width,height],"dialog")
+
! scope="col" | Parameter Name
.set('title',title);
+
! scope="col" | Optional
var myCanvas = window.createCanvas().set("background", canvas.style.getColor("bg_color"));
+
! scope="col" | Value
var root = myCanvas.createGroup();
+
! scope="col" | Default Value
var path = "buffer://test-compositor/test-buffer";
+
! scope="col" | Description
var child = root.createChild("image")
+
|-
    .setTranslation(50, 50)
+
! scope="row"| <tt>geometry</tt>
    .setSize(512, 512)
+
| {{Yes}}
    .setFile(path);
+
| float values for <code><x>, <y>, <width>, <height></code>
</syntaxhighlight>
+
| <tt>0.0, 0.0, 1.0, 1.0</tt> respectively
 +
| Size of the fullscreen quad inside the viewport using normalized coordinates.
 +
|-
 +
! scope="row"| <tt>effect</tt>
 +
| {{Yes}}
 +
| Valid Effect file
 +
| None
 +
| This Effect will be applied to the quad geometry
 +
|}
 +
 
 +
==== shadow-map ====
 +
Renders the scene from a light's point of view.
 +
{| class="wikitable" style="text-align: center; font-size: 85%; width: auto; table-layout: fixed;
 +
! scope="col" | Parameter Name
 +
! scope="col" | Optional
 +
! scope="col" | Value
 +
! scope="col" | Default Value
 +
! scope="col" | Description
 +
|-
 +
! scope="row"| <tt>light-name</tt>
 +
| {{No}}
 +
| Valid light name that exists in the scene graph
 +
|
 +
| The name of the <tt>osg::LightSource</tt> to use for this shadow map
 +
|-
 +
! scope="row"| <tt>near-m, far-m</tt>
 +
| {{No}}
 +
| float (meters)
 +
|
 +
| They specify the depth range of the shadow map
 +
|}
 +
 
 +
== TODO ==
 +
{{See also|Post FlightGear 2020.2 LTS changes}}
 +
 
 +
* Bring back distortion correction.
 +
* Some kind of versioning system to be able to make breaking changes in the future if/when the compositor is updated in any significant way, without people having to manually update their configs.
 +
* Rework node masks ({{simgear file|simgear/scene/util/RenderConstants.hxx}}). Each node type should have its own node mask bit, e.g. trees should only have <tt>TREE_BIT</tt> enabled, the main aircraft should only have <tt>AIRCRAFT_BIT</tt> enabled etc. This allows the Compositor cameras to properly filter scene graph nodes based on the cull mask. Shadow cameras can then choose what geometry they render dynamically.
 +
* Automatically calculate light source attenuation based on radius and radius based on attenuation.
 +
* Add clustered shading to every Effect as a separate technique.
 +
* Proper shader quality settings for both low-spec and ALS pipelines.
 +
* Launcher and UI:
 +
** [[Integrated Qt5 Launcher]]
 +
** [[About dialog]]
 +
** Rendering dialog
 +
* Separate [[Developing using CMake|cmake build target (binary)]] for a [http://download.flightgear.org/builds/nightly/ "preview" binary] with the [[Compositor]] enabled to get more testing/feedback and contributions from early-adopters <ref>https://sourceforge.net/p/flightgear/mailman/message/36977221/</ref> <ref>https://sourceforge.net/p/flightgear/mailman/message/36954518/</ref> <ref>https://sourceforge.net/p/flightgear/mailman/message/36679177/</ref> <ref>https://sourceforge.net/p/flightgear/mailman/message/37000737/</ref> {{Progressbar|90}}
  
 
== Known Issues ==
 
== Known Issues ==
  
* Canvas (sc::Image) doesn't update its texture unless .update() is invoked in Nasal. Is this a feature?
+
* Setting a buffer scale factor different from 1.0 and rendering to it might not scale the splash screen correctly.
* The scene graph and the viewer are still managed by FGRenderer. Maybe hijacking the init sequence is a good idea?
+
* There is some kind of moiré pattern at certain sunlight angles (specially at dusk/dawn). Shadow acne.
* light sources ?
+
* Spotlights sometimes disappear at certain view angles.
 +
* Some effects might not be ported to ALS completely. Feel free to play around with the shader settings to get something that works.
 +
* CRT effect as a default Effect. Look into reflect.eff
 +
* osgText doesn't work. See the 777 displays as well as some EC135 instruments. osgText doesn't use Effects so they are being rendered with incorrect depth.
 +
* Reloading shaders doesn't work.
 +
* Random buildings don't use correct depth.
  
== Related ==
+
== References ==
 
{{Appendix}}
 
{{Appendix}}
 +
 +
== Related content ==
 +
=== Wiki articles ===
 +
* [[Canvas View Camera Element]]
 +
* [[CompositeViewer Support]]
 +
* [[Uniform Buffer Objects]]
 +
* [[FlightGear CIGI Support (Common Image Generator Interface)]]
 +
 +
=== Forum topics ===
 +
* {{forum link|t=36269|text=The Compositor}}
 +
* {{forum link|t=35095|text=Clustered Forward Rendering}} (12/2018)
 +
* {{forum link|t=33045|text=Getting started with RTT}}
 +
 +
[[Category:Core development projects]]

Latest revision as of 14:06, 13 May 2020

Compositor Framework
ALS Compositor pipeline.jpg
Started in 01/2018 (Available since FlightGear 2019.2)
Description Dynamic rendering pipeline configured via the Property tree and XML
Contributor(s) Fernando García Liñán [1]
Status Stable (merged and actively maintained)
Folders
Changelog https://sourceforge.net/u/fgarlin/profile/feed.rss


The Compositor aims to bring multi-pass rendering to FlightGear. It encapsulates a rendering pipeline and exposes its parameters to a Property Tree interface. At startup, FlightGear reads the pipeline definition file for each physical viewport defined on the CameraGroup settings. If no Compositor file is specified for a physical camera, the one given by the --compositor= startup command will be used. If such startup option is not used either, FlightGear will look for a valid Compositor file in $FG_ROOT/Compositor/default.xml

The Compositor introduces a new dedicated fgdata directory for new/custom rendering pipelines: fgdata/Compositor.

The framework is now included in the latest stable release (FlightGear v2020.1) as a separate binary. Developers are encouraged to test the Compositor with their personal projects and report any breakages. Also keep in mind that some features might not work/look as intended - see Known Issues.

Background

First discussed in 03/2012 during the early Rembrandt days, Zan (Lauri Peltonen) came up with a set of patches demonstrating how to create an XML-configurable rendering pipeline.

Back then, this work was considered to look pretty promising [2] and at the time plans were discussed to unify this with the ongoing Rembrandt implementation (no longer maintained).

Adopting Zan's approach would have meant that efforts like Rembrandt (deferred rendering) could have been implemented without requiring C++ space modifications, i.e. purely in Base package space.

Rembrandt's developer (FredB) suggested to extend the format to avoid duplicating the stages when you have more than one viewport, i.e. specifying a pipeline as a template, with conditions like in effects, and have the current camera layout refer the pipeline that would be duplicated, resized and positioned for each declared viewport [3]

Zan's original patches can still be found in his newcameras branches which allow the user to define the rendering pipeline in preferences.xml: FlightGear, SimGear.

At that point, it didn't have everything Rembrandt's pipeline needs, but most likely could be easily enhanced to support those things.

Basically, the original version added support for multiple camera passes, texture targets, texture formats, passing textures from one pass to another etc, while preserving the standard rendering line if user wants that. [4]

Since the early days of Zan's groundwork, providing the (hooks) infrastructure to enable base package developers to prototype, test and develop distinct rendering pipelines without requiring C++ space modifications has been a long-standing idea, especially after the Canvas system became available in early 2012, which demonstrated how RTT-rendering buffers (FBOs) could be set up, created and manipulated procedurally (i.e. at run-time) using XML, the property tree and Nasal scripting. [5]

The new Compositor is an improved re-implementation of Zan's original work using not just XML, but also properties and a handful of Canvas concepts.

Features

  • Completely independent of other parts of the simulator, i.e. it's part of SimGear and can be used in a standalone fashion if needed, ala Canvas.
  • Although independent, its aim is to be fully compatible with the current rendering framework in FG. This includes the Effects system, CameraGroup, Rembrandt and ALS (and obviously the Canvas).
  • Its functionality overlaps Rembrandt: what can be done with Rembrandt can be done with the Compositor, but not vice versa.
  • Fully configurable via an XML interface without compromising performance (ala Effects, using PropertyList files).
  • Flexible, expandable and compatible with modern graphics.
  • It doesn't increase the hardware requirements, it expands the hardware range FG can run on. People with integrated GPUs (Intel HD etc) can run a Compositor with a single pass that renders directly to the screen like before, while people with more powerful cards can run a Compositor that implements deferred rendering, for example.
  • Static branching support. Every pipeline element can be enabled/disabled at startup via a <condition> block.

How to enable the Compositor

Currently, the Compositor can only be enabled at compile time via the -DENABLE_COMPOSITOR=ON CMake flag in FlightGear. SimGear doesn't require any extra parameters.

On Linux, one can use download_and_compile.sh --compositor (see Scripted Compilation on Linux Debian/Ubuntu).

Once you have a binary with the Compositor enabled and you run it, you will be presented with the default rendering pipeline. At the time of writing, this is the low spec rendering pipeline. If you want to try the ALS pipeline, start FlightGear with the command line argument: --compositor=Compositor/als

If you want to enable shadows on all objects in the ALS pipeline use these options as a startup parameters (in QT GUI or in the commandline) --prop:bool:/sim/rendering/als/shadows/enabled=true and --prop:int:/sim/rendering/als/shadows/sun-atlas-size=2048. If you feel like the shadows are too low-quality (specially in the cockpit), increase the shadow resolution to 4096 or 8192 instead of 2048.

If you are having trouble with lights using an integrated GPU and the Mesa drivers under Linux, try starting FlightGear with the environment variable MESA_GL_VERSION_OVERRIDE="3.1COMPAT"

Notes for aircraft developers

Lights

The Compositor introduces a new way of defining lights that is renderer agnostic, so every rendering pipeline will be able to access the lights that have been implemented like this. As of 2019/11, the only pipeline that supports dynamic lights is the ALS pipeline. The resulting light volumes can be visualized for debugging purposes by setting the property /sim/debug/show-light-volumes to true.

<light>
  <name>my-spotlight</name>
  <type>spot</type>
  <position>
    <x-m>-7.7476</x-m>
    <y-m>0</y-m>
    <z-m>-1.7990</z-m>
  </position>
  <direction>
    <x>-1.0</x>
    <y>0</y>
    <z>-0.013</z>
  </direction>
  <ambient>
    <r>0.03</r>
    <g>0.03</g>
    <b>0.03</b>
    <a>1</a>
  </ambient>
  <diffuse>
    <r>0.95</r>
    <g>0.9</g>
    <b>0.9</b>
    <a>1</a>
  </diffuse>
  <specular>
    <r>0.95</r>
    <g>0.9</g>
    <b>0.9</b>
    <a>1</a>
  </specular>
  <attenuation>
    <c>1.0</c>
    <l>0.09</l>
    <q>0.032</q>
  </attenuation>
  <spot-exponent>5</spot-exponent>
  <spot-cutoff>40</spot-cutoff>
  <range-m>50</range-m>
</light>
  • name. An <animation> will be able to reference the light by this name. Most animations will work as expected (rotate, translate, spin etc).
  • type. spot or point.
  • position. The position of the light source in model space and in meters.
  • direction. Only available in spot lights. It indicates the direction of the spotlight. This parameter can be specified in three different ways:
Direction vector Look-at point Rotation angles
A vector in model space that specifies the direction. Doesn't have to be normalized.
<x>-1.0</x>
<y>0</y>
<z>-0.013</z>
The spotlight will calculate its direction by looking at this position from the light position. The point is in model space and in meters.
<lookat-x-m>-8.031</lookat-x-m>
<lookat-y-m>0</lookat-y-m>
<lookat-z-m>-2</lookat-z-m>
A three angle rotation in degrees that rotates the spotlight around the three axes. A 0 degree angle in all axes makes the spotlight point downwards (negative Z).
<pitch-deg>90</pitch-deg>
<roll-deg>0</roll-deg>
<heading-deg>0</heading-deg>
  • ambient, diffuse and specular. Four-component vectors that specify the light color.
  • attenuation. Three-component vector where <c> specifies the constant factor, <l> specifies the linear factor and <q> specifies the quadratic factor. These factors are plugged into the OpenGL light attenuation formula Spotlight attenuation.png where d is the distance of the fragment to the light source. See this table for a list of attenuation values based on the range of the light.
  • range-m. Maximum range from the light source position in meters. This value will be used by the renderers to determine if a fragment is illuminated by this source. Every fragment outside this range isn't guaranteed to be affected by the light, even if the attenuation factor isn't 0 in that particular fragment.
  • cutoff. Only available in spot lights. It specifies the maximum spread angle of a light source. Only values in the range 0 90 are accepted. If the angle between the direction of the light and the direction from the light to the fragment being lighted is greater than the spot cutoff angle, it won't be lit.
  • exponent. Only available in spot lights. Higher spot exponents result in a more focused light source, regardless of the spot cutoff angle.
  • debug-color (Optional). Sets the color of the debug light volume. By default it's red.

Shadows

The shadow mapping algorithm can be customized entirely by the rendering pipeline. This means that each one will have its own requirements when it comes to shadows. Here are some general recommendations:

  • Use the <noshadow> animation to disable shadows on objects that don't need them. An example would be billboarded lights or really small cockpit elements that don't need shadows and would cause degraded performance.
  • Try to mark as many cockpit objects as possible as interior.
<model>
  <name>interior</name>
  <usage>interior</usage>
  <path>Aircraft/JA37/Models/ja37-interior.xml</path> <!-- All the objects that should only be seen when inside the cockpit are in this file -->
</model>
  • Unlike in Rembrandt, polygons facing the Sun are the ones used to generate the shadow map, so single sided surfaces and non-closed objects should be rendered correctly.

Porting and developing Effects/Shaders

Effects can now have different implementations depending on the Compositor pipeline being used. For example, a grass Effect implemented in the ALS pipeline might have much more detail than the one in the low-spec pipeline. Still, they both implement the "look" of grass, so they share the same Effect file (grass.eff).

The Compositor chooses which implementation of an Effect to render based on the <scheme> of the techniques.

<technique n="15">
  <scheme>als-lighting</scheme>
  [...]
</technique>

In this case the technique will be chosen if the Compositor pipeline <scene> pass uses the als-lighting Effect scheme. Consequently, porting an Effect to a pipeline will require knowing which Effect schemes it uses and writing a technique for each one. The only exception to this is the Default (Low-Spec) pipeline, which uses techniques with no scheme.

Generally porting legacy Effects will require these steps:

  • Remove /sim/rendering/shaders/skydome from the ALS techniques and add the als-lighting scheme.
  • Remove Rembrandt techniques.
  • Use technique numbers from 0 to 9 for the Default pipeline techniques and 10 to 19 for the ALS pipeline.
  • Move the shaders to their correct directory. Now the $FG_ROOT/Shaders directory is subdivided into different folders for each pipeline. For example, shaders related to the ALS pipeline are located in $FG_ROOT/Shaders/ALS.

To add features specific to a particular pipeline (like shadows in the ALS pipeline), see each pipeline's documentation below.

ALS

Adding shadows

Rendering correct depth

The ALS pipeline uses a logarithmic depth buffer. This is accomplished by writing to gl_FragDepth in the fragment shader or by modifying gl_Position.z in the vertex shader. The second option is faster but doesn't work for polygons that intersect the near plane of the camera (e.g. terrain), so you should use it by default unless visual bugs appear.

  • 1st option

Add the following to the vertex shader:

uniform float fg_Fcoef;
[...]
gl_Position = ...
gl_Position.z = (log2(max(1e-6, 1.0 + gl_Position.w)) * fg_Fcoef - 1.0) * gl_Position.w;
  • 2nd option

Add the following to the vertex shader:

varying float flogz;
[...]
gl_Position = ...
flogz = 1.0 + gl_Position.w;

And the following to the fragment shader:

uniform float fg_Fcoef;
varying float flogz;
[...]
gl_FragColor = ...
gl_FragDepth = log2(flogz) * fg_Fcoef * 0.5;

Pipeline technical notes

Default (Low-Spec)

A fixed function forward rendering pipeline mainly targeted to low spec systems. It imitates the classic forward pipeline used before multi-pass rendering was introduced by using two near/far cameras rendering directly to the screen.

Screenshot showing OSG stats of the Compositor-based low-spec rendering pipeline.

ALS

The ALS pipeline tries to bring multipass rendering to the current ALS framework, effectively combining the best from ALS and Project Rembrandt.

Cascaded shadow mapping

The main issue with shadow mapping in FlightGear is the complexity of the scene graph. Culling times can become huge if we don't carefully select which parts of the scene graph we want to render in the shadow maps. Some possible optimizations:

  • Study the minimum shadow map distance we can get without noticeable light leaking. Select an appropiate amount of cascades (more cascades = more passes over all geometry, and in general we want to keep the amount of forward passes to a minimum). We should have at least three cascades: the first just for cockpit/internal shadows, the second for the whole aircraft and the third for the rest of the scenery geometry. A fourth can be added if the transition between the second and the third is too harsh.
  • Improve the culling masks (simgear/scene/util/RenderConstants.hxx). The CASTSHADOW_BIT flag is present in almost every object in the scene graph. Turning this flag off for trees, random buildings and other geometry intensive objects improves framerates by a very considerable amount. Should the user be able to select which objects cast shadows?
  • Should the terrain cast shadows? The terrain is rarely steep enough to cast shadows. Apart from that, the terrain in FlightGear messes with automatic near/far computations for the shadow passes since the geometry is not tessellated enough. Also, the terrain LOD is not good enough to have decent cull times at far cascades.
  • Adding a "internal only" shadow flag for aircraft developers. This allows farther shadow cascades to cull complex objects that are only visible in the nearest cascades. (Very important optimization for aircraft with complex cockpit geometry).
  • Vegetation shadows will be done by the "legacy" method currently in use. Shadow mapping on vegetation is much more expensive in terms of performance and the current algorithm does the job well enough. [6]

Post-processing

Gamma correction, night vision and other ALS filters should happen in a quad pass. The current filter_combined() should be left for post-processing that requires as much precision as possible - e.g. dithering to prevent banding). HDR is not a planned feature for now so ALS will be using rgba8 buffers for most of its features.

Real-time dynamic reflections

Rendering dynamically to a cubemap is possible. As with shadow mapping, minimizing the object count and number of forward passes is vital to get good performance in FlightGear. Rendering to six cubemap faces requires six forward passes, but we can render to a dual paraboloid map instead, reducing this number to two.

Transparency

When shadows (and multipass rendering in general) come into play, transparent objects have to be treated differently, even when we are dealing with a forward renderer. In OSG there are two ways to separate transparent surfaces:

  • Using RenderBins. After a single scene cull traversal, surfaces which belong to a special RenderBin type (DepthSortedBin) are removed or moved to another camera. This is how Rembrandt does it and it is the most backwards compatible approach since RenderBins can be changed directly inside Effects.
  • Using cull masks. Two separate traversals are done: one for opaque objects and another for translucent objects. This requires offering aircraft developers another way of tagging a surface as transparent. A trivial approach would be to add a new <animation> type called 'transparent', but that wouldn't be backwards compatible. Maybe we can add some kind of system where we can change cull masks inside Effects? Would that be too hacky or out of place?

Creating a custom rendering pipeline

Since the Compositor is completely data-driven, new rendering pipelines can be created by writing a custom XML pipeline definition. This section tries to document most of the available parameters, but the best and most up-to-date resource is the Compositor parsing code in SimGear (simgear/simgear/scene/viewer). See existing pipelines in fgdata/Compositor for practical examples on how to use these parameters.

Buffers

A buffer represents a texture or, more generically, a region of GPU memory.

Parameter Name Optional Value Default Value Description
name No string Passes will be able to address the buffer by this name
type No 1d, 2d, 2d-array, 2d-multisample, 3d, rect, cubemap Any texture type allowed by OpenGL
width No Any unsigned integer or screen to use the physical viewport width. The <property> tag can also be used to use a property value Texture width
screen-width-scale Yes float 1.0 If screen was used, this controls the width scaling factor
height No Any unsigned integer or screen to use the physical viewport height. The <property> tag can also be used to use a property value Texture height
screen-height-scale Yes float 1.0 If screen was used, this controls the height scaling factor
depth No Any unsigned integer. The <property> tag can also be used to use a property value Texture depth
format Yes See simgear/simgear/scene/viewer/CompositorBuffer.cxx for the latest available values rgba8 Specifies the texture format. It corresponds to the internalformat, format and type arguments of the OpenGL function glTexImage2D
min-filter, mag-filter Yes linear, linear-mipmap-linear, linear-mipmap-nearest, nearest, nearest-mipmap-linear, nearest-mipmap-nearest linear Change the minification and magnification filtering respectively
wrap-s, wrap-t, wrap-r Yes clamp, clamp-to-edge, clamp-to-border, repeat, mirror clamp-to-border They change the wrap mode for each coordinate
anisotropy Yes float 1.0
border-color Yes vec4 (0.0f, 0.0f, 0.0f, 0.0f)
shadow-comparison Yes bool true
shadow-texture-mode Yes luminance, intensity, alpha luminance
shadow-compare-func Yes never, less, equal, lequal, greater, notequal, gequal, always lequal

Passes

A pass wraps around an osg::Camera. Passes all have some common parameters:

Parameter Name Optional Value Default Value Description
clear-color, clear-accum, clear-depth and clear-stencil Yes vec4 black, black, 1.0, 0 respectively Pass clear colors
clear-mask Yes color, stencil, depth, accum color depth Pass clear mask
effect-scheme Yes Valid effect scheme name None The pass will try to use the specified effect scheme to draw every object.

Passes can render to a buffer (Render to Texture), to several buffers (Multiple Render Targets) or directly to the framebuffer. This is accomplished by the <attachment> tag. Possible parameters of an attachment are:

Parameter Name Optional Value Default Value Description
buffer No Valid buffer name The name of the buffer to output to
component No color, color0 to color15, depth, stencil, depth-stencil FBO attachment point
level Yes int 0 Mipmap level of the texture that is attached
face Yes int 0 Face of cube map texture or z-level of 3d texture
mipmap-generation Yes bool false Whether mipmap generation should be done for texture
multisample-samples Yes int 0 Multisample anti-aliasing (MSAA) samples
multisample-color-samples Yes int 0 Multisample anti-aliasing (MSAA) color samples

Passes can also receive buffers as input and use them in their shaders. This is accomplished by the <binding> tag, which has the following parameters:

Parameter Name Optional Value Default Value Description
buffer No Valid buffer name The name of the buffer to bind
unit No int The texture unit to place the texture on. Effects will be able to access the buffer on this texture unit

There are specific pass types, each with their own set of custom parameters.

scene

Renders the scene from the point of view given by the CameraGroup.

Parameter Name Optional Value Default Value Description
cull-mask Yes A 32 bit number. See simgear/simgear/scene/util/RenderConstants.hxx to know which bits enable what 0xffffffff Specifies the cull mask to be used in the underlying osg::Camera
z-near, z-far Yes float 0.0 uses the value given by the CameraGroup Sets a custom near and far values. Useful for implementing depth partition and limiting the depth range on cubemap passes
cubemap-face Yes int -1 (don't use cubemap) Ignores the given view and projection matrices and uses a custom one that renders the scene as if it was seen from inside a cubemap looking towards the specified face
use-shadow-pass Yes string Empty Name of a shadow mapping pass. Exposes shadow mapping related uniforms to the shaders of the current pass

Scene passes can also use the tag <clustered-shading> to enable clustered shading (lights). The following parameters are available:

Parameter Name Optional Value Default Value Description
tile-size Yes int 128 Size of each clustered shading tile
num-threads Yes int 1 Number of threads to use during the light culling process. Keep in mind that a high thread count when there aren't many lights will worsen performance due to the thread creation overhead
depth-slices Yes int 1 Number of slices to partition the view frustum in the Z axis. Higher numbers will cull lights more aggressively, increasing performance if there are many lights further out that don't contribute much to the overall scene's lighting

quad

Renders a fullscreen quad with an optional effect applied. Useful for screen space shaders (like SSAO, Screen Space Reflections or bloom) and deferred rendering.

Parameter Name Optional Value Default Value Description
geometry Yes float values for <x>, <y>, <width>, <height> 0.0, 0.0, 1.0, 1.0 respectively Size of the fullscreen quad inside the viewport using normalized coordinates.
effect Yes Valid Effect file None This Effect will be applied to the quad geometry

shadow-map

Renders the scene from a light's point of view.

Parameter Name Optional Value Default Value Description
light-name No Valid light name that exists in the scene graph The name of the osg::LightSource to use for this shadow map
near-m, far-m No float (meters) They specify the depth range of the shadow map

TODO

  • Bring back distortion correction.
  • Some kind of versioning system to be able to make breaking changes in the future if/when the compositor is updated in any significant way, without people having to manually update their configs.
  • Rework node masks (simgear/simgear/scene/util/RenderConstants.hxx). Each node type should have its own node mask bit, e.g. trees should only have TREE_BIT enabled, the main aircraft should only have AIRCRAFT_BIT enabled etc. This allows the Compositor cameras to properly filter scene graph nodes based on the cull mask. Shadow cameras can then choose what geometry they render dynamically.
  • Automatically calculate light source attenuation based on radius and radius based on attenuation.
  • Add clustered shading to every Effect as a separate technique.
  • Proper shader quality settings for both low-spec and ALS pipelines.
  • Launcher and UI:
  • Separate cmake build target (binary) for a "preview" binary with the Compositor enabled to get more testing/feedback and contributions from early-adopters [7] [8] [9] [10] 90}% completed

Known Issues

  • Setting a buffer scale factor different from 1.0 and rendering to it might not scale the splash screen correctly.
  • There is some kind of moiré pattern at certain sunlight angles (specially at dusk/dawn). Shadow acne.
  • Spotlights sometimes disappear at certain view angles.
  • Some effects might not be ported to ALS completely. Feel free to play around with the shader settings to get something that works.
  • CRT effect as a default Effect. Look into reflect.eff
  • osgText doesn't work. See the 777 displays as well as some EC135 instruments. osgText doesn't use Effects so they are being rendered with incorrect depth.
  • Reloading shaders doesn't work.
  • Random buildings don't use correct depth.

References

References

Related content

Wiki articles

Forum topics