Compositor
This article describes content/features that may not yet be available in the latest stable version of FlightGear (2020.3). You may need to install some extra components, use the latest development (Git) version or even rebuild FlightGear from source, possibly from a custom topic branch using special build settings: -DENABLE_COMPOSITOR=ON .This feature is scheduled for FlightGear 2019.2. If you'd like to learn more about getting your own ideas into FlightGear, check out Implementing new features for FlightGear. |
The FlightGear forum has a subforum related to: Effects & Shaders |
Started in | 01/2018 (Available since FlightGear 2019.2) |
---|---|
Description | Dynamic rendering pipeline configured via the property tree and XML |
Contributor(s) | Fernando García Liñán [1] |
Status | Stable |
Folders | |
Changelog | https://sourceforge.net/u/fgarlin/profile/feed.rss |
The Compositor aims to bring multi-pass rendering to FlightGear. It encapsulates a rendering pipeline and exposes its parameters to a Property Tree interface. At startup, FlightGear reads the pipeline definition file for each physical viewport defined on the CameraGroup settings. If no Compositor file is specified for a physical camera, the one given by the --compositor=
startup command will be used. If such startup option is not used either, FlightGear will look for a valid Compositor file in $FG_ROOT/Compositor/default.xml
The Compositor introduces a new dedicated fgdata directory for new/custom rendering pipelines: fgdata/Compositor.
Background
First discussed in 03/2012 during the early Rembrandt days, Zan (Lauri Peltonen) came up with a set of patches demonstrating how to create an XML-configurable rendering pipeline.
Back then, this work was considered to look pretty promising [2] and at the time plans were discussed to unify this with the ongoing Rembrandt implementation (no longer maintained).
Adopting Zan's approach would have meant that efforts like Rembrandt (deferred rendering) could have been implemented without requiring C++ space modifications, i.e. purely in Base package space.
Rembrandt's developer (FredB) suggested to extend the format to avoid duplicating the stages when you have more than one viewport, i.e. specifying a pipeline as a template, with conditions like in effects, and have the current camera layout refer the pipeline that would be duplicated, resized and positioned for each declared viewport [3]
Zan's original patches can still be found in his newcameras branches which allow the user to define the rendering pipeline in preferences.xml: FlightGear, SimGear.
At that point, it didn't have everything Rembrandt's pipeline needs, but most likely could be easily enhanced to support those things.
Basically, the original version added support for multiple camera passes, texture targets, texture formats, passing textures from one pass to another etc, while preserving the standard rendering line if user wants that. [4]
Since the early days of Zan's groundwork, providing the (hooks) infrastructure to enable base package developers to prototype, test and develop distinct rendering pipelines without requiring C++ space modifications has been a long-standing idea, especially after the Canvas system became available in early 2012, which demonstrated how RTT-rendering buffers (FBOs) could be set up, created and manipulated procedurally (i.e. at run-time) using XML, the property tree and Nasal scripting. [5]
The new Compositor is an improved re-implementation of Zan's original work using not just XML, but also properties and a handful of Canvas concepts.
Features
- Completely independent of other parts of the simulator, i.e. it's part of SimGear and can be used in a standalone fashion if needed, ala Canvas.
- Although independent, its aim is to be fully compatible with the current rendering framework in FG. This includes the Effects system, CameraGroup, Rembrandt and ALS (and obviously the Canvas).
- Its functionality overlaps Rembrandt: what can be done with Rembrandt can be done with the Compositor, but not vice versa.
- Fully configurable via an XML interface without compromising performance (ala Effects, using PropertyList files).
- Flexible, expandable and compatible with modern graphics.
- It doesn't increase the hardware requirements, it expands the hardware range FG can run on. People with integrated GPUs (Intel HD etc) can run a Compositor with a single pass that renders directly to the screen like before, while people with more powerful cards can run a Compositor that implements deferred rendering, for example.
- Static branching support. Every pipeline element can be enabled/disabled at startup via a <condition> block.
How to enable the Compositor
Currently the Compositor can only be enabled at compile time via the -DENABLE_COMPOSITOR=ON
CMake flag in FlightGear. SimGear doesn't require any extra parameters. Once you have a binary with the Compositor enabled and you run it, you will be presented with the default rendering pipeline. At the time of writing, this is the low spec rendering pipeline. If you want to try the ALS pipeline, start FlightGear with the command line argument: --compositor=Compositor/ALS/als
If you want to enable shadows on all objects in the ALS pipeline use these options as a startup parameters (in QT GUI or in the commandline) --prop:bool:/sim/rendering/als/shadows/enabled=true
and --prop:int:/sim/rendering/als/shadows/sun-atlas-size=2048
. If you feel like the shadows are too low-quality (specially in the cockpit), increase the shadow resolution to 4096 or 8192 instead of 2048.
If you are having trouble with lights using an integrated GPU and the Mesa drivers under Linux, try starting FlightGear with the environment variable MESA_GL_VERSION_OVERRIDE="3.1COMPAT"
Notes for aircraft developers
Lights
The Compositor introduces a new way of defining lights that is renderer agnostic, so every rendering pipeline will be able to access the lights that have been implemented like this. As of 2019/11, the only pipeline that supports dynamic lights is the ALS pipeline. The resulting light volumes can be visualized for debugging purposes by setting the property /sim/debug/show-light-volumes to true.
<light>
<name>my-spotlight</name>
<type>spot</type>
<position>
<x-m>-7.7476</x-m>
<y-m>0</y-m>
<z-m>-1.7990</z-m>
</position>
<direction>
<x>-1.0</x>
<y>0</y>
<z>-0.013</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
<b>0.03</b>
<a>1</a>
</ambient>
<diffuse>
<r>0.95</r>
<g>0.9</g>
<b>0.9</b>
<a>1</a>
</diffuse>
<specular>
<r>0.95</r>
<g>0.9</g>
<b>0.9</b>
<a>1</a>
</specular>
<attenuation>
<c>1.0</c>
<l>0.09</l>
<q>0.032</q>
</attenuation>
<spot-exponent>5</spot-exponent>
<spot-cutoff>40</spot-cutoff>
<range-m>50</range-m>
</light>
|
|
Shadows
The shadow mapping algorithm can be customized entirely by the rendering pipeline. This means that each one will have its own requirements when it comes to shadows. Here are some general recommendations:
- Use the
<noshadow>
animation to disable shadows on objects that don't need them. An example would be billboarded lights or really small cockpit elements that don't need shadows and would cause degraded performance. - Try to mark as many cockpit objects as possible as interior.
<model>
<name>interior</name>
<usage>interior</usage>
<path>Aircraft/JA37/Models/ja37-interior.xml</path> <!-- All the objects that should only be seen when inside the cockpit are in this file -->
</model>
- Unlike in Rembrandt, polygons facing the Sun are the ones used to generate the shadow map, so single sided surfaces and non-closed objects should be rendered correctly.
Pipelines
Low-Spec pipeline
A fixed function forward rendering pipeline mainly targeted to low spec systems. It imitates the classic forward pipeline used before multi-pass rendering was introduced by using two near/far cameras rendering directly to the screen.
ALS
The ALS pipeline tries to bring multipass rendering to the current ALS framework, effectively combining the best from ALS and Project Rembrandt.
Cascaded shadow mapping
The main issue with shadow mapping in FlightGear is the complexity of the scene graph. Culling times can become huge if we don't carefully select which parts of the scene graph we want to render in the shadow maps. Some possible optimizations:
- Study the minimum shadow map distance we can get without noticeable light leaking. Select an appropiate amount of cascades (more cascades = more passes over all geometry, and in general we want to keep the amount of forward passes to a minimum). We should have at least three cascades: the first just for cockpit/internal shadows, the second for the whole aircraft and the third for the rest of the scenery geometry. A fourth can be added if the transition between the second and the third is too harsh.
- Improve the culling masks (simgear/scene/util/RenderConstants.hxx). The CASTSHADOW_BIT flag is present in almost every object in the scene graph. Turning this flag off for trees, random buildings and other geometry intensive objects improves framerates by a very considerable amount. Should the user be able to select which objects cast shadows?
- Should the terrain cast shadows? The terrain is rarely steep enough to cast shadows. Apart from that, the terrain in FlightGear messes with automatic near/far computations for the shadow passes since the geometry is not tessellated enough. Also, the terrain LOD is not good enough to have decent cull times at far cascades.
- Adding a "internal only" shadow flag for aircraft developers. This allows farther shadow cascades to cull complex objects that are only visible in the nearest cascades. (Very important optimization for aircrafts with complex cockpit geometry).
- Vegetation shadows will be done by the "legacy" method currently in use. Shadow mapping on vegetation is much more expensive in terms of performance and the current algorithm does the job well enough. [6]
Post-processing
Gamma correction, night vision and other ALS filters should happen in a quad pass. The current filter_combined() should be left for post-processing that requires as much precision as possible - e.g. dithering to prevent banding). HDR is not a planned feature for now so ALS will be using rgba8 buffers for most of its features.
Real-time dynamic reflections
Rendering dynamically to a cubemap is possible. As with shadow mapping, minimizing the object count and number of forward passes is vital to get good performance in FlightGear. Rendering to six cubemap faces requires six forward passes, but we can render to a dual paraboloid map instead, reducing this number to two.
Transparency
When shadows (and multipass rendering in general) come into play, transparent objects have to be treated differently, even when we are dealing with a forward renderer. In OSG there are two ways to separate transparent surfaces:
- Using RenderBins. After a single scene cull traversal, surfaces which belong to a special RenderBin type (DepthSortedBin) are removed or moved to another camera. This is how Rembrandt does it and it is the most backwards compatible approach since RenderBins can be changed directly inside Effects.
- Using cull masks. Two separate traversals are done: one for opaque objects and another for translucent objects. This requires offering aircraft developers another way of tagging a surface as transparent. A trivial approach would be to add a new <animation> type called 'transparent', but that wouldn't be backwards compatible. Maybe we can add some kind of system where we can change cull masks inside Effects? Would that be too hacky or out of place?
Creating a custom rendering pipeline
Since the Compositor is completely data-driven, new rendering pipelines can be created by writing a custom XML pipeline definition. This section tries to document most of the available parameters, but the best and most up-to-date resource is the Compositor parsing code in SimGear (simgear/simgear/scene/viewer). See existing pipelines in fgdata/Compositor for practical examples on how to use these parameters.
Buffers
A buffer represents a texture or, more generically, a region of GPU memory.
Parameter Name | Optional | Value | Default Value | Description |
---|---|---|---|---|
name | No | string | Passes will be able to address the buffer by this name | |
type | No | 1d, 2d, 2d-array, 2d-multisample, 3d, rect, cubemap | Any texture type allowed by OpenGL | |
width | No | Any unsigned integer or screen to use the physical viewport width. The <property> tag can also be used to use a property value
|
Texture width | |
screen-width-scale | Yes | float | 1.0 | If screen was used, this controls the width scaling factor |
height | No | Any unsigned integer or screen to use the physical viewport height. The <property> tag can also be used to use a property value
|
Texture height | |
screen-height-scale | Yes | float | 1.0 | If screen was used, this controls the height scaling factor |
depth | No | Any unsigned integer. The <property> tag can also be used to use a property value
|
Texture depth | |
format | Yes | See simgear/simgear/scene/viewer/CompositorBuffer.cxx for the latest available values | rgba8 | Specifies the texture format. It corresponds to the internalformat, format and type arguments of the OpenGL function glTexImage2D |
min-filter, mag-filter | Yes | linear, linear-mipmap-linear, linear-mipmap-nearest, nearest, nearest-mipmap-linear, nearest-mipmap-nearest | linear | Change the minification and magnification filtering respectively |
wrap-s, wrap-t, wrap-r | Yes | clamp, clamp-to-edge, clamp-to-border, repeat, mirror | clamp-to-border | They change the wrap mode for each coordinate |
anisotropy | Yes | float | 1.0 | |
border-color | Yes | vec4 | (0.0f, 0.0f, 0.0f, 0.0f) | |
shadow-comparison | Yes | bool | true | |
shadow-texture-mode | Yes | luminance, intensity, alpha | luminance | |
shadow-compare-func | Yes | never, less, equal, lequal, greater, notequal, gequal, always | lequal |
Passes
A pass wraps around an osg::Camera. Passes all have some common parameters:
Parameter Name | Optional | Value | Default Value | Description |
---|---|---|---|---|
clear-color, clear-accum, clear-depth and clear-stencil | Yes | vec4 | black, black, 1.0, 0 respectively | Pass clear colors |
clear-mask | Yes | color, stencil, depth, accum | color depth | Pass clear mask |
effect-scheme | Yes | Valid effect scheme name | None | The pass will try to use the specified effect scheme to draw every object. |
Passes can render to a buffer (Render to Texture), to several buffers (Multiple Render Targets) or directly to the framebuffer. This is accomplished by the <attachment>
tag. Possible parameters of an attachment are:
Parameter Name | Optional | Value | Default Value | Description |
---|---|---|---|---|
buffer | No | Valid buffer name | The name of the buffer to output to | |
component | No | color, color0 to color15, depth, stencil, depth-stencil | FBO attachment point | |
level | Yes | int | 0 | Mipmap level of the texture that is attached |
face | Yes | int | 0 | Face of cube map texture or z-level of 3d texture |
mipmap-generation | Yes | bool | false | Whether mipmap generation should be done for texture |
multisample-samples | Yes | int | 0 | Multisample anti-aliasing (MSAA) samples |
multisample-color-samples | Yes | int | 0 | Multisample anti-aliasing (MSAA) color samples |
Passes can also receive buffers as input and use them in their shaders. This is accomplished by the <binding>
tag, which has the following parameters:
Parameter Name | Optional | Value | Default Value | Description |
---|---|---|---|---|
buffer | No | Valid buffer name | The name of the buffer to bind | |
unit | No | int | The texture unit to place the texture on. Effects will be able to access the buffer on this texture unit |
There are specific pass types, each with their own set of custom parameters.
scene
Renders the scene from the point of view given by the CameraGroup.
Parameter Name | Optional | Value | Default Value | Description |
---|---|---|---|---|
cull-mask | Yes | A 32 bit number. See simgear/simgear/scene/util/RenderConstants.hxx to know which bits enable what | 0xffffffff | Specifies the cull mask to be used in the underlying osg::Camera |
z-near, z-far | Yes | int | Default Z range in the CameraGroup | They change the depth range to be used |
cubemap-face | Yes | int | -1 (don't use cubemap) | Ignores the given view and projection matrices and uses a custom one that renders the scene as if it was seen from inside a cubemap looking towards the specified face |
quad
Renders a fullscreen quad with an optional effect applied. Useful for screen space shaders (like SSAO, Screen Space Reflections or bloom) and deferred rendering.
Parameter Name | Optional | Value | Default Value | Description |
---|---|---|---|---|
geometry | Yes | float values for <x>, <y>, <width>, <height>
|
0.0, 0.0, 1.0, 1.0 respectively | Size of the fullscreen quad inside the viewport using normalized coordinates. |
effect | Yes | Valid Effect file | None | This Effect will be applied to the quad geometry |
shadow-map
Renders the scene from a light's point of view.
Parameter Name | Optional | Value | Default Value | Description |
---|---|---|---|---|
light-name | No | Valid light name that exists in the scene graph | The name of the osg::LightSource to use for this shadow map | |
near-m, far-m | No | Valid Effect file | They specify the depth range of the shadow map |
TODO
- Bring back distortion correction.
- Some kind of versioning system to be able to make breaking changes in the future if/when the compositor is updated in any significant way, without people having to manually update their configs.
- Bring back Canvas integration so aircraft devs have access to the rendering pipeline. This allows to render exterior views in cockpit displays etc.
- Automatically calculate light source attenuation based on radius and radius based on attenuation.
- Add 1-bit transparency to shadow mapping (maybe even full blown transparency with multiple depth buffers?).
Known Issues
- Setting a buffer scale factor different from 1.0 and rendering to it might not scale the splash screen correctly.
- Clustered shading crashes FG if compiled under OSG 3.6. This is related to osg::TextureBuffer changing definition from OSG 3.4 to OSG 3.6 (Images vs BufferData).
- Relative path Effects in aircrafts don't work. This is not permanent, they will work again once the Compositor replaces the legacy renderer.
- Earthview doesn't work.
- There is some kind of moiré pattern at certain sunlight angles (specially at dusk/dawn).
- hud.eff doesn't work under the ALS pipeline.
- Spotlights sometimes disappear at certain view angles.
- Vegetation appears to flicker at long distances.
References
References
|
Related content
Wiki articles
- Uniform Buffer Objects
- CompositeViewer Support
- FlightGear CIGI Support (Common Image Generator Interface)