Compositor

From FlightGear wiki
Revision as of 16:56, 2 January 2019 by Icecode (talk | contribs)
Jump to navigation Jump to search
This article describes content/features that may not yet be available in the latest stable version of FlightGear (2020.3).
You may need to install some extra components, use the latest development (Git) version or even rebuild FlightGear from source, possibly from a custom topic branch using special build settings: .

This feature is scheduled for FlightGear 2019.3. 80}% completed

If you'd like to learn more about getting your own ideas into FlightGear, check out Implementing new features for FlightGear.


Compositor Subsystem
Clouds experiment.jpg
Started in 01/2018
Description Dynamic rendering pipeline configured via the property tree
Maintainer(s) none
Contributor(s) Icecode
Status
  • experimental as of 02/2018
  • put up for review/discussion as of 08/2018
  • pending review [1]
Topic branches:
$FG_SRC https://sourceforge.net/u/fgarlin/flightgear-src/ci/new-compositor/tree/
fgdata https://sourceforge.net/u/fgarlin/flightgear/ci/new-compositor/tree/

The Compositor aims to bring multi-pass rendering to FlightGear. It encapsulates a rendering pipeline and exposes its parameters to the Property Tree. The pipeline used on each physical viewport defined on the CameraGroup settings can be changed via a <compositor/> tag containing a valid path to the Compositor XML file.

Background

First discussed in 03/2012 during the early Rembrandt days, Zan (Lauri Peltonen) came up with a set of patches demonstrating how to create an XML-configurable rendering pipeline.

Back then, this work was considered to look pretty promising [2] and at the time plans were discussed to unify this with the ongoing Rembrandt implementation (no longer maintained).

Adopting Zan's approach would have meant that efforts like Rembrandt (deferred rendering) could have been implemented without requiring C++ space modifications.

Rembrandt's developer (FredB) suggested to extend the format to avoid duplicating the stages when you have more than one viewport, i.e. specifying a pipeline as a template, with conditions like in effects, and have the current camera layout refer the pipeline that would be duplicated, resized and positioned for each declared viewport [3]


Zan's original patches can still be found in his newcameras branches which allow the user to define the rendering pipeline in preferences.xml:

At that point, it didn't have everything Rembrandt's pipeline needs, but most likely could be easily enhanced to support those things.

Basically the original version added support for multiple camera passes, texture targets, texture formats, passing textures from one pass to another etc, while preserving the standard rendering line if user wants that. [4]

Status

01/2019

The corresponding set of patches (topic branches) were put up for review in 08/2018 to discuss the underlying approach and hopefully get this merged in 2019. [5]

Features

  • Completely independent of other parts of the simulator, i.e. it's part of SimGear and can be used in a standalone fashion if needed, ala Canvas.
  • Although independent, its aim is to be fully compatible with the current rendering framework in FG. This includes the Effects system, CameraGroup, Rembrandt and ALS (and obviously the Canvas).
  • Its functionality overlaps Rembrandt: what can be done with Rembrandt can be done with the Compositor, but not vice versa.
  • Fully configurable via a XML interface without compromising performance (ala Effects, using PropertyList files).
  • Flexible, expandable and compatible with modern graphics.
  • It doesn't increase the hardware requirements, it expands the hardware range FG can run on. People with integrated GPUs (Intel HD etc) can run a Compositor with a single pass that renders directly to the screen like before, while people with more powerful cards can run a Compositor that implements deferred rendering, for example.

The Compositor is in an usable state right now: it works but there are no effects or pipelines developed for it. There are also some bugs and features that don't work as expected because of some hardcoded assumptions in the FlightGear Viewer code.

Elements

Buffers

A buffer represents a texture or, more generically, a region of GPU memory. It can have the following parameters:

name
Passes will be able to address the buffer by this name.
type
Any texture type allowed by OpenGL: 1d, 2d, 2d-array, 2d-multisample, 3d, rect or cubemap.
width
Texture width. It's possible to write 'screen' to use the physical viewport width.
screen-width-scale
If 'screen' was used, this controls the width scaling factor.
height
Texture height. It's possible to write 'screen' to use the physical viewport height.
screen-height-scale
If 'screen' was used, this controls the height scaling factor.
depth
Texture depth.
internal-format, source-format and source-type
They specify the texture format. They correspond to the arguments internalformat, format and type respectively of the OpenGL function glTexImage2D.
min-filter and mag-filter (Optional)
They change the minification and magnification filtering respectively. Possible values are: linear, linear-mipmap-linear, linear-mipmap-nearest, nearest, nearest-mipmap-linear and nearest-mipmap-nearest. The default value for both filters is linear.
wrap-s, wrap-t and wrap-r (Optional)
They change the wrap mode for each coordinate. Possible values are: clamp, clamp-to-edge, clamp-to-border, repeat and mirror. The default value for every coordinate is clamp-to-border.

A typical property tree structure describing a buffer may be as follows:

<buffer>
  <name>color</name>
  <type>2d</type>
  <width>screen</width>
  <height>screen</height>
  <screen-width-scale>1.5</screen-width-scale>
  <screen-height-scale>1.5</screen-height-scale>
  <internal-format>rgba8</internal-format>
  <source-format>rgba</source-format>
  <source-type>ubyte</source-type>
  <min-filter>nearest-mipmap-nearest</min-filter>
  <mag-filter>nearest-mipmap-nearest</mag-filter> 
  <wrap-s>repeat</wrap-s>
  <wrap-t>repeat</wrap-t>
</buffer>

Passes

A pass wraps around an osg::Camera. The following types of passes are supported:

scene
Renders the scene from the point of view given by the CameraGroup.
shadow-map
Renders the scene from a light's point of view.
quad
Renders a fullscreen quad with an optional effect applied. Useful for screen space shaders (like SSAO, Screen Space Reflections or bloom) and deferred rendering.

Passes can render to a buffer (Render to Texture), to several buffers (Multiple Render Targets) or directly to the framebuffer. This is accomplished by the <attachment/> tag. Possible parameters of an attachment are:

buffer
The name of the buffer to output to.
component
FBO attachment point. Possible values are color0 to color15, depth, stencil and depth-stencil.
level (Optional)
Controls the mip map level of the texture that is attached. Default value is 0.
face (Optional)
Controls the face of texture cube map or z level of 3d texture. Default value is 0.
mipmap-generation (Optional)
Controls whether mipmap generation should be done for texture. Default value is false.
multisample-samples (Optional)

MSAA samples. Default value is 0.

multisample-color-samples (Optional)

MSAA color samples. Default value is 0.

Passes can also receive buffers as input and use them in their shaders. This is accomplished by the <binding/> tag, which has the following parameters:

buffer
The name of the buffer to bind.
unit
The texture unit to place the texture on. Effects will be able to access the buffer on this texture unit.

Example XML for a scene type pass:

<pass>
  <name>forward-lighting</name>
  <type>scene</type>
  <clear-color type="vec4d">0 0 0 0</clear-color>

  <clustered-forward/>

  <binding>
    <buffer>shadowmap</buffer>
    <unit>10</unit>
  <attachment>
    <buffer>color</buffer>
    <component>color0</component>
    <multisample-samples>4</multisample-samples>
    <multisample-color-samples>4</multisample-color-samples>
  </attachment>
  <attachment>
    <buffer>depth</buffer>
    <component>depth</component>
  </attachment>
</pass>

Canvas integration

Post processed image (right) after applying gaussian blur and a blue filter to a scene pass (left).

Apart from serving as a debugging tool for visualizing the contents of a buffer, integrating the Compositor with Canvas allows aircraft developers to access RTT capabilities. Compositor buffers can be accessed within Canvas via a new custom Canvas Image protocol buffer://. For example, the path buffer://test-compositor/test-buffer displays the buffer test-buffer declared in test-compositor.

var (width,height) = (612,612);
var title = 'Compositor&Canvas test';
var window = canvas.Window.new([width,height],"dialog")
 .set('title',title);
var myCanvas = window.createCanvas().set("background", canvas.style.getColor("bg_color"));
var root = myCanvas.createGroup();
var path = "buffer://test-compositor/test-buffer";
var child = root.createChild("image")
    .setTranslation(50, 50)
    .setSize(512, 512)
    .setFile(path);

TODO

  • Bring back distortion correction.
  • Finish Clustered Forward Rendering.
  • Rewrite the uniform system (maybe integrating with Effects?).
  • Find a way to override an entire pass stateset with an Effect.
  • Some kind of versioning system to be able to make breaking changes in the future if/when the compositor is updated in any significant way, without people having to manually update their configs.

Known Issues

  • Setting a buffer scale factor different from 1.0 and rendering to it might not scale the splash screen correctly.
  • No error handling when the Compositor wasn't found. FlightGear should quit gracefully but right now it segfaults.
  • Hardcoded Z handling. Better handling of depth is required to allow, for example, near and far cameras to prevent Z-fighting.
  • computeIntersections() always uses pass 0, which might fail. The scene pass should be used (which one if there are several?).

Related

References