Encapsulating rendering code using Canvas (RFC)

Work in progress This article or section will be worked on in the upcoming hours or days. See history for the latest developments.

Objective

Discuss if/how the Canvas system and its architecture can be used to help encapsulate rendering related OpenGL/OSG code in FlightGear with a focus on supporting distributed setups using HLA to make better use of multicore/multi-gpu platforms.

Background

It's been 3 years since the addition of the Canvas system in FlightGear, and it is becoming increasingly clear that the Canvas system provides a number of unprecedented opportunities for unifying the 2D rendering back-end, i.e. features like the built-in GUI, the old HUD and 2D panels code, as well as other legacy code like the splash screen routines.

So far, it's exactly this legacy code implementing these features that is preventing FlightGear from using a more recent version of OpenGL, because it is usually old code that was written with a fixed function OpenGL pipeline in mind, which is interfering with more recent additions using shaders/effects.

The Canvas system can help modernize these features by re-implementing them in scripting space which is mapped onto Canvas properties (elements) that are mapped to their corresponding OSG equivalents.

Originally, much of the discussion revolving around unifying the rendering back-end in FlightGear was about the 2D back-end in particular (GUI, HUDs, 2D panels) to get rid of this legacy code - meanwhile, it is becoming increasingly obvious, that Canvas can help us identify and unify other legacy code that is de-facto unmaintained.

Canvas Concepts

In its current form, the Canvas system can be understood to be the logical evolution of a number of related features that were added to FlightGear over time, namely:

• RTT features like the Map (GUI) and NavDisplay (cockpits) rendering to an offscreen texture that is replaced using the visitor design pattern
• SGSubsystems mapped to the property tree using SGPropertyChangeListener
• Property events mapped to C++ methods using propertyObjects
• $FG_ROOT/Docs/README.osgtext

The Canvas system generalizes these use-cases by coming up with generic building blocks that can be parameterized and customized at run-time, primarily using the following concepts:

• an OpenGL texture (OSG FBO)
• a placement of the texture in the scene (cockpit/aircraft, PUI dialog, native OSG window, scenery)
• a top-level/root group with custom elements that provide general-purpose functionality that can be combined with other primitives to model a scene graph using the property tree
• GUI related events (mouse/keyboard and joystick) for interacting with a Canvas element

How Canvas works

Fundamentally, a Canvas node in the property tree merely represents an offscreen rendering context using OSG helpers which are internally using RTT/FBOs for the texture mapped to properties, and it turns out that this is an extremely common application in FlightGear, i.e. textures that are procedurally created/updated and rendered in response to properties/events (more or less directly as we will see below).

Under the hood, the main building blocks of a canvas are therefore the texture itself, as well as an arbitrary number of "placements" in the virtual FlightGear world/scene.

Internally, each Canvas (FBO/texture) consists of a top-level "group" element which is basically the mechanism used for structuring an OSG scenegraph on top of the FlightGear property tree (analogous to a folder in a disk system).

In other words, people can create nodes in a texture-specific sub-tree of the FlightGear property tree to create/update well-defined events (properties), which are internally mapped to callbacks (using listeners), so that different elements are created/updated and removed, with each element supporting an arbitrary number of element specific operating modes and attributes (events).

To make all this more convenient, certain methods are exposed to scripting space using the cppbind framework, as well as a few scripting space wrappers and higher-level frameworks for specific use-cases (e.g. MFDs, HUDs, PFDs, NDs etc).

At the time of writing, the main building blocks for any Canvas based feature are simply:

• osg::Text nodes (fonts)
• osg::Image nodes (used for raster images)
• Shiva/OpenVG nodes (also used for vector images)
• Maps (can be used to directly project child nodes using lat/lon nodes)

With all of these inheriting from a common Canvas::Element parent class that provides a common interface for any shared functionality.

And with higher-level elements inheriting from a matching parent class (e.g. CanvasWindow inheriting from CanvasImage).

Offscreen rendering in FlightGear

With this introduction in mind, we can see immediately that a Canvas merely represents an FBO that can have an arbitrary number of child elements, which are usually used to implement higher-level features.

Equally, the separation of a virtual placement and underlying elements maps nicely onto pretty much all existing use-cases in FlightGear, including much of the code that predates Canvas by years or even a whole decade.

For instance, all existing hard-coded gauges (agradar, navdisplay, wxradar) using the ODGauge helper could be easily re-implemented in Canvas space, with the added benefit of being nicely encapsulated and using modern OSG code.

In addition, even much of the Rembrandt initialization code is dealing with the setup of rendering buffers, aka "FBOs" (offscreen textures), that are used by Rembrandt for different rendering stages.

It is worth keeping in mind that this would not necessarily introduce much/any Nasal dependencies, because a Canvas could just as well be set up by loading an XML file into the property tree using the include directive in a PropertyList file.

In other words, most of the hard-coded, and unmaintained, initialization code dealing with buffer setup in Rembrandt could be easily re-implemented in XML/fgdata space by using a Canvas texture as the underlying abstraction layer.

About placements

It also turns out that the notion of a virtual "placement" for an arbirary RTT/FBO texture maps nicely to other use-cases in FlightGear - for instance, the CameraGroup code in $FG_SRC/Viewer/CameraGroup.cxx deals with the setup of dedicated OS windows, and splitting up each window into "views" using an osg::TextureRectangle.

Thus, should Canvas -some day- support placements to native OS windows, the underlying CameraGroup code could also be greatly simplified, with the added benefit of becoming fully runtime configurable, i.e. allowing windows to be created/removed arbitrarily.

Redundant legacy code

Since the adoption of Canvas, a number of legacy features have become obsolete and could be re-implemented on top of Canvas with the added benefit of unifying, and minimizing, the underlying C++ code - for example:

• NavDisplay
• Map (and any custom PUI widgets)
• $FG_ROOT/Docs/README.osgtext

Shortcomings of Canvas

Looking back in time, there only seem to be a handful of issues preventing FlightGear from adopting the Canvas system for encapsulating, unifying and optimizing the rendering back-end as a whole using the concepts pioneered by the Canvas system: