CompositeViewer Support

From FlightGear wiki
Revision as of 04:17, 5 October 2020 by Cgdae (Talk | contribs) (CompositeViewer work)

Jump to: navigation, search


Note  No one is claiming CompositeViewer support is anywhere near being ready to be enabled by default - there are lots of issues to sort out.

Also, the potential benefits we get with an improved viewing system will be really significant. (For example easy creation at runtime of multiple arbitrary top-level windows showing Tower View / Helicopter View etc on one's own or multiplayer aircraft, new dynamic views such as the view from one aircraft to another, etc etc.)

With free software one has to rely on developer enthusiasm for getting things done, and this doesn't always last forever and doesn't always match up with roadmaps. Furthermore, Jules stated it could be years before we actually make significant progress to VSG, and it would be a real shame to block other projects in the meantime.[1]

In 07/2020, Julian Smith reported some success by changing flightgear's FGRenderer's osgViewer::Viewer to an osgViewer::CompositeViewer with a single osgViewer::View, and patching up all the calling code so it compiles.

It all builds ok, and he now got fgfs mostly working with CompositeViewer and the usual single View.[2]

Julian made the use of CompositeViewer configurable at runtime, which might simplify getting involved in testing/further development - the same binary can now be used for normal use and for investigating CompositeViewer behaviour.

CompositeViewer is not a build-time option any more. It is disabled by default, and enabled on startup with a new --composite-viewer=1 option.[3]

To execute the patched binary with the compositeviewer enabled, use: fgfs --composite-viewer=1 [4]

Note  According to Richard, when testing the database pager extensively (running with 20 DB pager threads), the main problem is the way that random objects (buildings, vegetation) are inserted into the scene graph doesn't seem to be compatible with lots of pager threads.[5]

In August 2020 Julian reported additional progress: He set things up so that one can 'clone' the current view using a key-press. This creates a new top-level view window with its own copy of the current view's camera projection matrix etc. The view is added to the CompositeViewer so it sees the same scene.

CompositeViewer Support
Started in 07/2020 (early prototype)
Description Support for multiple independent scene views
Contributor(s) Julian Smith[6] [7]
Status experimental (building, mostly working, supports cloning independent views into distinct windows [8] [9])
40}% completed [10]

CompositeViewer work

Status updates


  • Issues with window resize/close appear to be bugs in OpenSceneGraph-3.4, and are fixed by building with OpenSceneGraph-3.6.


  • Extra view windows now show textures and clouds etc, and rendering appears to be identical to the main view.
  • This works by creating a new Compositor instance for each extra view window, and calling its update() method each frame.

General information

  • Use of CompositeViewer is enabled at runtime with: --composite-viewer=1
  • CompositeViewer requires a Compositor build.
  • CompositeViewer requires OpenSceneGraph-3.6 to work well. See below for limitations when using OpenSceneGraph-3.4.
  • The current UI for creating new view windows works by using new View menu items to clone the current view or or use eye/target points from two earlier views. This seems to be convenient and avoids the need for a separate dialogue box or similar.
  • Extra view windows can be resized.
  • Extra view windows can clone Pilot View, Helicopter View, Chase View or Tower View.
  • One can create an extra view window that keeps two aircraft in view with one in the foreground. E.g. see this video from 2020-9-6 (prior to getting textures working):
  • One can create an extra view window that uses the eye points from two recent views as eye and target. For example this allows a window to show a view from one aircraft to another.
  • There doesn't seem to be any noticeable speed penalty if no extra view windows are opened.
  • Extra view windows use a new view system called sview, which allows multiple instances and allows dynamic specification of eye and target points.

Current limitations that we know how to fix

  • Extra view windows don't have event handling, so for example one cannot change the angle of a cloned Helicopter view.
  • Sview does not support some views, e.g. Tower View AGL, Fly-past view.
  • Sview does not support aircraft-specific views.

Issues when using OpenSceneGraph-3.4

OpenSceneGraph-3.4 seems to get event handling wrong which causes problems when we open extra view windows - resize/close events get sent to our main window event handler which confuses things. We don't know of a workaround for this.

These problems appear to be fixed in OpenSceneGraph-3.6.

Symptoms with OpenSceneGraph-3.4 include:

  • Creating extra view windows can cause the main window to seemingly think it has same size as the new window, so menubar and dialogues are shown partially or not at all. A manual resize of the main window seems to sort things out.
  • Attempting to closing an extra view windows sometimes closes a different extra view window.
  • Attempting to closing an extra view windows sometimes closes Flightgear down because the code thinks the main window has been closed.

Bugs that we don't understand


Latest code can be found at:


For the time being, FlightGear only supports one view position at a time. Multiple independent view positions (e.g. one screen for the tower view and a second screen for the plane) would complicate a "locked to cache" flag quite a lot...[11].

Aircraft can define their own views and so on. But only one view can be active at a time. So no matter how many windows and cameras you define in Defaults.xml, they all are relative to the current view in FG (i.e. cockpit, tower...). [12]

Back in 2008, Tim Moore provided a patch (mailing lists search for CameraGroup FlightGear Mailing Lists) to use the osgViewer class to set up windows, manage the main camera, etc. [13]

However, these windows have to use the same camera group as the main window so can only show the view from the same eye position, though typically at a different angle/offset so that one can emulate things like side windows of a cockpit displayed in a different window or monitor.[14]

Mathias Fröhlich used the slave camera feature of osgViewer to provide a "video wall" style of multiple displays that was demonstrated at LinuxTag for years. Later on, Tim generalized this to support general monitor arrangements (like a panoramic arc) and general combinations of screens and graphics cards. [15]

The default OSG model is that slave cameras are different views offset from a common viewpoint. This is easy to understand when considering a camera's view matrix, but not necessarily intuitive when thinking about the projection matrix. Because FG has its own view system we mostly treat the slaves as independent. It seems that most other uses of cameras during rendering -- for example, render to texture cameras for effects -- are best handled by slave cameras with independent views as well.[16]

People requiring multiple independent views on the same scenery, e.g. cockpit and tower view [...] these each need their own camera groups and so require OSG's CompositeViewer.[17]

And that's not really supported by the current architecture, neither by the tile cache nor by osgViewer::Viewer. We would need to move to a CompositeViewer model, which supports several scene graphs, and rely completely on the osg database paging machinery.[18]

That would require a change in current fg architecture to use a CompositeViewer instead of a single Viewer, but we're contemplating that anyway.[19]

The cameras in a camera group don't need to render directly to the screen. They can render to a texture which can be used either in the scene, like in a video screen in the instrument panel, or for distortion correction in a projected or dome environment. [20]

Open Scene Graph supports a CompositeViewer object that supports rendering from several widely separated viewpoints, complete with support for multiple terrain pager threads. We could move to CompositeViewer and support simultaneous views from e.g., the tower, AI models, drones, etc.[21]

Neither of these are supported at the present time, but it would be a good project. We would have to start using a different OSG class, CompositeViewer, to support multiple views from independent view points. Our terrain pager would need a complete overhaul to use the PagedLOD scheme of OSG, and the Flightgear View manager would need to be aware multiple active views.[22]

Open Questions

Status 08/2020

We are dropping compositor on next but we didn’t pull the switch yet, focusing on getting 2020.2 as stable as possible.

It would be very nice if CompositeViewer just works. However, James emphasized that we have a very complex display stack, with lots of permutations, and frankly we’re trying to /simplify/ the display layer, in anticipation of VSG in the future, not add more special cases to it. Since we don’t really have a large pool of 3D renderer developers, we need to be able to use a standard one without too much additional work or customisation.

Getting rid of Rembrandt, and making Compositor the only rendering pipeline, will drastically help with reducing the number of code paths we need to maintain. So we should not in a hurry to add a new optional code path we have to test for, and which might interact weirdly with multi-camera / fgviewer / stereoscopic / etc. [23]

What we need is a bit more analysis of what it might impact, in terms of multi-camera support, performance across different platforms and so on. Especially, knowing how it will impact switching to VSG in the future, if VSG isn’t going to have an equivalent, then we should be be quite reluctant to start using it.[24]

Jules stated, performance is guaranteed unchanged if CompositeViewer is disabled. But he hasn't profiled performance with it enabled, but certainly hasn't noticed any particular differences and wouldn't expect any such difference in the case where one is only using a single view.[25]

Regarding VSG specifically, Jules asked on the vsg-users group and it looks like VSG will certainly support what OSG CompositeView does[26]. See:!topic/vsg-users/a68-CjA8kso

About this specific feature, there’s one concern: from memory (which may be incorrect), the CompositeViewer changes the OSG multi-threading model significantly, because there are multiple contexts. And since most of our long-standing crashing bugs are around OSG multi-threading, usually in combination with some other feature (eg, particles, or reset), James is not in agreement about the ‘risk free’ nature, if it touches threading at all. But, hopes, if it’s off by default, we can find out.

James added, he would also be happier if we could get Fernando to do a code review, to make sure we’re not touching anything which might affect the CompositOR. James also offered to help with reviewing things [27].

Supporting an optional CompositeViewer mode does not need to cause any significant problems. The changes to the code are actually very simple and well-defined, and it'll be easy to keep things very separate as things develop.

Jules and James agreed about the need to avoid adding to complexity in the rendering pipeline, and Jules expects as long as care is taken it will be fairly easy to keep CompositeViewer-specific code logically separate.

Jules hopes that CompositeViewer is fundamentally risk-free, in a way that most significant changes to a project like Flightgear are not, because ultimately OSG CompositeViewer is a very simple generalisation of OSG Viewer - instead of a Viewer also /being/ a View, a CompositeViewer /has/ one or more Views. That's all there is to it.

So we've always got the basic scenario to fall back on where there is just a single view and things are essentially identical to a standard non-CompositeViewer setup.[28]

Use Cases

Known Limitations

There are a few limitations of the FlightGear multi-camera/view/display system. Tim Moore is the person who developed this feature (nothing existed before his efforts) and maybe he can offer more insight.

1rightarrow.png See FlightGear and OpenGL ES for the main article about this subject.

For example, in the case of:

there would need to be some significant code restructuring to allow these to be displayed on other windows.[29]

There's a limitation in Plib that forces the GUI to be drawn on one window.[30]






  22. =