|Started in||07/2020 (functional prototype)|
|Description||Support for multiple independent scene views|
|Contributor(s)||Julian Smith |
experimental (building, mostly working, supports cloning independent views into distinct windows  ) |
|Note While simply using OSG 3.6 for the Compositor/CompositeViewer does solve a few issues, that switch is causing new complications elsewhere|
- Issues with window resize/close appear to be bugs in OpenSceneGraph-3.4, and are fixed by building with OpenSceneGraph-3.6.
- Extra view windows now show textures and clouds etc, and rendering appears to be identical to the main view.
- This works by creating a new Compositor instance for each extra view window, and calling its update() method each frame.
- Use of CompositeViewer is enabled at runtime with: --composite-viewer=1
- CompositeViewer requires a Compositor build.
- CompositeViewer requires OpenSceneGraph-3.6 to work well. See below for limitations when using OpenSceneGraph-3.4.
- The UI for creating new view windows uses new View menu items to clone the current view or take eye/target points from two earlier views. This seems to be convenient and avoids the need for a separate dialogue box or similar.
- Extra view windows can be resized.
- Extra view windows can clone Pilot View, Helicopter View, Chase View or Tower View.
- One can create an extra view window that keeps two aircraft in view with one in the foreground. E.g. see this video from 2020-9-6 (prior to getting textures working): http://op59.net/fg-cv-demo3.mpeg
- One can create an extra view window that uses the eye points from two recent views as eye and target. For example this allows a window to show a view from one aircraft to another.
- There doesn't seem to be any noticeable speed penalty if no extra view windows are opened.
- Extra view windows use a new view system called sview, which allows multiple instances and dynamic specification of eye and target points.
Current limitations that we know how to fix
- Extra view windows don't have event handling, so for example one cannot change the angle of a cloned Helicopter view.
- Sview does not support some views, e.g. Tower View AGL, Fly-past view.
- Sview does not support aircraft-specific views.
Issues when using OpenSceneGraph-3.4
OpenSceneGraph-3.4 seems to get event handling wrong which causes problems when we open extra view windows - resize/close events get sent to our main window event handler which confuses things. We don't know of a workaround for this.
These problems appear to be fixed in OpenSceneGraph-3.6.
Symptoms with OpenSceneGraph-3.4 include:
- Creating extra view windows can cause the main window to seemingly think it has same size as the new window, so menubar and dialogues are shown partially or not at all. A manual resize of the main window seems to sort things out.
- Attempting to closing an extra view windows sometimes closes a different extra view window.
- Attempting to closing an extra view windows sometimes closes Flightgear down because the code thinks the main window has been closed.
Bugs that we don't understand
Latest code can be found at:
Until mid-2020, FlightGear only supported one view position at a time. Multiple independent view positions (e.g. one screen for the tower view and a second screen for the plane) would complicate a "locked to cache" flag quite a lot....
Aircraft can define their own views and so on. But only one view can be active at a time. So no matter how many windows and cameras you define in Defaults.xml, they all are relative to the current view in FG (i.e. cockpit, tower...). 
Back in 2008, Tim Moore provided a patch (mailing lists search for CameraGroup FlightGear Mailing Lists) to use the osgViewer class to set up windows, manage the main camera, etc. 
However, these windows have to use the same camera group as the main window so can only show the view from the same eye position, though typically at a different angle/offset so that one can emulate things like side windows of a cockpit displayed in a different window or monitor.
Mathias Fröhlich used the slave camera feature of osgViewer to provide a "video wall" style of multiple displays that was demonstrated at LinuxTag for years. Later on, Tim generalized this to support general monitor arrangements (like a panoramic arc) and general combinations of screens and graphics cards. 
The default OSG model is that slave cameras are different views offset from a common viewpoint. This is easy to understand when considering a camera's view matrix, but not necessarily intuitive when thinking about the projection matrix. Because FG has its own view system we mostly treat the slaves as independent. It seems that most other uses of cameras during rendering -- for example, render to texture cameras for effects -- are best handled by slave cameras with independent views as well.
People requiring multiple independent views on the same scenery, e.g. cockpit and tower view [...] these each need their own camera groups and so require OSG's CompositeViewer.
And that's not really supported by the current architecture, neither by the tile cache nor by osgViewer::Viewer. We would need to move to a CompositeViewer model, which supports several scene graphs, and rely completely on the osg database paging machinery.
That would require a change in current fg architecture to use a CompositeViewer instead of a single Viewer, but we're contemplating that anyway.
The cameras in a camera group don't need to render directly to the screen. They can render to a texture which can be used either in the scene, like in a video screen in the instrument panel, or for distortion correction in a projected or dome environment. 
Open Scene Graph supports a CompositeViewer object that supports rendering from several widely separated viewpoints, complete with support for multiple terrain pager threads. We could move to CompositeViewer and support simultaneous views from e.g., the tower, AI models, drones, etc.
Neither of these are supported at the present time, but it would be a good project. We would have to start using a different OSG class, CompositeViewer, to support multiple views from independent view points. Our terrain pager would need a complete overhaul to use the PagedLOD scheme of OSG, and the Flightgear View manager would need to be aware multiple active views.
It would be very nice if CompositeViewer just works. However, James emphasized that we have a very complex display stack, with lots of permutations, and frankly we’re trying to /simplify/ the display layer, in anticipation of VSG in the future, not add more special cases to it. Since we don’t really have a large pool of 3D renderer developers, we need to be able to use a standard one without too much additional work or customisation.
Getting rid of Rembrandt, and making Compositor the only rendering pipeline, will drastically help with reducing the number of code paths we need to maintain. So we should not be in a hurry to add a new optional code path we have to test for, and which might interact weirdly with multi-camera / fgviewer / stereoscopic / etc. 
What we need is a bit more analysis of what it might impact, in terms of multi-camera support, performance across different platforms and so on. Especially, knowing how it will impact switching to VSG in the future, if VSG isn’t going to have an equivalent, then we should be be quite reluctant to start using it.
Jules stated, performance is guaranteed unchanged if CompositeViewer is disabled. But he hasn't profiled performance with it enabled, but certainly hasn't noticed any particular differences and wouldn't expect any such difference in the case where one is only using a single view.
Regarding VSG specifically, Jules asked on the vsg-users group and it looks like VSG will certainly support what OSG CompositeView does. See: https://groups.google.com/forum/#!topic/vsg-users/a68-CjA8kso
About this specific feature, there’s one concern: from memory (which may be incorrect), the CompositeViewer changes the OSG multi-threading model significantly, because there are multiple contexts. And since most of our long-standing crashing bugs are around OSG multi-threading, usually in combination with some other feature (eg, particles, or reset), James is not in agreement about the ‘risk free’ nature, if it touches threading at all. But, hopes, if it’s off by default, we can find out.
James added, he would also be happier if we could get Fernando to do a code review, to make sure we’re not touching anything which might affect the CompositOR. James also offered to help with reviewing things .
- VSG Roadmap
- VSG High Level Design
Supporting an optional CompositeViewer mode does not need to cause any significant problems. The changes to the code are actually very simple and well-defined, and it'll be easy to keep things very separate as things develop.
Jules and James agreed about the need to avoid adding to complexity in the rendering pipeline, and Jules expects as long as care is taken it will be fairly easy to keep CompositeViewer-specific code logically separate.
Jules hopes that CompositeViewer is fundamentally risk-free, in a way that most significant changes to a project like Flightgear are not, because ultimately OSG CompositeViewer is a very simple generalisation of OSG Viewer - instead of a Viewer also /being/ a View, a CompositeViewer /has/ one or more Views. That's all there is to it.
So we've always got the basic scenario to fall back on where there is just a single view and things are essentially identical to a standard non-CompositeViewer setup.
There are a few limitations of the FlightGear multi-camera/view/display system. Tim Moore is the person who developed this feature (nothing existed before his efforts) and maybe he can offer more insight.
See FlightGear and OpenGL ES for the main article about this subject.
For example, in the case of:
- the built-in GUI (PUI)
- HUD (scheduled to be re-implemented via the Canvas system as per Post FlightGear 2020.2 LTS changes)
- 2D instrument panels (scheduled to be re-implemented via the Canvas system as per Post FlightGear 2020.2 LTS changes)
there would need to be some significant code restructuring to allow these to be displayed on other windows.
There's a limitation in Plib that forces the GUI to be drawn on one window.