View source for CompositeViewer Support

<!--
{{Affected by Compositor}}
-->

{{infobox subsystem
|image       = Fg-cv-textures2.jpeg|Demonstration of multiple view windows in flightgear
|name        = CompositeViewer Support
|started     = 07/2020 
|description = Support for multiple independent scene views
|status      = awaiting merge 11/2020 <ref>https://sourceforge.net/p/flightgear/mailman/message/37148460/</ref> <p/>{{Progressbar|70}} 
|developers  = Julian Smith<ref>https://sourceforge.net/p/flightgear/mailman/message/37062319/</ref> <ref>https://sourceforge.net/p/flightgear/mailman/message/37077854/</ref>
|changelog = https://sourceforge.net/u/cgdae/profile/feed.rss, https://sourceforge.net/u/cgdae/flightgear/ci/CompositeViewer/tree/
<!--
|topic-fg= https://sourceforge.net/u/cgdae/flightgear/ci/CompositeViewer
|topic-sg= https://sourceforge.net/u/cgdae/simgear/ci/CompositeViewer
-->
<!--
|folders = 
* {{flightgear file|src/Viewer}}
* {{simgear file|simgear/scene/viewer}}
* {{fgdata file|Compositor}}
-->
}}


{{Rendering}}
<!--
{{Multicore}}
-->

== CompositeViewer work ==

=== Status updates ===
2020-11-16
* Some [[Hackathon_Proposal:_CompositeViewer_and_Canvas|progress on Canvas Views]] was made at the Hackathon.
* Compositor is now the default on next

2020-11-11
* concrete merge discussions, scheduled for 11/2020 <ref>https://sourceforge.net/p/flightgear/mailman/message/37148460/</ref>

2020-10-17:
* Allow non-Compositor builds (without support for CompositeViewer).

2020-10-5:
* Issues with window resize/close appear to be bugs in OpenSceneGraph-3.4, and are fixed by building with OpenSceneGraph-3.6.

2020-9-27:
* Extra view windows now show textures and clouds etc, and rendering appears to be identical to the main view.
* This works by creating a new Compositor instance for each extra view window, and calling its '''update()''' method each frame.

=== General information ===

* Video showing extra view windows and initial implementation of Canvas View: [http://op59.net/cvcanvas-demo.ogv cvcanvas-demo.ogv]
* Use of CompositeViewer is enabled at runtime with: '''--composite-viewer=1'''
* CompositeViewer requires a [[Compositor]] build.
* When enabled, CompositeViewer requires OpenSceneGraph-3.6 to work well.
* The UI for creating new view windows uses new View menu items to clone the current view or take eye/target points from two earlier views. This seems to be convenient and avoids the need for a separate dialogue box or similar.
* Extra view windows can be resized.
* Extra view windows can clone Pilot View, Helicopter View, Chase View or Tower View.
* One can create an extra view window that keeps two aircraft in view with one in the foreground. E.g. see this video from 2020-9-6 (prior to getting textures working): http://op59.net/fg-cv-demo3.mpeg
* One can create an extra view window that uses the eye points from two recent views as eye and target. For example this allows a window to show a view from one aircraft to another.
* There doesn't seem to be any noticeable speed penalty if no extra view windows are opened.
* Extra view windows use a new view system called sview (step view), which allows multiple instances and dynamic specification of eye and target points.
* CompositeViewer support will allow us to render a view to a canvas and implement things like rear-view mirrors etc - see: [[Canvas_View_Camera_Element]].

=== Problems ===

* Using OpenSceneGraph-3.6 causes problems elsewhere: [[OSGText Issues]]

=== Current limitations that we know how to fix ===

* Extra view windows don't have event handling, so for example one cannot change the angle of a cloned Helicopter view.
* Sview does not support some views, e.g. Tower View AGL, Fly-past view.
* Sview does not support aircraft-specific views.

=== Issues when using OpenSceneGraph-3.4 ===

OpenSceneGraph-3.4 seems to get event handling wrong which causes problems if we open extra view windows - resize/close events get sent to our main window event handler which confuses things. There is currently no workaround for this.

These problems appear to be fixed in OpenSceneGraph-3.6.

Symptoms with OpenSceneGraph-3.4 include:

* Creating extra view windows can cause the main window to seemingly think it has same size as the new window, so menubar and dialogues are shown partially or not at all. A manual resize of the main window seems to sort things out.
* Attempting to closing an extra view windows sometimes closes a different extra view window.
* Attempting to closing an extra view windows sometimes closes Flightgear down because the code thinks the main window has been closed.

=== Code ===

Latest code can be found at:

    https://sourceforge.net/u/cgdae/flightgear/ci/CompositeViewer
    https://sourceforge.net/u/cgdae/simgear/ci/CompositeViewer
    https://sourceforge.net/u/cgdae/fgdata/ci/CompositeViewer

== Motivation ==

Until mid-2020, FlightGear only supported one view position at a time. Multiple independent view positions (e.g. one screen for the tower view and a second screen for the plane) would complicate a "locked to cache" flag quite a lot...<ref>http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg28864.html</ref>.

Aircraft can define their own views and so on. But only one view can be active at a time. So no matter how many windows and cameras you define in [[Defaults.xml]], they all are relative to the current view in FG (i.e. cockpit, tower...). <ref>http://forum.flightgear.org/viewtopic.php?p=146136#p146136</ref> 

Back in 2008, Tim Moore provided a patch ({{Search|keywords=CameraGroup|mode=mailing lists}}) to use the osgViewer class to [[Howto:Configure camera view windows|set up windows, manage the main camera]], etc. <ref>https://sourceforge.net/p/flightgear/mailman/message/19718339/</ref> 

However, these windows have to use the same camera group as the main window so can only show the view from the same eye position, though typically at a different angle/offset so that one can emulate things like side windows of a cockpit displayed in a different window or monitor.<ref>https://sourceforge.net/p/flightgear/mailman/message/37059117/</ref>

<!--
There is some tight coupling between the Renderer.cxx and CameraGroup files (now to be found in SimGear<ref>v</ref>) The camera-infos can be configured from XML, eg to additional post-processing stages, but within each CameraInfo the actual osg::Cameras (each of which is really a rendering pass/stage) is determined by one of the buildXXXXX methods.<ref>https://sourceforge.net/p/flightgear/mailman/message/35828189/</ref>
-->

Mathias Fröhlich used the slave camera feature of osgViewer to provide a "video wall" style of multiple displays that was demonstrated at LinuxTag for years. Later on, Tim generalized this to support general monitor arrangements (like a panoramic arc) and general combinations of screens and graphics cards. <ref>https://sourceforge.net/p/flightgear/mailman/message/24811861/</ref>

The default OSG model is that slave cameras are different views offset from a common viewpoint. This is easy to understand when considering  a camera's view matrix, but not necessarily  intuitive when thinking about the projection matrix. Because FG has its own view system we mostly treat the slaves as independent. It seems that most other uses of cameras during rendering -- for example, render to texture cameras for effects -- are best handled by slave cameras with independent views as well.<ref>https://sourceforge.net/p/flightgear/mailman/message/36295606/</ref>

People requiring multiple independent views on the same scenery, e.g. cockpit and tower view [...] these each need their own camera groups and so require OSG's CompositeViewer.<ref>https://sourceforge.net/p/flightgear/mailman/message/37059117/</ref>

And that's not really supported by the current architecture, neither by the tile cache nor by osgViewer::Viewer. We would need to move to a CompositeViewer model, which supports several scene graphs, and rely completely on the osg database paging machinery.<ref>http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg28869.html</ref>

That would require a change in current fg architecture to use a CompositeViewer instead of a single Viewer, but we're contemplating that anyway.<ref>http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg17263.html</ref>

The cameras in a camera group don't need to render directly to the screen. They can render to a texture which can be used either in the scene, like in a video screen in the instrument panel, or for distortion correction in a projected or dome environment. <ref>http://sourceforge.net/p/flightgear/mailman/message/19718339/</ref>

Open Scene Graph supports a CompositeViewer object that supports rendering from several widely separated viewpoints, complete with support for multiple terrain pager threads. We could move to CompositeViewer and support simultaneous views from e.g., the tower, AI models, drones, etc.<ref>http://sourceforge.net/p/flightgear/mailman/message/19718339/</ref>

Neither of these are supported at the present time, but it would be a good project. We would have to start using a different OSG class, CompositeViewer, to support multiple views from independent view points. Our terrain pager would need a complete overhaul to use the [[PagedLOD]] scheme of OSG, and the Flightgear [[View manager]] would need to be aware multiple active views.<ref>=http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg27134.html</ref>

== Related ==
=== Docs ===
* http://www.openscenegraph.org/index.php/documentation/guides/programming-guides/93-viewer-vs-compositeviewer
* http://www.openscenegraph.org/documentation/OpenSceneGraphReferenceDocs/a00108.html
=== Discussions ===
* http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg17001.html
* http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg17263.html
* http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg27134.html
* http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg28869.html
* http://forum.flightgear.org/viewtopic.php?f=17&t=18813
* http://forum.flightgear.org/viewtopic.php?f=4&t=18785&p=199916#p199916
=== Search ===
* https://www.mail-archive.com/search?q=compositeviewer&l=osg-users%40lists.openscenegraph.org
* https://www.mail-archive.com/search?q=pagedlod&l=osg-users%40lists.openscenegraph.org
* http://www.mail-archive.com/search?q=compositeviewer&l=flightgear-devel%40lists.sourceforge.net
* http://www.mail-archive.com/search?l=flightgear-devel%40lists.sourceforge.net&q=PagedLOD+
* https://sourceforge.net/p/flightgear/mailman/search/?q=%22CompositeViewer%22
* https://sourceforge.net/p/flightgear/mailman/search/?q=%22PagedLOD%22
* https://forum.flightgear.org/search.php?st=0&sk=t&sd=d&sr=posts&keywords=CompositeViewer

== References ==

{{Appendix}}
[[Category:Developer Plans]]