CompositeViewer support: Difference between revisions

From FlightGear wiki
Jump to navigation Jump to search
m (→‎Motivation: update)
(46 intermediate revisions by 2 users not shown)
Line 2: Line 2:
{{Affected by Compositor}}
{{Affected by Compositor}}
-->
-->
== Status ==
{{Out of date}}
{{Note|While it's certainly possible to have a look at the changes <ref>https://sourceforge.net/p/flightgear/mailman/message/37077854/</ref> and understand how things are working, the code isn't really ready for review and push yet - there's still loads of debug diagnostics and a few dead-ends in there. Jules is hoping to be able to address some of these over the next few days.<ref>https://sourceforge.net/p/flightgear/mailman/message/37077148/</ref>
In other words: No one is claiming CompositeViewer support is anywhere near being ready to be  enabled by default - there are lots of issues to sort out.
Also, the potential benefits we get with an improved viewing system will be really significant. (For example easy creation at runtime of multiple arbitrary top-level
windows showing Tower View / Helicopter View etc on one's own or multiplayer aircraft, new dynamic views such as the view from one aircraft to another, etc etc.)
With free software one has to rely on developer enthusiasm for getting things done, and this doesn't always last forever and doesn't always match up with roadmaps.
Furthermore, Jules stated it could be years before we actually make significant progress to VSG, and it would be a real shame to block other projects in the meantime.<ref>https://sourceforge.net/p/flightgear/mailman/message/37076818/</ref>}}
In 07/2020, Julian Smith reported some  success by changing flightgear's FGRenderer's osgViewer::Viewer to an osgViewer::CompositeViewer with a single osgViewer::View, and patching up all the calling code so it compiles.
It all builds ok, and he now got fgfs mostly working with CompositeViewer and the usual single View.<ref>https://sourceforge.net/p/flightgear/mailman/flightgear-devel/</ref>
Julian made the use of CompositeViewer configurable at runtime, which might simplify getting involved in testing/further development - the same binary can now be used for normal use and for investigating CompositeViewer behaviour.
CompositeViewer is not a build-time option any more. It is disabled by default, and enabled on startup with a new --composite-viewer=1 option.<ref>https://sourceforge.net/p/flightgear/mailman/message/37075130/</ref>
To execute the patched binary with the compositeviewer enabled, use: <code>fgfs  --composite-viewer=1</code> <ref>https://sourceforge.net/u/cgdae/flightgear/ci/059cf073c74d43a57c11f363cae6c9271772d869</ref>
Unfortunately the property system seemed to be not set up early enough to read in FGRenderer's constructor so we do a few calls of fgGetNode("/sim/composite_viewer", true)->getBoolValue() at runtime. Julian suspects they aren't called very frequently, in which case this branch (minus the diagnostics changes probably) might be ok to push to next at
some point, so that it's easier to test/investigate things.<ref>https://sourceforge.net/p/flightgear/mailman/message/37062319/</ref>
For some reason buildings don't appear, but basic ground scenery now loads so fgfs progresses to loading the fdm etc and one can fly the aircraft as normal.
{{Note|According to Richard, when testing the database pager extensively (running with 20 DB pager threads), the main problem is the way that random objects (buildings, vegetation) are inserted into the scene graph doesn't seem  to be compatible with lots of pager threads.<ref>https://sourceforge.net/p/flightgear/mailman/message/36536293/</ref>}}
In August 2020 Julian reported additional progress: He set things up so that one can 'clone' the current view using a key-press. This creates a new top-level view window with its own copy of the current view's camera projection matrix etc. The view is added to the CompositeViewer so it sees the same scene.
For now at least, cloned views' projection matrices don't change so the cloned views are completely independent from movement of the aircraft or changes to the main view. Eventually he would like to make them behave more dynamically, e.g. follow the aircraft if cloned from Helicopter View.
At the moment most of the scenery doesn't show up and the background colours are wrong, but this screen shot shows it working: http://op59.net/fg-cv.png
The cloned view on left was cloned from Pilot View when the aircraft was at the end of the runway. Since then the aircraft has been moved up the runway and is hovering. The cloned view on the right was cloned from a Tower AGL view of the hovering aircraft.<ref>https://sourceforge.net/p/flightgear/mailman/message/37075206/</ref>
In 09/2020, Jules reported that he's still unable to get textures to work in cloned views, so he's been playing around with various new viewing features using the CompositeViewer support.
Here's a short and very grainy video showing a 'double view' in a separate window on the left of the main window. The double view keeps two aircraft (a Harrier-GR3 and a SU-37 here) in view at all times, keeping one (the Harrier) at a small constant distance in the foreground <ref>https://sourceforge.net/p/flightgear/mailman/message/37101960/</ref>: http://op59.net/fg-cv-demo3.mpeg
{{See also|Draw masks}}
{| class="wikitable"
|-
! Feature !! Status !! Notes
|-
| Scenery/Terrain || {{Progressbar|70}} || buildings missing (this seems line line with Richard's findings mentioned at <ref>https://sourceforge.net/p/flightgear/mailman/message/36536293/</ref><ref>https://sourceforge.net/p/flightgear/mailman/message/37004640/</ref>)
|-
| Aircraft || {{Done}} || -
|-
| Skydome || {{Done}} || -
|-
| PUI GUI || {{Done}} || -
|}


{{infobox subsystem
{{infobox subsystem
|image      = [[File:Fg-cv-textures2.jpeg|thumb|Demonstration of multiple view windows in flightgear.]]
|image      = Fg-cv-textures2.jpeg|Demonstration of multiple view windows in flightgear
|name        = CompositeViewer Support
|name        = CompositeViewer Support
|started    = 07/2020 (early prototype)
|started    = 07/2020  
|description = Support for multiple independent scene views
|description = Support for multiple independent scene views
|status      = experimental (building, mostly working, buildings missing, supports cloning independent views into distinct windows <ref>https://sourceforge.net/p/flightgear/mailman/message/37086211/</ref> <ref>http://wiki.flightgear.org/CompositeViewer_Support#cite_note-22</ref>) <br/>{{Progressbar|40}} <ref>https://sourceforge.net/p/flightgear/mailman/message/37070203/</ref>
|status      = merged 11/2020, awaiting feedback <ref>https://sourceforge.net/p/flightgear/mailman/message/37158652/</ref> <ref>https://sourceforge.net/p/flightgear/mailman/message/37159361/</ref>
|developers  = Julian Smith<ref>https://sourceforge.net/p/flightgear/mailman/message/37062319/</ref> <ref>https://sourceforge.net/p/flightgear/mailman/message/37077854/</ref>
|developers  = Julian Smith<ref>https://sourceforge.net/p/flightgear/mailman/message/37062319/</ref> <ref>https://sourceforge.net/p/flightgear/mailman/message/37077854/</ref>
|changelog = https://sourceforge.net/u/cgdae/profile/feed.rss, https://sourceforge.net/u/cgdae/flightgear/ci/CompositeViewer/tree/
|changelog = https://sourceforge.net/u/cgdae/profile/feed.rss, https://sourceforge.net/u/cgdae/flightgear/ci/CompositeViewer/tree/
Line 87: Line 29:
-->
-->


== CompositeViewer work ==


=== Status updates ===
2020-11-22
* To encourage wider testing, '''CompositeViewer support''' is expected to be defaulted to '''on''' <ref>https://sourceforge.net/p/flightgear/mailman/message/37159361/</ref> <ref>https://sourceforge.net/p/flightgear/mailman/message/37159461/</ref> <ref>https://sourceforge.net/p/flightgear/mailman/message/37160096/</ref>
2020-11-21
* merged into next: {{flightgear commit|f62e5b9ce3462758b48f4c711eb7c3bf4bcc7061|CompositeViewer:Support for multiple view windows using osgViewer::CompositeViewer}} <ref>https://sourceforge.net/p/flightgear/mailman/message/37158652/</ref>


== Jules' CompositeViewer work ==
2020-11-16
* Some [[Hackathon_Proposal:_CompositeViewer_and_Canvas|progress on Canvas Views]] was made at the Hackathon.
* Compositor is now the default on next


=== Status 2020-9-27 ===
2020-11-11
* concrete merge discussions, scheduled for 11/2020 <ref>https://sourceforge.net/p/flightgear/mailman/message/37148460/</ref>


2020-10-17:
* Allow non-Compositor builds (without support for CompositeViewer).
2020-10-5:
* Issues with window resize/close appear to be bugs in OpenSceneGraph-3.4, and are fixed by building with OpenSceneGraph-3.6.
2020-9-27:
* Extra view windows now show textures and clouds etc, and rendering appears to be identical to the main view.
* Extra view windows now show textures and clouds etc, and rendering appears to be identical to the main view.
* This works by creating a new Compositor instance for each extra view window, and calling its '''update()''' method each frame.
* This works by creating a new Compositor instance for each extra view window, and calling its '''update()''' method each frame.


=== General information: ===
=== General information ===


* Use of composite viewer is enabled on the command line with: '''--composite-viewer=1'''
* Video showing extra view windows and initial implementation of Canvas View: [http://op59.net/cvcanvas-demo.ogv cvcanvas-demo.ogv]
* The UI for creating new view windows works by cloning the current view, or using eye/target points from two earlier views. This seems to be a convenient way of creating new view windows without having to open a dialogue box or similar.
* Use of CompositeViewer is enabled at runtime with: '''--composite-viewer=1'''
* Extra view windows can be resized but they do not (currently) respond to mouse/keyboard.
* CompositeViewer requires a [[Compositor]] build.
* Extra view windows can clone Pilot, Helicopter View or Chase View.
* When enabled, CompositeViewer requires OpenSceneGraph-3.6 to work well.
* Closing extra view windows sometimes causes a crash.
* The UI for creating new view windows uses new View menu items to clone the current view or take eye/target points from two earlier views. This seems to be convenient and avoids the need for a separate dialogue box or similar.
* Extra view windows can be resized.
* Extra view windows can clone Pilot View, Helicopter View, Chase View or Tower View.
* One can create an extra view window that keeps two aircraft in view with one in the foreground. E.g. see this video from 2020-9-6 (prior to getting textures working): http://op59.net/fg-cv-demo3.mpeg
* One can create an extra view window that keeps two aircraft in view with one in the foreground. E.g. see this video from 2020-9-6 (prior to getting textures working): http://op59.net/fg-cv-demo3.mpeg
* One can create an extra view window that uses the eye points from two recent views as eye and target. For example this allows a window to show a view from one aircraft to another.
* One can create an extra view window that uses the eye points from two recent views as eye and target. For example this allows a window to show a view from one aircraft to another.
* Extra view windows can be resized but they do not (currently) respond to mouse/keyboard.
* After creation of an extra view window, the main window needs to be resized, otherwise menus etc don't appear properly.
* There doesn't seem to be any noticeable speed penalty if no extra view windows are opened.
* There doesn't seem to be any noticeable speed penalty if no extra view windows are opened.
* Extra view windows use a new view system called sview, which allows multiple instances and allows dynamic specification of eye and target points.
* Extra view windows use a new view system called sview (step view), which allows multiple instances and dynamic specification of eye and target points.
* CompositeViewer support will allow us to render a view to a canvas and implement things like rear-view mirrors etc - see: [[Canvas_View_Camera_Element]].
 
=== Problems ===
 
* Using OpenSceneGraph-3.6 causes problems elsewhere: [[OSGText Issues]] (under investigation as of 11/2020) <ref>https://sourceforge.net/p/flightgear/mailman/message/37157550/</ref>
 
=== Current limitations that we know how to fix ===
 
* Extra view windows don't have event handling, so for example one cannot change the angle of a cloned Helicopter view.
* Sview does not support some views, e.g. Tower View AGL, Fly-past view.
* Sview does not support aircraft-specific views.


=== Code ===
=== Issues when using OpenSceneGraph-3.4 ===
 
OpenSceneGraph-3.4 seems to get event handling wrong which causes problems if we open extra view windows - resize/close events get sent to our main window event handler which confuses things. There is currently no workaround for this.
 
These problems appear to be fixed in OpenSceneGraph-3.6.


Latest code can be found at:
Symptoms with OpenSceneGraph-3.4 include:


    https://sourceforge.net/u/cgdae/flightgear/ci/CompositeViewer
* Creating extra view windows can cause the main window to seemingly think it has same size as the new window, so menubar and dialogues are shown partially or not at all. A manual resize of the main window seems to sort things out.
    https://sourceforge.net/u/cgdae/simgear/ci/CompositeViewer
* Attempting to closing an extra view windows sometimes closes a different extra view window.
    https://sourceforge.net/u/cgdae/fgdata/ci/CompositeViewer
* Attempting to closing an extra view windows sometimes closes Flightgear down because the code thinks the main window has been closed.


=== Code ===


Latest code can be found on the 'next' branch of flightgear, simgear and fgdata.


== History ==
== Motivation ==
[[File:CompositeViewer 3 cloned views and OSG stats with draw masks.png|thumb|Experimental [[CompositeViewer Support]] showing 3 cloned views, with draw masks set (only skydome shown) + OSG stats (~ 120 fps/window) <ref>https://forum.flightgear.org/viewtopic.php?f=6&t=38334</ref>]]


For the time being, FlightGear only supports one view position at a time. Multiple independent view positions (e.g. one screen for the tower view and a second screen for the plane) would complicate a "locked to cache" flag quite a lot...<ref>http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg28864.html</ref>.
Until mid-2020, FlightGear only supported one view position at a time. Multiple independent view positions (e.g. one screen for the tower view and a second screen for the plane) would complicate a "locked to cache" flag quite a lot...<ref>http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg28864.html</ref>.


Aircraft can define their own views and so on. But only one view can be active at a time. So no matter how many windows and cameras you define in [[Defaults.xml]], they all are relative to the current view in FG (i.e. cockpit, tower...). <ref>http://forum.flightgear.org/viewtopic.php?p=146136#p146136</ref>  
Aircraft can define their own views and so on. But only one view can be active at a time. So no matter how many windows and cameras you define in [[Defaults.xml]], they all are relative to the current view in FG (i.e. cockpit, tower...). <ref>http://forum.flightgear.org/viewtopic.php?p=146136#p146136</ref>  
Line 149: Line 125:


Neither of these are supported at the present time, but it would be a good project. We would have to start using a different OSG class, CompositeViewer, to support multiple views from independent view points. Our terrain pager would need a complete overhaul to use the [[PagedLOD]] scheme of OSG, and the Flightgear [[View manager]] would need to be aware multiple active views.<ref>=http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg27134.html</ref>
Neither of these are supported at the present time, but it would be a good project. We would have to start using a different OSG class, CompositeViewer, to support multiple views from independent view points. Our terrain pager would need a complete overhaul to use the [[PagedLOD]] scheme of OSG, and the Flightgear [[View manager]] would need to be aware multiple active views.<ref>=http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg27134.html</ref>
=== Open Questions ===
{{See also|Vulkan Scene Graph}}
'''Status 08/2020'''
We are dropping compositor on next but we didn’t pull the switch yet, focusing on getting 2020.2 as stable as possible.
It would be very nice if CompositeViewer just works. However, James emphasized that we have a very complex display stack, with lots of permutations, and frankly we’re trying to /simplify/ the display layer, in anticipation of VSG in the future, not add more special cases to it.  Since we don’t really have a large pool of 3D renderer developers, we need to be able to use a standard one without too much additional work or customisation.
Getting rid of Rembrandt, and making Compositor the only rendering pipeline, will drastically help with reducing the number of code paths we need to maintain. So we should not in a hurry to add a new optional code path we have to test for, and which might interact weirdly with multi-camera / fgviewer / stereoscopic / etc. <ref>https://sourceforge.net/p/flightgear/mailman/message/37075182/</ref>
What we need is a bit more analysis of what it might impact, in terms of multi-camera support, performance across different platforms and so on.  Especially, knowing how it will impact switching to VSG in the future, if VSG isn’t going to have an equivalent, then we should be be quite reluctant to start using it.<ref>https://sourceforge.net/p/flightgear/mailman/message/37075174/</ref>
Jules stated, performance is guaranteed unchanged if CompositeViewer is disabled. But he hasn't profiled performance with it enabled, but certainly hasn't
noticed any particular differences and  wouldn't expect any such difference in the case where one is only using a single view.<ref>https://sourceforge.net/p/flightgear/mailman/message/37076818/</ref>
Regarding VSG specifically, Jules asked on the vsg-users group and it looks like VSG will certainly support what OSG CompositeView does<ref>https://sourceforge.net/p/flightgear/mailman/message/37076818/</ref>. See: https://groups.google.com/forum/#!topic/vsg-users/a68-CjA8kso
About this specific feature, there’s one concern: from memory (which may be incorrect), the CompositeViewer changes the OSG multi-threading model significantly, because there are multiple contexts. And since most of our long-standing crashing bugs are around OSG multi-threading, usually in combination with some other feature (eg, particles, or reset), James is not in agreement about the ‘risk free’ nature, if it touches threading at all. But, hopes, if it’s off by default, we can find out.
James added, he would also be happier if we could get Fernando to do a code review, to make sure we’re not touching anything which might affect the CompositOR. James also offered to  help with reviewing things <ref>https://sourceforge.net/p/flightgear/mailman/message/37077081/</ref>.
* [https://github.com/vsg-dev/VulkanSceneGraph/blob/master/ROADMAP.md VSG Roadmap]
* [https://vsg-dev.github.io/VulkanSceneGraph/docs/Design/HighLevelDesignDecisions.html VSG High Level Design]
* https://github.com/vsg-dev/VulkanSceneGraph/tree/master/src/vsg/viewer
Supporting an '''optional''' CompositeViewer mode does not need to cause any significant problems. The changes to the code are actually very simple and well-defined, and it'll be easy to keep things very separate as things develop.
Jules and James agreed about the need to avoid adding to complexity in the rendering pipeline, and Jules expects as long as care is taken it will be fairly easy to keep CompositeViewer-specific code logically separate.
Jules hopes that CompositeViewer is fundamentally risk-free, in a way that most significant changes to a project like Flightgear are not, because ultimately OSG CompositeViewer is a very simple generalisation of OSG Viewer - instead of a Viewer also /being/ a View, a CompositeViewer /has/ one or more Views. That's all there is to it.
So we've always got the basic scenario to fall back on where there is just a single view and things are essentially identical to a standard non-CompositeViewer setup.<ref>https://sourceforge.net/p/flightgear/mailman/message/37076818/</ref>
=== Potential Issues ===
So far, it seemed we 'Cant Use' CompositeViewer because of $reasons in the Effects code<ref>https://sourceforge.net/p/flightgear/mailman/message/26189393/</ref>. Maybe the reason is simply that it needs some work to move away from the single CameraGroup, however.<ref>https://sourceforge.net/p/flightgear/mailman/message/37059939/</ref>
Also, a number of senior contributors have previously pointed out how our existing tile manager/paging infrastructure may not yet be in shape for a CompositeViewer port, poweroftwo's osgEarth port however is using OSG's PagedLOD exclusively, and osgEarth itself is known to support CompositeViewer<ref>https://github.com/gwaldron/osgearth/issues/1300#issuecomment-565734550</ref>, too: https://gitlab.com/poweroftwo 
So, to troubleshoot this further, it'd make sense to exclude the usual suspects one by one:
* disable multi-threading (see Torsten's comments about known issues about multi-screen/multi-GPU setups at [[Howto:Activate multi core and multi GPU support]])
* disable scenery/terrain
* use [[Draw masks]]
* disable effects
At that point, we should be able to nail down where the culprit is, i.e. what needs changing.
Jules shared the patches (topic branch, see above) so that more people can get involved in testing and making progress.<ref>https://sourceforge.net/p/flightgear/mailman/message/37058676/</ref>
=== OSG stuff ===
{{Note|As a general note, osgViewer::CompositeViewer should have osgViewer::View attached to it NOT osgViewer::Viewer. It's a composite of View's not composite of Viewer.  If you look at all the OSG example they illustrate the correct usage.<ref>https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg70462.html</ref>}}
The OSG's osgViewer is designed to support multiple views on to one or more scenes using the osgViewer::CompositeViewer class.  See the osgcompositeviewer example.<ref>https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg69640.html</ref>
The CompositeViewer was written specifically for multiple Views.
Robert wouldn't recommend using slave Camera's for doing multiple views, while possible it's just a mess in terms of management.  slave Camera's are tools for helping rendering a single view, but with a view that is composed of several components - either spread across multiple windows, or a view that requires multiple passes such as distortion correction, field of view etc.<ref>https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg74474.html</ref>
The osgViewer has a mechanism for avoid multiple traversals of shared
scene graphs if mutlple View's share the same root node of the scene
graph.  If shared component isn't the topmost node then the OSG has no
straight forward way to know whether a subgraph has been traversed or
not that frame.  One could implement a mechanism to avoid this
visiting a node multiple times in one frame but it would be really
costly to do, an expense that would only be a benefit for a very small
number of users, but would slow performance for everyone else.
The most efficient way to avoid this multiple traversals issue is to
implement to have a custom callback that is tailored to the specific
usage case that a user has.  I don't know enough about the specific
callbacks and scene graph set up you have so I can't pinpoint the best
route.
If you have a shared subgraph that you don't want traversed multiple
times per frame then use an UpdateCallback that has a frameNumber
member variable that keep track of the the frameNumber (use
NodeVisitor::getFrameStamp()'s FrameNumber) of the last traversal,
when a traversal calls the update callback you only traverse the
subgraph if the frameNumber is different and then set the frameNumber
to the present frame,  if the frameNumber is the same then you just
return immediately.  This custom UpdateCallback you'd place as high as
you can in your scene graph to make sure the traversal stops as soon
as possible.
Another approach is to move this frameNumber tracking into your
existing update callbacks, and simple return right away with the
frameNumber is the same.  This requires a small tweak to the callbacks
but is such a small change it's generally pretty easy to integrate.
Finally you can simple make your callbacks resilient so that the are
no ill effects from being called multiple times.<ref>https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg74476.html</ref>
* For multiple views the way to do it with the OSG is to use the osgViewer::CompositeViewer class, and then each osgViewer::View you set up the approrpiate Camera's CullMask.<ref>https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg78309.html</ref>
* The draw traversal doesn't touch the scene graph nodes so NodeMasks/TraversalMask have no baring on it. The OSG's cull traversal builds the internal rendering graphs each frame and the draw traversal simply traversals these rendering graphs. So to control the draw traversal you use the masks in the cull traversal.<ref>https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg78308.html</ref>
* shader handling with multiple views: The OSG provides equivalents to the OpenGL built ins for you so you don't need to provide them yourself, so just ditch the callbacks and just use gl_ModelViewMatrix or osg_ModelViewMatrix.  The OSG has provision for doing automatic conversion from gl_ModelViewMatrix to osg_ModelViewMartrix, it's enabled by default for GL3 and GLES2 builds, but if not then you can enable if you want to use it.  See the osgsimplegl3 example. These uniforms that the OSG provides for the projection and modelview matrices work seamlesly across multiple views. Just use them and it'll work out of the box. <ref>https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg75995.html</ref>
The two options for off screen rendering are a  PixelBuffer context or
FrameBufferObject, if you have an existing on screen window in your
application then using a FrameBufferObject becomes preferable.  The
way to do it would be to set up your viewer's off screen Camera with
FBO settings and a custom final draw callback to do the read to main
memory.  The read will be more efficient if you use a pair of
PixelBufferObjects - the osgscreencapture example illustrates this in
action.
When already using CompositeViewer the most natural thing to do
would be to have a dedicated View with it's master Camera as the
offscreen camera, this way you can control the Camera's view matrix in
straight forward manner in the same way to the rest of the Camera's.
Also during debugging having the option of making this Camera an
onscreen one would give you means to visually QA things as you go
along.<ref>https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg70703.html</ref>
== Use Cases ==
{{See also|Canvas Popout Windows|View Manager Improvements}}
{{FGCquote
  |I have to create two master cameras to controls the two different views in two different scene rendering dynamically.<br/>
<br/>
but flightgear only using viewer class . This is createing one master camera no.of slave cameras. but i need CompositeViewer class. how to use CompositeViewer class in flightgear and how to render through CompositeViewer class.
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=227827#p227827
    |title=<nowiki>How to create 2 master camera and 2 views  in flightgear 3.0</nowiki>
    |author=<nowiki>Divi</nowiki>
    |date=<nowiki>Tue Dec 23</nowiki>
  }}
}}
{{cquote|<nowiki>What i want to achieve: Multiple views of the same scene . Lets call them View1 and View2(they may be on 2 separate screens). There should be only one
instance of flightgear running. I should be able to have "View1 = Cockpit View & View2 = Chase View" or "View1 = Helicopter View & View2 = Cockpit
View" or any other such combination of views. I am using FlightGear 2.0.0
I should be able to switch the view in each viewport without affecting the view in the other view port. i.e i should be able to double right click and
change the view on each display with the mouse without affecting the view in the other display.
README.multiscreen in the FlightGear documentation says its possible to create a master camera and a slave camera that is offset from the master
camera. BUT I want 2 master cameras that can switch to any view at any time.</nowiki><ref>{{cite web |url=http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg28647.html|title=<nowiki>Multiple views without slaving to master camera</nowiki>|author=<nowiki></nowiki>|date=<nowiki></nowiki>}}</ref>|<nowiki></nowiki>}}
{{cquote|I have just been trying out the multiple screen feature in FG. I found that the GUI camera (including the menu bar, hud and 2D panel) appears in only
one of the windows. Is there any way I can make the GUI to appear in all the windows? Actually I want to be able to view the hud and 2D panel in all the
windows.
Also when I change the view in any one of the windows, the view changes in the other windows as well. Is it possible to make the windows independent of
each other. I want to display the cockpit in one window at all times, in the second window I want to be able to shuttle between helicopter / chase or
model views.
Also I have observed that in the second screen where I'm displaying lets say the Helicopter view the aircraft remains static while the environment moves.
This is because the cockpit view in my Master screen is defined as 'lookfrom'. Can I define 'lookfrom' for one screen and 'lookat' for the
other screen.<ref>{{cite web |url=http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg27112.html|title=<nowiki>[Flightgear-devel] Help needed with multi-screen</nowiki>|author=Kavya Meyyappan|date=Fri, 19 Mar 2010 03:31:50 -0700}}</ref>|Kavya Meyyappan}}
== Known Limitations ==
{{See also|Unifying the 2D rendering backend via canvas|Howto:Using_FlightGear_as_an_Image_Generator_(IG)#Challenges}}
There are a few limitations of the FlightGear multi-camera/view/display system. Tim Moore is the person who developed this feature (nothing existed before his efforts) and maybe he can offer more insight. 
{{Main article|FlightGear and OpenGL ES}}
For example, in the case of:
* the built-in GUI ([[PUI]])
* HUD (scheduled to be re-implemented via the [[Canvas]] system as per [[Post FlightGear 2020.2 LTS changes]])
* 2D instrument panels (scheduled to be re-implemented via the [[Canvas]] system as per [[Post FlightGear 2020.2 LTS changes]])
there would need to be some significant code restructuring to allow these to be displayed on other windows.<ref>http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg27114.html</ref>
There's a limitation in Plib that forces the GUI to be drawn on one window.<ref>http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg27134.html</ref>


== Related ==
== Related ==
{{See also|FlightGear CIGI Support (Common Image Generator Interface)}}
<!-- FIXME: These should probably be using infrastructure link templates ? -->
=== Docs ===
=== Docs ===
* http://www.openscenegraph.org/index.php/documentation/guides/programming-guides/93-viewer-vs-compositeviewer
* http://www.openscenegraph.org/index.php/documentation/guides/programming-guides/93-viewer-vs-compositeviewer

Revision as of 23:23, 25 November 2020


CompositeViewer Support
Fg-cv-textures2.jpeg
Started in 07/2020
Description Support for multiple independent scene views
Contributor(s) Julian Smith[1] [2]
Status merged 11/2020, awaiting feedback [3] [4]
Changelog https://sourceforge.net/u/cgdae/profile/feed.rss, https://sourceforge.net/u/cgdae/flightgear/ci/CompositeViewer/tree/


Rendering

Rendering Pipeline

Effects and Shaders

Ongoing Efforts

Standalone Tools

IPC/Networking

CompositeViewer work

Status updates

2020-11-22

  • To encourage wider testing, CompositeViewer support is expected to be defaulted to on [5] [6] [7]

2020-11-21

2020-11-16

2020-11-11

  • concrete merge discussions, scheduled for 11/2020 [9]

2020-10-17:

  • Allow non-Compositor builds (without support for CompositeViewer).

2020-10-5:

  • Issues with window resize/close appear to be bugs in OpenSceneGraph-3.4, and are fixed by building with OpenSceneGraph-3.6.

2020-9-27:

  • Extra view windows now show textures and clouds etc, and rendering appears to be identical to the main view.
  • This works by creating a new Compositor instance for each extra view window, and calling its update() method each frame.

General information

  • Video showing extra view windows and initial implementation of Canvas View: cvcanvas-demo.ogv
  • Use of CompositeViewer is enabled at runtime with: --composite-viewer=1
  • CompositeViewer requires a Compositor build.
  • When enabled, CompositeViewer requires OpenSceneGraph-3.6 to work well.
  • The UI for creating new view windows uses new View menu items to clone the current view or take eye/target points from two earlier views. This seems to be convenient and avoids the need for a separate dialogue box or similar.
  • Extra view windows can be resized.
  • Extra view windows can clone Pilot View, Helicopter View, Chase View or Tower View.
  • One can create an extra view window that keeps two aircraft in view with one in the foreground. E.g. see this video from 2020-9-6 (prior to getting textures working): http://op59.net/fg-cv-demo3.mpeg
  • One can create an extra view window that uses the eye points from two recent views as eye and target. For example this allows a window to show a view from one aircraft to another.
  • There doesn't seem to be any noticeable speed penalty if no extra view windows are opened.
  • Extra view windows use a new view system called sview (step view), which allows multiple instances and dynamic specification of eye and target points.
  • CompositeViewer support will allow us to render a view to a canvas and implement things like rear-view mirrors etc - see: Canvas_View_Camera_Element.

Problems

  • Using OpenSceneGraph-3.6 causes problems elsewhere: OSGText Issues (under investigation as of 11/2020) [10]

Current limitations that we know how to fix

  • Extra view windows don't have event handling, so for example one cannot change the angle of a cloned Helicopter view.
  • Sview does not support some views, e.g. Tower View AGL, Fly-past view.
  • Sview does not support aircraft-specific views.

Issues when using OpenSceneGraph-3.4

OpenSceneGraph-3.4 seems to get event handling wrong which causes problems if we open extra view windows - resize/close events get sent to our main window event handler which confuses things. There is currently no workaround for this.

These problems appear to be fixed in OpenSceneGraph-3.6.

Symptoms with OpenSceneGraph-3.4 include:

  • Creating extra view windows can cause the main window to seemingly think it has same size as the new window, so menubar and dialogues are shown partially or not at all. A manual resize of the main window seems to sort things out.
  • Attempting to closing an extra view windows sometimes closes a different extra view window.
  • Attempting to closing an extra view windows sometimes closes Flightgear down because the code thinks the main window has been closed.

Code

Latest code can be found on the 'next' branch of flightgear, simgear and fgdata.

Motivation

Experimental CompositeViewer Support showing 3 cloned views, with draw masks set (only skydome shown) + OSG stats (~ 120 fps/window) [11]

Until mid-2020, FlightGear only supported one view position at a time. Multiple independent view positions (e.g. one screen for the tower view and a second screen for the plane) would complicate a "locked to cache" flag quite a lot...[12].

Aircraft can define their own views and so on. But only one view can be active at a time. So no matter how many windows and cameras you define in Defaults.xml, they all are relative to the current view in FG (i.e. cockpit, tower...). [13]

Back in 2008, Tim Moore provided a patch (mailing lists search for CameraGroup FlightGear Mailing Lists) to use the osgViewer class to set up windows, manage the main camera, etc. [14]

However, these windows have to use the same camera group as the main window so can only show the view from the same eye position, though typically at a different angle/offset so that one can emulate things like side windows of a cockpit displayed in a different window or monitor.[15]


Mathias Fröhlich used the slave camera feature of osgViewer to provide a "video wall" style of multiple displays that was demonstrated at LinuxTag for years. Later on, Tim generalized this to support general monitor arrangements (like a panoramic arc) and general combinations of screens and graphics cards. [16]

The default OSG model is that slave cameras are different views offset from a common viewpoint. This is easy to understand when considering a camera's view matrix, but not necessarily intuitive when thinking about the projection matrix. Because FG has its own view system we mostly treat the slaves as independent. It seems that most other uses of cameras during rendering -- for example, render to texture cameras for effects -- are best handled by slave cameras with independent views as well.[17]

People requiring multiple independent views on the same scenery, e.g. cockpit and tower view [...] these each need their own camera groups and so require OSG's CompositeViewer.[18]

And that's not really supported by the current architecture, neither by the tile cache nor by osgViewer::Viewer. We would need to move to a CompositeViewer model, which supports several scene graphs, and rely completely on the osg database paging machinery.[19]

That would require a change in current fg architecture to use a CompositeViewer instead of a single Viewer, but we're contemplating that anyway.[20]

The cameras in a camera group don't need to render directly to the screen. They can render to a texture which can be used either in the scene, like in a video screen in the instrument panel, or for distortion correction in a projected or dome environment. [21]

Open Scene Graph supports a CompositeViewer object that supports rendering from several widely separated viewpoints, complete with support for multiple terrain pager threads. We could move to CompositeViewer and support simultaneous views from e.g., the tower, AI models, drones, etc.[22]

Neither of these are supported at the present time, but it would be a good project. We would have to start using a different OSG class, CompositeViewer, to support multiple views from independent view points. Our terrain pager would need a complete overhaul to use the PagedLOD scheme of OSG, and the Flightgear View manager would need to be aware multiple active views.[23]

Related

Docs

Discussions

Search

References

References
  1. ↑ https://sourceforge.net/p/flightgear/mailman/message/37062319/
  2. ↑ https://sourceforge.net/p/flightgear/mailman/message/37077854/
  3. ↑ https://sourceforge.net/p/flightgear/mailman/message/37158652/
  4. ↑ https://sourceforge.net/p/flightgear/mailman/message/37159361/
  5. ↑ https://sourceforge.net/p/flightgear/mailman/message/37159361/
  6. ↑ https://sourceforge.net/p/flightgear/mailman/message/37159461/
  7. ↑ https://sourceforge.net/p/flightgear/mailman/message/37160096/
  8. ↑ https://sourceforge.net/p/flightgear/mailman/message/37158652/
  9. ↑ https://sourceforge.net/p/flightgear/mailman/message/37148460/
  10. ↑ https://sourceforge.net/p/flightgear/mailman/message/37157550/
  11. ↑ https://forum.flightgear.org/viewtopic.php?f=6&t=38334
  12. ↑ http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg28864.html
  13. ↑ http://forum.flightgear.org/viewtopic.php?p=146136#p146136
  14. ↑ https://sourceforge.net/p/flightgear/mailman/message/19718339/
  15. ↑ https://sourceforge.net/p/flightgear/mailman/message/37059117/
  16. ↑ https://sourceforge.net/p/flightgear/mailman/message/24811861/
  17. ↑ https://sourceforge.net/p/flightgear/mailman/message/36295606/
  18. ↑ https://sourceforge.net/p/flightgear/mailman/message/37059117/
  19. ↑ http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg28869.html
  20. ↑ http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg17263.html
  21. ↑ http://sourceforge.net/p/flightgear/mailman/message/19718339/
  22. ↑ http://sourceforge.net/p/flightgear/mailman/message/19718339/
  23. ↑ =http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg27134.html
Retrieved from "https://wiki.flightgear.org/w/index.php?title=CompositeViewer_support&oldid=129206"