CompositeViewer support: Difference between revisions

From FlightGear wiki
Jump to navigation Jump to search
Line 75: Line 75:
! Feature !! Status !! Notes
! Feature !! Status !! Notes
|-
|-
| Scenery/Terrain || {{Progressbar|70}} || buildings missing (this seems line line with Richard's findings mentioned at <ref>https://sourceforge.net/p/flightgear/mailman/message/36536293/</ref>)
| Scenery/Terrain || {{Progressbar|70}} || buildings missing (this seems line line with Richard's findings mentioned at <ref>https://sourceforge.net/p/flightgear/mailman/message/36536293/</ref><ref>https://sourceforge.net/p/flightgear/mailman/message/37004640/</ref>)
|-
|-
| Aircraft || {{Done}} || -
| Aircraft || {{Done}} || -

Revision as of 11:52, 31 July 2020

IMPORTANT: Some, and possibly most, of the features/ideas discussed below are likely to be affected, and possibly even deprecated, by the ongoing work on providing a property tree-configurable rendering pipeline accessible via XML using the new Compositor system available since FlightGear 2020.3 (11/2020): The main rendering pipeline (on next at least) is now the Compositor using ALS. The "classical renderer" and "Project Rembrandt" have been superseded. From the perspective of creating regional material definitions, you can just develop against ALS and you will be fine.[1]

Please see: Post FlightGear 2020.2 LTS changes for further information You are advised not to start working on anything directly related to this without first discussing/coordinating your ideas with other FlightGear contributors using the FlightGear developers mailing list or the Effects & Shaders subforum This is a link to the FlightGear forum.. talk page.

Compositor logo.png


CompositeViewer Support
Started in 07/2020 (early prototype)
Description Support for multiple independent scene views
Contributor(s) Julian Smith[2]
Status experimental (building, mostly working in single-view mode, buildings missing)
20}% completed [3]
Changelog https://sourceforge.net/u/cgdae/profile/feed.rss
Topic branches:
$FG_SRC https://sourceforge.net/u/cgdae/flightgear/ci/CompositeViewer


For the time being, FlightGear only supports one view position at a time. Multiple independent view positions (e.g. one screen for the tower view and a second screen for the plane) would complicate a "locked to cache" flag quite a lot...[4].

Aircraft can define their own views and so on. But only one view can be active at a time. So no matter how many windows and cameras you define in Defaults.xml, they all are relative to the current view in FG (i.e. cockpit, tower...). [5]

Back in 2008, Tim Moore provided a patch (mailing lists search for CameraGroup FlightGear Mailing Lists) to use the osgViewer class to set up windows, manage the main camera, etc. [6]

However, these windows have to use the same camera group as the main window so can only show the view from the same eye position, though typically at a different angle/offset so that one can emulate things like side windows of a cockpit displayed in a different window or monitor.[7]

There is some tight coupling between the Renderer.cxx and CameraGroup files (now to be found in SimGear[8]) The camera-infos can be configured from XML, eg to additional post-processing stages, but within each CameraInfo the actual osg::Cameras (each of which is really a rendering pass/stage) is determined by one of the buildXXXXX methods.[9]

Mathias Fröhlich used the slave camera feature of osgViewer to provide a "video wall" style of multiple displays that was demonstrated at LinuxTag for years. Later on, Tim generalized this to support general monitor arrangements (like a panoramic arc) and general combinations of screens and graphics cards. [10]

The default OSG model is that slave cameras are different views offset from a common viewpoint. This is easy to understand when considering a camera's view matrix, but not necessarily intuitive when thinking about the projection matrix. Because FG has its own view system we mostly treat the slaves as independent. It seems that most other uses of cameras during rendering -- for example, render to texture cameras for effects -- are best handled by slave cameras with independent views as well.[11]

People requiring multiple independent views on the same scenery, e.g. cockpit and tower view [...] these each need their own camera groups and so require OSG's CompositeViewer.[12]

And that's not really supported by the current architecture, neither by the tile cache nor by osgViewer::Viewer. We would need to move to a CompositeViewer model, which supports several scene graphs, and rely completely on the osg database paging machinery.[13]

That would require a change in current fg architecture to use a CompositeViewer instead of a single Viewer, but we're contemplating that anyway.[14]

The cameras in a camera group don't need to render directly to the screen. They can render to a texture which can be used either in the scene, like in a video screen in the instrument panel, or for distortion correction in a projected or dome environment. [15]

Open Scene Graph supports a CompositeViewer object that supports rendering from several widely separated viewpoints, complete with support for multiple terrain pager threads. We could move to CompositeViewer and support simultaneous views from e.g., the tower, AI models, drones, etc.[16]

Neither of these are supported at the present time, but it would be a good project. We would have to start using a different OSG class, CompositeViewer, to support multiple views from independent view points. Our terrain pager would need a complete overhaul to use the PagedLOD scheme of OSG, and the Flightgear View manager would need to be aware multiple active views.[17]

Status

In 07/2020, Julian Smith reported some success by crudely changing flightgear's FGRenderer's osgViewer::Viewer to an osgViewer::CompositeViewer with a single osgViewer::View, and patching up all the calling code so it compiles.

It all builds ok, and he now got fgfs mostly working with CompositeViewer and the usual single View.[18]

Julian made the use of CompositeViewer configurable at runtime, which might simplify getting involved in testing/further development - the same binary can now be used for normal use and for investigating CompositeViewer behaviour.

To execute the patched binary with the compositeviewer enabled, use: fgfs --composite-viewer=1 [19]

Unfortunately the property system seemed to be not set up early enough to read in FGRenderer's constructor so we do a few calls of fgGetNode("/sim/composite_viewer", true)->getBoolValue() at runtime. Julian suspects they aren't called very frequently, in which case this branch (minus the diagnostics changes probably) might be ok to push to next at some point, so that it's easier to test/investigate things.[20]

The previous problem with loading scenery was caused by code using the single View's getFrameStamp(), which appears to be never updated and always has _frameNumber=0. We need to use the CompositeViewer's getFrameStamp() instead.

For some reason buildings don't appear, but basic ground scenery now loads so fgfs progresses to loading the fdm etc and one can fly the aircraft as normal.

Note  According to Richard, the main problem is the way that random objects (buildings, vegetation) are inserted into the scene graph doesn't seem to be compatible with lots of pager threads.[21]
Feature Status Notes
Scenery/Terrain 70}% completed buildings missing (this seems line line with Richard's findings mentioned at [22][23])
Aircraft Done Done -
Skydome Done Done -
PUI GUI Done Done -


Potential Issues

So far, it seemed we 'Cant Use' CompositeViewer because of $reasons in the Effects code[24]. Maybe the reason is simply that it needs some work to move away from the single CameraGroup, however.[25]

Also, a number of senior contributors have previously pointed out how our existing tile manager/paging infrastructure may not yet be in shape for a CompositeViewer port, poweroftwo's osgEarth port however is using OSG's PagedLOD exclusively, and osgEarth itself is known to support CompositeViewer[26], too: https://gitlab.com/poweroftwo

So, to troubleshoot this further, it'd make sense to exclude the usual suspects one by one:

  • disable multi-threading
  • disable scenery/terrain
  • use Draw masks
  • disable effects

At that point, we should be able to nail down where the culprit is, i.e. what needs changing.

Jules shared the patches (topic branch, see above) so that more people can get involved in testing and making progress.[27]

OSG stuff

Note  As a general note, osgViewer::CompositeViewer should have osgViewer::View attached to it NOT osgViewer::Viewer. It's a composite of View's not composite of Viewer. If you look at all the OSG example they illustrate the correct usage.[28]

The OSG's osgViewer is designed to support multiple views on to one or more scenes using the osgViewer::CompositeViewer class. See the osgcompositeviewer example.[29]

The CompositeViewer was written specifically for multiple Views.

Robert wouldn't recommend using slave Camera's for doing multiple views, while possible it's just a mess in terms of management. slave Camera's are tools for helping rendering a single view, but with a view that is composed of several components - either spread across multiple windows, or a view that requires multiple passes such as distortion correction, field of view etc.[30]

The osgViewer has a mechanism for avoid multiple traversals of shared scene graphs if mutlple View's share the same root node of the scene graph. If shared component isn't the topmost node then the OSG has no straight forward way to know whether a subgraph has been traversed or not that frame. One could implement a mechanism to avoid this visiting a node multiple times in one frame but it would be really costly to do, an expense that would only be a benefit for a very small number of users, but would slow performance for everyone else.


The most efficient way to avoid this multiple traversals issue is to implement to have a custom callback that is tailored to the specific usage case that a user has. I don't know enough about the specific callbacks and scene graph set up you have so I can't pinpoint the best route.

If you have a shared subgraph that you don't want traversed multiple times per frame then use an UpdateCallback that has a frameNumber member variable that keep track of the the frameNumber (use NodeVisitor::getFrameStamp()'s FrameNumber) of the last traversal, when a traversal calls the update callback you only traverse the subgraph if the frameNumber is different and then set the frameNumber to the present frame, if the frameNumber is the same then you just return immediately. This custom UpdateCallback you'd place as high as you can in your scene graph to make sure the traversal stops as soon as possible.

Another approach is to move this frameNumber tracking into your existing update callbacks, and simple return right away with the frameNumber is the same. This requires a small tweak to the callbacks but is such a small change it's generally pretty easy to integrate.

Finally you can simple make your callbacks resilient so that the are no ill effects from being called multiple times.[31]

  • For multiple views the way to do it with the OSG is to use the osgViewer::CompositeViewer class, and then each osgViewer::View you set up the approrpiate Camera's CullMask.[32]
  • The draw traversal doesn't touch the scene graph nodes so NodeMasks/TraversalMask have no baring on it. The OSG's cull traversal builds the internal rendering graphs each frame and the draw traversal simply traversals these rendering graphs. So to control the draw traversal you use the masks in the cull traversal.[33]
  • shader handling with multiple views: The OSG provides equivalents to the OpenGL built ins for you so you don't need to provide them yourself, so just ditch the callbacks and

just use gl_ModelViewMatrix or osg_ModelViewMatrix. The OSG has provision for doing automatic conversion from gl_ModelViewMatrix to osg_ModelViewMartrix, it's enabled by default for GL3 and GLES2 builds, but if not then you can enable if you want to use it. See the osgsimplegl3 example. These uniforms that the OSG provides for the projection and modelview matrices work seamlesly across multiple views. Just use them and it'll work out of the box. [34]

The two options for off screen rendering are a PixelBuffer context or FrameBufferObject, if you have an existing on screen window in your application then using a FrameBufferObject becomes preferable. The way to do it would be to set up your viewer's off screen Camera with FBO settings and a custom final draw callback to do the read to main memory. The read will be more efficient if you use a pair of PixelBufferObjects - the osgscreencapture example illustrates this in action.


When already using CompositeViewer the most natural thing to do would be to have a dedicated View with it's master Camera as the offscreen camera, this way you can control the Camera's view matrix in straight forward manner in the same way to the rest of the Camera's. Also during debugging having the option of making this Camera an onscreen one would give you means to visually QA things as you go along.[35]

Troubleshooting

Julian also added lots of diagnostics https://sourceforge.net/u/cgdae/flightgear/ci/eea77c18e29a7a8660a0e4b61fa687d937d0997e/

Use Cases

Cquote1.png I have to create two master cameras to controls the two different views in two different scene rendering dynamically.


but flightgear only using viewer class . This is createing one master camera no.of slave cameras. but i need CompositeViewer class. how to use CompositeViewer class in flightgear and how to render through CompositeViewer class.


Cquote2.png


Cquote1.png What i want to achieve: Multiple views of the same scene . Lets call them View1 and View2(they may be on 2 separate screens). There should be only one instance of flightgear running. I should be able to have "View1 = Cockpit View & View2 = Chase View" or "View1 = Helicopter View & View2 = Cockpit View" or any other such combination of views. I am using FlightGear 2.0.0 I should be able to switch the view in each viewport without affecting the view in the other view port. i.e i should be able to double right click and change the view on each display with the mouse without affecting the view in the other display. README.multiscreen in the FlightGear documentation says its possible to create a master camera and a slave camera that is offset from the master camera. BUT I want 2 master cameras that can switch to any view at any time.[36]
Cquote2.png
Cquote1.png I have just been trying out the multiple screen feature in FG. I found that the GUI camera (including the menu bar, hud and 2D panel) appears in only

one of the windows. Is there any way I can make the GUI to appear in all the windows? Actually I want to be able to view the hud and 2D panel in all the windows.

Also when I change the view in any one of the windows, the view changes in the other windows as well. Is it possible to make the windows independent of each other. I want to display the cockpit in one window at all times, in the second window I want to be able to shuttle between helicopter / chase or model views.

Also I have observed that in the second screen where I'm displaying lets say the Helicopter view the aircraft remains static while the environment moves. This is because the cockpit view in my Master screen is defined as 'lookfrom'. Can I define 'lookfrom' for one screen and 'lookat' for the

other screen.[37]
— Kavya Meyyappan
Cquote2.png

Known Limitations

There are a few limitations of the FlightGear multi-camera/view/display system. Tim Moore is the person who developed this feature (nothing existed before his efforts) and maybe he can offer more insight.


1rightarrow.png See FlightGear and OpenGL ES for the main article about this subject.

For example, in the case of:

there would need to be some significant code restructuring to allow these to be displayed on other windows.[38]

There's a limitation in Plib that forces the GUI to be drawn on one window.[39]

Related

Docs

Discussions

Search

References

References
  1. https://sourceforge.net/p/flightgear/mailman/message/37758886/
  2. https://sourceforge.net/p/flightgear/mailman/message/37062319/
  3. https://sourceforge.net/p/flightgear/mailman/message/37070203/
  4. http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg28864.html
  5. http://forum.flightgear.org/viewtopic.php?p=146136#p146136
  6. https://sourceforge.net/p/flightgear/mailman/message/19718339/
  7. https://sourceforge.net/p/flightgear/mailman/message/37059117/
  8. v
  9. https://sourceforge.net/p/flightgear/mailman/message/35828189/
  10. https://sourceforge.net/p/flightgear/mailman/message/24811861/
  11. https://sourceforge.net/p/flightgear/mailman/message/36295606/
  12. https://sourceforge.net/p/flightgear/mailman/message/37059117/
  13. http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg28869.html
  14. http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg17263.html
  15. http://sourceforge.net/p/flightgear/mailman/message/19718339/
  16. http://sourceforge.net/p/flightgear/mailman/message/19718339/
  17. =http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg27134.html
  18. https://sourceforge.net/p/flightgear/mailman/flightgear-devel/
  19. https://sourceforge.net/u/cgdae/flightgear/ci/059cf073c74d43a57c11f363cae6c9271772d869
  20. https://sourceforge.net/p/flightgear/mailman/message/37062319/
  21. https://sourceforge.net/p/flightgear/mailman/message/36536293/
  22. https://sourceforge.net/p/flightgear/mailman/message/36536293/
  23. https://sourceforge.net/p/flightgear/mailman/message/37004640/
  24. https://sourceforge.net/p/flightgear/mailman/message/26189393/
  25. https://sourceforge.net/p/flightgear/mailman/message/37059939/
  26. https://github.com/gwaldron/osgearth/issues/1300#issuecomment-565734550
  27. https://sourceforge.net/p/flightgear/mailman/message/37058676/
  28. https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg70462.html
  29. https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg69640.html
  30. https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg74474.html
  31. https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg74476.html
  32. https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg78309.html
  33. https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg78308.html
  34. https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg75995.html
  35. https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg70703.html
  36.  (). Multiple views without slaving to master camera.
  37. Kavya Meyyappan (Fri, 19 Mar 2010 03:31:50 -0700). [Flightgear-devel] Help needed with multi-screen.
  38. http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg27114.html
  39. http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg27134.html