Supporting multiple renderers

From FlightGear wiki
Jump to: navigation, search
Note  In its current form, this section/article is largely based on quotes collected from various related discussions/channels (devel list, forum etc) using the Instant-Cquotes script. Wiki users and other contributors are encouraged to help rewrite/edit contents accordingly to help get rid of unnecessary quoting (i.e. while the wiki is not intended to be a collection of quotes, quotes are sometimes the best/easiest way to bootstrap new articles, while also providing a good way to link back to related discussions in the archives).

While rewriting usually only entails changing first person speech to 3rd person. However, please try to retain references/links to the original discussion whenever possible.

Cquote1.png It would be very nice for lots of reasons to be able to separate the simulator and rendering into discrete processes (optionally!). My personal aim there is to be able to hack on an alternative renderer using more modern graphics techniques, but without jeopardising, or indeed touching the existing rendering code. Of course being able to run multiple viewers is also very desirable.
Cquote2.png
Cquote1.png Depsite appearances moving the rendering to a seperate node/process is probably the easiest to implement as the inter module dependencies aren't that strong, or at least those that are strong can be shipped as a block update at the end of the simulation frame.
— Richard Harrison (Nov 19th, 2015). Re: [Flightgear-devel] HLA developments.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png it would be good to have the IG's seperated from the physics engine. But that is a relatively huge task.
— Mathias  (May 14th, 2007). Re: [Flightgear-devel] More ideas on dogfighting.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png some people want a Space Shuttle simulator, others want to do nape-of-the-earth helicopter flying. IFR people don't care :) The point is that the rendering should be flexible enough to handle these different scenarios, especially given that so much work is going into building detailed terrain.
— Tim Moore (Nov 27th, 2013). Re: [Flightgear-devel] Rendering strategies.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png The important requirement for me is having the depth buffer and triangle normals available as input to following render stages
— Tim Moore (Nov 27th, 2013). Re: [Flightgear-devel] Rendering strategies.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png the kind of thing I would prefer to see done as a new renderer application once the HLA split is under way, so that the current forward-render + scenery continue to exist for older systems. Adding Rembrandt to the existing renderer codebase already added a considerable quantity of ‘if (rembrandt) { …. codepath } else { other codepath }’ complication. (It will also be much easier to hack on a new renderer when you don’t have to restart the sim every time)
— James Turner (Nov 26th, 2013). Re: [Flightgear-devel] New scenery.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png I will provide a different renderer sometime in the future where I aim to provide better isolation of this kind of renderer options so that people with very different aims can probably coexist better.
— Mathias Fröhlich (Jan 27th, 2013). Re: [Flightgear-devel] Color-shifts for textures.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png I hope once we have things split up people will be easier able to experiment with more radical rendering solutions. (Although, the current monolithic architecture hasn’t stopped Jeff from doing osgEarth integration, which proves anything is possible!)
— James Turner (Nov 26th, 2013). Re: [Flightgear-devel] New scenery.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png supporting the option of different viewer / rasteriser front-ends is by far the best solution here, since there are different requirements for different use-cases and target users. Plenty of people are interested in the kind of effects Thorsten's uber-shader and atmospheric model can provide, but plenty of others care more about a solid 60H (or even higher), and still others would prefer a solution that works with the fixed-funtion pipeline. And then there's Rembrandt as yet another rendering configuration. My plan in parallel with / after 2.12 is to work with Mathias' HLA code to make this separation possible
— James Turner (Jan 27th, 2013). Re: [Flightgear-devel] Color-shifts for textures.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png The solution probably lies along the lines of having different rendering back ends, as James alluded to.
— Tim Moore (Nov 27th, 2013). Re: [Flightgear-devel] Rendering strategies.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png concerning the larger issue of different rendering pipelines / approaches, my opinion is, and remains, that the long-term solution is separate viewer codebases - while a plethora would be bad, we would already benefit from a 'fixed-function, no shaders' renderer codebase distinct from a Rembrandt renderer and modern, forward-rendering OpenGL 3.x pipeline. This needs the viewer to be cleanly split out from the simulation backend, via HLA, which is exactly what Mathias (and soon, myself) are working towards, but slowly.
— James Turner (Apr 25th, 2013). Re: [Flightgear-devel] Atmospheric Light Scattering.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png I believe that we need to distinguish between different render to texture cameras. That ones that will end in mfd displays or hud or whatever that is pinned into models. And one that are real application windows like what you describe - additional fly by view, and so on. And I believe that we should keep that separate and not intermix the code required for application level stuff with building of 3d models that do not need anything application level code to animate the models ... I think of some kind of separation that will also be good if we would do HLA between a viewer and an application computing physical models or controlling an additional view hooking into a federate ...
— Mathias Fröhlich (Jul 1st, 2008). Re: [Flightgear-devel] RFC: changes to views and cameras.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png I want to start restructuring the codebase into multiple processes, so we can support different rendering architectures (and use multiple CPUs) better.
— James Turner (Mar 11th, 2013). Re: [Flightgear-devel] Generating ground textures on the fly?.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png moving onto more ambitious plans, including the Holy Grail of separating the viewer from the FDM.
— Stuart Buchanan (Nov 6th, 2015). [Flightgear-devel] HLA developments.
(powered by Instant-Cquotes)
Cquote2.png

Objective

Status

Background

Related Efforts

Cquote1.png A number of core developers have been wanting to unify the FGRenderer infrastructure so that different rendering schemes (think fixed-function vs. Rembrandt vs. ALS) can be prototyped/implemented and maintained
— Hooray (Dec 21st, 2015). Re: OpenIG Lights.
(powered by Instant-Cquotes)
Cquote2.png

Rembrandt (FredB)

Cquote1.png Internally, a Rembrandt buffer is not much different from any other RTT context - Canvas is all about rendering to a dynamic texture and updating it dynamically by modifying a sub-tree in the property tree - but its primary primitives are 1) osgText, 2) shivaVG/OpenVG paths, 3) static raster images, 3) groups/maps - none of these would be particularly useful in this context. But Zan's newcamera work could be turned into a new "CanvasCamera" element to allow camera views to be rendered to a Canvas, including not just scenery views - but also individual rendering stages. Canvas itself maintains a FBO for each texture, which is also the mechanism in use by Rembrandt. Tim's CameraGroup code is designed such that it does expose a bunch of windowing-related attributes to the property tree - equally, our view manager is property-controlled.
— Hooray (Oct 19th, 2014). Re: Orbital Makes the Sky Black.
(powered by Instant-Cquotes)
Cquote2.png

newcamera (Zan)

Cquote1.png A few years ago, even prior to Rembrandt, Zan was experimenting with the whole CameraGroup code to make it entirely XML-configurable: Subject: Improved camera configuration options
— Hooray (Dec 21st, 2015). Re: OpenIG Lights.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png I want to point out my work on my "newcameras" branch: https://gitorious.org/fg/zans-flightgear?p=fg:zans-flightgear.git;a=shortlog;h=refs/heads/newcameras which allows user to define the rendering pipeline in preferences.xml. It does not (yet?) have everything Rembrandt's pipeline needs, but most likely is easily enhanced to support those things. Basically this version adds support for multiple camera passes, texture targets, texture formats, passing textures from one pass to another etc, while preserving the standard rendering line if user wants that. I wish this work could be extended (or maybe even I can extend it ;) to handle the Rembrandt camera system. This will not solve all problems in the merge, but some of them.
— Lauri Peltonen (Mar 7th, 2012). [Flightgear-devel] [Rembrandt] the plan.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png Back then, Mathias, Fred and Tim were contemplating to use Zan's code as the foundation for features like Rembrandt: http://sourceforge.net/p/flightgear/mailman/message/28946515/
— Hooray (Dec 21st, 2015). Re: OpenIG Lights.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png I would like to extend the format to avoid duplicating the stages when you have more than one viewport. What I see is to specify a pipeline as a template, with conditions like in effects, and have the current camera layout refer the pipeline that would be duplicated, resied and positioned for each declared viewport
— Frederic Bouvier (Mar 7th, 2012). Re: [Flightgear-devel] [Rembrandt] the plan.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png Something like this would allow schemes like Rembrandt to be entirely prototyped/configured and implemented using XML, properties and effects/shaders - without requiring any major C++ changes. For the full discussion, see: http://sourceforge.net/p/flightgear/mailman/message/28939805/
— Hooray (Dec 21st, 2015). Re: OpenIG Lights.
(powered by Instant-Cquotes)
Cquote2.png

Canvas (TheTom)

Cquote1.png At some point, it is likely that this will be overlapping with ongoing Canvas developments, simply because Canvas encapsulates the whole concept of RTT/FBO by providing a SGPropertyChangeListener-based subsystem on top of OSG RTT/FBO functionality, so that it would make sense to adapt Zan's newcameras and Fred's Rembrandt code accordingly.
— Hooray (Dec 21st, 2015). Re: OpenIG Lights.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png this is a worthwhile thing to aim for, as it would help unify (and simplify) most RTT/FBO handling in SG/FG, and make this available to people like Thorsten who have a track record of doing really fancy, unprecedented stuff, with this flexibility.
— Hooray (Oct 17th, 2015). Re: WINDOW IN WINDOW.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png Both, Fred and Mathias, seemed pretty eager to adopt Zan's work back then:

https://www.mail-archive.com/flightgear-devel%40lists.sourceforge.net/msg36481.html

https://www.mail-archive.com/flightgear-devel%40lists.sourceforge.net/msg36486.html

Meanwhile, we ended up with Canvas as an abstraction mechanism for FBO management via properties - so integrating Canvas would indeed be a logical choice, unrelated to any particular manifestation like ALS or Rembrandt - integrating these technologies would primarily mean that new features could be prototyped without necessarily having to customie the hard-coded renderer logic - including things like our hard-coded skydome for example, which could be implemented in fgdata space then - which would not just be relevant for efforts like Earthview (orbital flight), but also make other things possible that would currently require a fair amount of tinkering with the C++ code.
— Hooray (Oct 19th, 2014). Re: Orbital Makes the Sky Black.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png the main challenge being the lack of accessibility when it comes to required structural changes to the C++ code - regardless of the concrete renderer - the lack of Rembrandt maintenance, and the slow response whenever ALS requires C++ level changes, is primarily because the corresponding renderer code is not being maintained actively - moving this into fgdata space via effects and shaders is a logical thing to do, and will allow people like Thorsten (or yourself) to make corresponding modifications without facing a core development bottleneck when it comes to Rembrandt/FGRenderer or any other $FG_SRC/Viewer modifications. The CameraGroup.cxx file is basically begging to be refactored sooner or later. None of this needs to involve Canvas, it would just be a straightforward and generic approach to do so, but certainly not mandatory - Zan's original work was implemented using directly XML and the property tree - however, Canvas contains a few helpers to make this increasingly straightforward, requiring very little in terms of code (e.g. PropertyBasedElement as a container for subsystems implemented on top of the property tree).
— Hooray (Oct 19th, 2014). Re: Orbital Makes the Sky Black.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png people would need to be convinced that they want to let Canvas evolve beyond the 2D use-case, i.e. by allowing effects/shaders per element, but also to let Cameras be created/controlled easily.

Personally, I do believe that this is a worthwhile thing to aim for, as it would help unify (and simplify) most RTT/FBO handling in SG/FG, and make this available to people like Thorsten who have a track record of doing really fancy, unprecedented stuff, with this flexibility. Equally, there are tons of use-cases where aircraft/scenery developers may want to set up custom cameras (A380 tail cam, space shuttle) and render those to an offscreen texture (e.g. GUI dialog and/or MFD screen).

It is true that "slaved views" are kinda limited at the moment, but they are also comparatively easy to set up, so I think that supporting slaved camera views via Canvas could be a good way to bootstrap/boost this development and pave the way for CompositeViewer adoption/integration in the future.
— Hooray (Oct 17th, 2015). Re: WINDOW IN WINDOW.
(powered by Instant-Cquotes)
Cquote2.png