Project Rembrandt: Difference between revisions

Jump to navigation Jump to search

Project Rembrandt (view source)

Revision as of 15:01, 17 February 2019

12,516 bytes removed ,  17 February 2019
m
→‎Status (07/2014): obsolete due to Compositor
m (→‎Status (07/2014): obsolete due to Compositor)
(7 intermediate revisions by 3 users not shown)
Line 1: Line 1:
<!-- {{WIP}} -->
== Why the name? ==
 
{{Wikipedia|Rembrandt}} was a dutch painter living in the 17th century, famous as one of the master of {{Wikipedia|chiaroscuro}}.
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].


This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.
This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.
Line 8: Line 6:
{{forum|47|Effects & Shaders}}
{{forum|47|Effects & Shaders}}


== Status (07/2014) ==
== What is it ? ==
For the time being, Rembrandt's main developer (FredB) is not actively involved in FlightGear, and Rembrandt must be considered "unmaintained" currently. However, there are 2-3 overlapping efforts, some of which were discussed on the developers mailing list. First of all, there's Zan's newcamera branch, which has never been merged with FlightGear, but which exposes FlightGear's rendering pipelines to XML/fgdata space via the effects framework - which basically means that large portions of Rembrandt could be moved out of the C++ code back into GLSL/effects space, which is something that was originally discussed by a few core developers. This has never materalized, because Zan's work took place pretty much at the same time when Rembrandt was being developed. Likewise, some technical challenges could be easily addressed by exposing each rendering stage to a [[Canvas]] texture, which would mainly require two new features: 1) [[Canvas_Development#Supporting_Cameras|supporting camera views as a new Canvas::Element]], and 2) [[Canvas_Development#Effects_.2F_Shaders|supporting effects/shaders]] per Canvas/Element.
From then on, most [[Howto:Use_a_Camera_View_in_an_Instrument|Rembrandt-like functionality, including mirror views, tail cameras or FLIR-like processing]], could be fully implemented and maintained by fgdata contributors familiar with the effects framework and shaders, without necessarily being specific to just deferred rendering: 


{{FGCquote
{{FGCquote
   |Personally, I see the potential of deferred rendering, but if it ends up with 15 fps on a gaming rig, then that's what it's going to remain - potential waiting for the next or next-to-next hardware generation. For me personally, real secondary light sources and shadows aren't a big deal. They're important on the ground, at an airport, but I'm usually flying. <br/>
   |Deferred {{=}} rendering a geometry pass into the geometry (g-)buffer which holds positions, normals, materials, then doing all operations (texel, light, ...) in fragment passes on that buffer (or any others).<br/>
<br/>
<br/>
Self-shadowing of the terrain can be done in 90% of the cases accurately without Rembrandt because the terrain is rarely that steeply sloped, and the interesting other shadows in nature - clouds and trees - can't be done by Rembrandt in any case.
Forward {{=}} (usually) rendering part of the shading (typically ambient and diffuse light) per vertex, extrapolate all quantities  which are declared as varying across triangles and hand them via rasterizer to the fragment shader which does the rest
   |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=204372#p204372
   |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=221222#p221222
     |title=<nowiki>Re: A long shot</nowiki>
     |title=<nowiki>Re: Orbital Makes the Sky Black</nowiki>
     |author=<nowiki>Thorsten</nowiki>
     |author=<nowiki>Thorsten</nowiki>
     |date=<nowiki>Thu Mar 27</nowiki>
     |date=<nowiki>Sun Oct 19</nowiki>
  }}
}}
 
{{FGCquote
  |Rembrandt is just one  particular (and experimental) implementation of deferred rendering implemented via FlightGear - when running other simulators/games -with comparable scenes (not necessarily 400km of visibility!)- you'll see that performance isn't typically that much affected. So I guess there's nothing wrong about Rembrandt per se, but it just isn't very optimized at the moment, and it not being maintained doesn't really help. A few weeks ago when we exchanged screen shots and Rembrandt stats, there were apparently a few low-hanging fruits, and Rembrandt was obviously doing stuff that wasn't really necessary (e.g. computing shadows at night time without any light sources within 50km...)
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=204374#p204374
    |title=<nowiki>Re: A long shot</nowiki>
    |author=<nowiki>Hooray</nowiki>
    |date=<nowiki>Thu Mar 27</nowiki>
  }}
}}
 
{{FGCquote
  |The situations I'm aware of were deferred rendering is used are frequently first person shooters. From the ground, where you have plenty of objects obscuring each other, deferred rendering would give you a pronounced boost in framerate as you spend minimal time with vertices you're not going to see.  That advantage never really materializes for a flightsim.<br/>
<br/>
Imo, what makes our performance in comparison worse than other software where a large scene is rendered is the lack of a terrain LOD system. Our vertex count just explodes at high visibility. <br/>
<br/>
I mean, FredB obviously spent quite some time with Rembrandt. If someone can make it run much faster, than that'd change the game, but so far we have mostly talk.
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=204375#p204375
    |title=<nowiki>Re: A long shot</nowiki>
    |author=<nowiki>Thorsten</nowiki>
    |date=<nowiki>Thu Mar 27</nowiki>
  }}
}}
 
{{FGCquote
  |Given how FlightGear has evolved over time, not just regarding effects/shaders, but also complementary efforts like deferred rendering (via rembrandt), we'll probably see cameras (and maybe individual rendering stages) exposed as Canvases, so that there's a well-defined interface for hooking up custom effects/shaders to each stage in the pipeline - Zan's newcamera work demonstrates just how much flexibility can be accomplished this way, basically schemes like Rembrandt could then be entirely maintained in XML/effects and shader (fgdata) space.<br/>
And even the fgviewer code base could be significantly unified by just working in terms of canvases that deal with camera views, which also simplifies serialization for HLA.
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=216741#p216741
    |title=<nowiki>Re: create window</nowiki>
    |author=<nowiki>Hooray</nowiki>
    |date=<nowiki>Sat Aug 16</nowiki>
  }}
}}
{{FGCquote
  |So we would need to extend CanvasElement then to support effects - according to some of your postings, you played with that already a while ago using code from SGMaterialAnimation apparently, as in 2 years ago ? Is that something that we should reuse ?<br/>
<br/>
Specifically, I've been in touch with some of our effects/shader guys, and we're wondering if we could directly register a Canvas/Effects texture as a texture available to effects, which is something that Thorsten and Tim talked about a while ago, being a limitation in effects that they would like to get rid of
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=201852#p201852
    |title=<nowiki>Re: </nowiki>
    |author=<nowiki>Hooray</nowiki>
    |date=<nowiki>Tue Feb 25</nowiki>
  }}
}}
 
{{FGCquote
  |this all predates Canvas - but if we could not just support glsl/effects in Canvas, but also expose those canvas textures to the effects system, that would bring a lot of possibilities
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=201852#p201852
    |title=<nowiki>Re: </nowiki>
    |author=<nowiki>Hooray</nowiki>
    |date=<nowiki>Tue Feb 25</nowiki>
  }}
}}
 
{{FGCquote
  |regarding Rembrandt in particular, we are only missing 2-3 [[Canvas]] enhancements to allow effects/shader guys to pick up where FredB left off - which are primarily,
# camera support
# effects/shader support
 
And people have this working in various stages already, sometimes unrelated to [[Canvas]] (Zan's newcameras branch), and sometimes not yet fully integrated (e.g. the shader stuff). But generally it is foreseeable that additional rendering schemes (deferred rendering) could then be implemented by people without having to touch any C++ code.
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=204338#p204338
    |title=<nowiki>Re: A long shot</nowiki>
    |author=<nowiki>Hooray</nowiki>
    |date=<nowiki>Wed Mar 26</nowiki>
  }}
}}
 
{{FGCquote
  |So, it's not really Rembrandt that's the issue here, the bottleneck is the number of active expert C++ developers (you cannot really touch the 3D parts of the simulator with "just" C++ knowledge, you also need strong maths skills and OpenGL/OSG 3D rendering experience), especially once they're no longer around to help integrate their code and if there's little documentation to help newcomers maintain their code, because that's when base package development is often taking over eventually.
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=194296#p194296
    |title=<nowiki>Re: What frame rate (fps) do you have?</nowiki>
    |author=<nowiki>Hooray</nowiki>
    |date=<nowiki>Mon Nov 18</nowiki>
   }}
   }}
}}
}}
{{FGCquote
  |For example, I think it's pretty safe to say that the Canvas system was never intended to "compete" with Rembrandt, and most people here will be wondering how it could possibly be competing at all ? But ultimately it really is an integration and accessiblity issue, too. Admitedly, you need to be familiar with the way the project works to foresee what could be happening here, but it's not really far-fetched at all:<br/>
<br/>
I had a quick look, and according to the initial Rembrandt commits by FredB, they were all about decoupling rendering into multiple stages and doing offscreen rendering to textures. This is also supported by the wiki and the initial Rembrandt discussions on the devel list. So far, straightforward to understand - even without knowing any GLSL.<br/>
<br/>
So basically it's about decoupling each rendering step and rendering each stage to a separate offscreen texture, and then it's mostly about exposing each rendering step/stage to the effects/GLSL framework for further processing there.<br/>
<br/>
Rembrandt obviously predates the Canvas system, but these days that's exactly what the Canvas is good at: offscreen rendering to an osg::Image, without having to touch any C++ code, and without having to build FlightGear from source.<br/>
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=194296#p194296
    |title=<nowiki>Re: What frame rate (fps) do you have?</nowiki>
    |author=<nowiki>Hooray</nowiki>
    |date=<nowiki>Mon Nov 18</nowiki>
  }}
}}
{{FGCquote
  |Tim Moore (developer of the effects framework) did mention several times how it would be great to extend the effects framework to allow  arbitrary textures to be used as inputs/outputs for the effects frame. For example, see: [http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg37873.html http://www.mail-archive.com/flightgear- ... 37873.html]<br/>
<br/>
So consider a Canvas texture as the potential input/output for an effect, and there you have it: Canvas + effects/shader {{=}}&gt; one important ingredient for a Canvas-based Rembrandt port.
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=194296#p194296
    |title=<nowiki>Re: What frame rate (fps) do you have?</nowiki>
    |author=<nowiki>Hooray</nowiki>
    |date=<nowiki>Mon Nov 18</nowiki>
  }}
}}
{{FGCquote
  |we also have dozens of feature requests from aircraft developers to allow scenery views to be rendered as cockpit textures, e.g. to render mirrors or tail camera images. A feature also commonly seen on FSX and XPlane. <br/>
Several years ago, Zan made lots of progress doing this sort of thing, long before the Canvas again - so it's also an integration issue.<br/>
<br/>
These days, this would require the view_mgr code to be adapted and turned into a Canvas::Element. For example, see:<br/>
[http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg36673.html http://www.mail-archive.com/flightgear- ... 36673.html]<br/>
[[Howto:Use_a_Camera_View_in_an_Instrument]]<br/>
<br/>
Other aircraft developers have asked for effects/GLSL being available to cockpit textures for other reasons, e.g. to implement night vision views/FLIR imagings. And supporting effects/shaders in Canvas has been discussed various times on the forum too.<br/>
<br/>
So, these are just two new features: 1) making effects/GLSL usable to Canvas textures and 2) rendering scenery views - and suddenly you have two major ingredients for a deferred shading scheme in place, without requiring extensive C++ changes, and without necessarily having to even look at Rembrandt ...
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=194296#p194296
    |title=<nowiki>Re: What frame rate (fps) do you have?</nowiki>
    |author=<nowiki>Hooray</nowiki>
    |date=<nowiki>Mon Nov 18</nowiki>
  }}
}}
{{FGCquote
  |I don't think that anybody is working on replacing Rembrandt, but I also don't think that we have any active core developers who could easily help maintain Rembrandt (except for FredB obviosuly) - still it is foreseeable that the Canvas system will -eventually- provide all essential building blocks to allow deferred rendering to be built on top, it's really just a matter of time.<br/>
<br/>
At that point revisiting the Rembrandt code and moving its rendering stages into cascaded Canvas textures that make use of scenery views and effects/shaders with an osgViewer-based placement, would be straightforward for any GLSL developer, much more so than getting to grips with the underlying Rembrandt C++ code itself. <br/>
<br/>
So, nobody  is working on revamping/replacing Rembrandt, but even if it  were to continue to be stalled, the idea could still evolve - C++ templates were also never intended to be turing-complete, it just happened by accident :lol:<br/>
<br/>
As we know, even for good ideas, it usually takes ~3-5 years for them to be implemented (remember: OSG port, Rembrandt, Canvas etc).<br/>
<br/>
So I still wouldn't hold my breath. But re-considering the way Rembrandt is done and integrating it with the Canvas system would help make it less obscure and more accessible to effects/GLSL developers, especially for times when there are no C++/3D experts around.
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=194296#p194296
    |title=<nowiki>Re: What frame rate (fps) do you have?</nowiki>
    |author=<nowiki>Hooray</nowiki>
    |date=<nowiki>Mon Nov 18</nowiki>
  }}
}}
== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading  
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading  
Line 196: Line 55:


== Caveats ==
== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.
Deferred rendering is not able to display transparency. For the moment, clouds are rendered separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.


It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.
It also doesn't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.


The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.
The glow pass can make certain MFDs (that use emissive color) unreadable because of blurring. Should be treated as transparent.


== Implementation ==
== Implementation ==
Line 221: Line 80:


<syntaxhighlight lang="xml">
<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
<map-size type="int">8192</map-size>
</syntaxhighlight>
</syntaxhighlight>


And put 4096 or 2048 instead.
And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048
You can also use a startup parameter <code>--prop:/sim/rendering/shadows/map-size=2048</code>


=== Configurable pipeline ===
=== Configurable pipeline ===
The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :
The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :


Line 706: Line 564:


If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.
If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.
A surface that uses the [[ALS technical notes#ALS glass effect|ALS glass effect]] does not need to be separately registered for Rembrandt, this is done automatically.


=== Making sure that all geometry will cast shadow ===
=== Making sure that all geometry will cast shadow ===
Retrieved from "https://wiki.flightgear.org/Special:MobileDiff/77229...117372"

Navigation menu