Canvas view camera element: Difference between revisions

Jump to navigation Jump to search
(15 intermediate revisions by the same user not shown)
Line 5: Line 5:
|name        = Canvas Camera Views
|name        = Canvas Camera Views
|started    = 08/2016 (prototyped by F-JJTH)
|started    = 08/2016 (prototyped by F-JJTH)
|description = adds support for rendering slave scenery views to a FBO/RTT (texture)
|description = adds support for rendering slave scenery views to a FBO/RTT (offscreen-texture)
|status      = experimental/known issues
|status      = experimental/known issues (being prepared for review/integration)
|maintainers = none
|maintainers = none
|developers  = F-JJTH, Icecode GL, Hooray
|developers  = F-JJTH, Icecode GL, Hooray, cyrfer <ref>{{cite web  |url    =  https://forum.flightgear.org/viewtopic.php?p=320012#p320012  |title  =  <nowiki> Re: getting started with RTT </nowiki>  |author =  <nowiki> cyrfer </nowiki>  |date  =  Oct 5th, 2017  |added  =  Oct 5th, 2017  |script_version = 0.36  }}</ref><ref>{{cite web  |url    =  https://forum.flightgear.org/viewtopic.php?p=324494#p324494  |title  =  <nowiki> Re: Canvas:View development </nowiki>  |author =  <nowiki> cyrfer </nowiki>  |date  =  Dec 14th, 2017  |added  =  Dec 14th, 2017  |script_version = 0.36  }}</ref>, Stuart (review <ref>{{cite web
  |url    =  https://forum.flightgear.org/viewtopic.php?p=317967#p317967
  |title  =  <nowiki> Re: Gear view in cockpit computer </nowiki>
  |author =  <nowiki> stuart </nowiki>
  |date  =  Sep 1st, 2017
  |added  =  Sep 1st, 2017
  |script_version = 0.40
  }}</ref>)
<!--
|topic-sg = https://sourceforge.net/u/fgarlin/simgear/
-->
}}
}}


Line 15: Line 25:


== Summary ==
== Summary ==
{{Main article|Compositor}}


Several aircraft developers have manifested their interest in being able to render to a texture (RTT) {{Wikipedia|Render target}} and use it inside cockpits as mirrors, external cameras (so called tail cams) and other uses. Effects and shaders developers have also reached a point where ignoring RTT is a waste of resources and a limitating factor when creating new effects. Although this Canvas element is directed mainly towards the first need, it is a first step forward in terms of finally exposing Render To Texture capabilities to non-C++ space too.
Several aircraft developers have manifested their interest in being able to render to a texture (RTT) {{Wikipedia|Render target}} and use it inside cockpits as mirrors, external cameras (so called tail cams) and other uses. Effects and shaders developers have also reached a point where ignoring RTT is a waste of resources and a limitating factor when creating new effects. Although this Canvas element is directed mainly towards the first need, it is a first step forward in terms of finally exposing Render To Texture capabilities to non-C++ space too.
Line 22: Line 33:
* Mirrors
* Mirrors
* In-sim view configuration
* In-sim view configuration
* On demand creation of views and windows
* On demand creation of views and windows (e.g. [[FGCamera]] previews)
* Prototyping/testing HUDs or PFDs requiring synthetic terrain to work properly
* Prototyping/testing HUDs or PFDs requiring synthetic terrain to work properly


Line 66: Line 77:
   |date  =  Sep 2nd, 2017  
   |date  =  Sep 2nd, 2017  
   |added  =  Sep 2nd, 2017  
   |added  =  Sep 2nd, 2017  
  |script_version = 0.40
  }}</ref>
it would be possible to also come up with an XML-configurable rendering pipeline - in fact, Zan once came up with a modified CameraGroup design that would move the whole hard-coded pipeline to XML space: [[Canvas Development#Supporting Cameras]]<ref>{{cite web
  |url    =  https://forum.flightgear.org/viewtopic.php?p=275228#p275228
  |title  =  <nowiki> Re: Review of FG on reddit: xpost </nowiki>
  |author =  <nowiki> Hooray </nowiki>
  |date  =  Feb 6th, 2016
  |added  =  Feb 6th, 2016
  |script_version = 0.40
  }}</ref>
Ideally, something like this would be integrated with the existing view manager, i.e. using the same property names (via property objects), and then hooked up to CanvasImage, e.g. as a custom '''camera://''' protocol (we already support canvas:// and http(s)://)
So some kind of dedicated CanvasCamera element would make sense, possibly inheriting from CanvasImage.
And it would also make sense to look at Zan's new-cameras patches, because those add tons of features to CameraGroup.cxx
This would already allow arbitrary views slaved to the main view (camera)
<ref>{{cite web
  |url    =  https://forum.flightgear.org/viewtopic.php?p=261665#p261665
  |title  =  <nowiki> Re: WINDOW IN WINDOW </nowiki>
  |author =  <nowiki> Hooray </nowiki>
  |date  =  Oct 25th, 2015
  |added  =  Oct 25th, 2015
  |script_version = 0.40
  }}</ref>
As has been said previously, the proper way to support "cameras" via Canvas is using CompositeViewer, which does require a re-architecting of several parts of FG: [[CompositeViewer Support]]
Given the current state of things, that seems at least another 3-4 release cycles away.
So, short of that, the only thing that we can currently support with reasonable effort is "slaved views" (as per $FG_ROOT/Docs/README.multiscreen).
That would not require too much in terms of coding, because the code is already there - in fact, CameraGroup.cxx already contains a RTT/FBO (render-to-texture) implementation that renders slaved views to an offscreen context. This is also how Rembrandt buffers are set up behind the scenes.
So basically, the code is there, it would need to be extracted/genralied and turned into a CanvasElement, and possibly integrated with the existing view manager code.
And then, there also is Zan's newcameras branch, which exposes rendering stages (passes) to XML/property tree space, so that individual stages are made accessible to shaders/effects.
Thus, most of the code is there, it is mainly a matter of integrating things, i.e. that would require someone able to build SG/FG from source, familiar with C++ and willing/able to work through some OSG tutorials/docs to make this work<ref>{{cite web
  |url    =  https://forum.flightgear.org/viewtopic.php?p=260810#p260810
  |title  =  <nowiki> Re: WINDOW IN WINDOW </nowiki>
  |author =  <nowiki> Hooray </nowiki>
  |date  =  Oct 17th, 2015
  |added  =  Oct 17th, 2015
  |script_version = 0.40
  }}</ref>
Canvas is/was primarily about exposing 2D rendering to fgdata space, so that fgdata developers could incorporatedevelop and maintain 2D rendering related features without having to be core developers (core development being an obvious bottleneck, as well as having  significant barrier to entry).
In other words, people would need to be convinced that they want to let Canvas evolve beyond the 2D use-case, i.e. by allowing effects/shaders per element, but also to let Cameras be created/controlled easily.
Personally, I do believe that this is a worthwhile thing to aim for, as it would help unify (and simplify) most RTT/FBO handling in SG/FG, and make this available to people like Thorsten who have a track record of doing really fancy, unprecedented stuff, with this flexibility.
Equally, there are tons of use-cases where aircraft/scenery developers may want to set up custom cameras (A380 tail cam, space shuttle) and render those to an offscreen texture (e.g. GUI dialog and/or MFD screen).
It is true that "slaved views" are kinda limited at the moment, but they are also comparatively easy to set up, so I think that supporting slaved camera views via Canvas could be a good way to bootstrap/boost this development and pave the way for CompositeViewer adoption/integration in the future. <ref>{{cite web
  |url    =  https://forum.flightgear.org/viewtopic.php?p=260810#p260810
  |title  =  <nowiki> Re: WINDOW IN WINDOW </nowiki>
  |author =  <nowiki> Hooray </nowiki>
  |date  =  Oct 17th, 2015
  |added  =  Oct 17th, 2015
   |script_version = 0.40  
   |script_version = 0.40  
   }}</ref>
   }}</ref>
Line 77: Line 140:
| [[File:Canvas-view-element-prototype-by-icecode gl.png|thumb|upright|[[Canvas]] gui dialog with a custom canvas element to display view manager views (based on code prototyped by F-JJTH)]]
| [[File:Canvas-view-element-prototype-by-icecode gl.png|thumb|upright|[[Canvas]] gui dialog with a custom canvas element to display view manager views (based on code prototyped by F-JJTH)]]
|}
|}
== Getting involved ==
Icecode GL has been quite interested in this subject lately. [...] For shader support it'd use the current effects framework, and to display stuff on the screen it'd either use Canvas or a OSG window directly.If you are interested feel free to contact him so he can share with you some code and pointers/ideas that he's been collecting along the way. I guess that'd be better than starting from zero.<ref>{{cite web  |url    =  https://forum.flightgear.org/viewtopic.php?p=320025#p320025  |title  =  <nowiki> Re: getting started with RTT </nowiki>  |author =  <nowiki> Icecode GL </nowiki>  |date  =  Oct 5th, 2017  |added  =  Oct 5th, 2017  |script_version = 0.36  }}</ref>
The Canvas system is currently constrained to being 2D specific, i.e. to be useful for any of the "3d scene" stuff you're interested in, you'd need to read up on adding new/custom elements - e.g. those useful for 3d stuff, and effects/shaders specifically.As Icecode GL mentioned, this is something that he's been tinkering with lately. Like he also mentioned, based on Zan's groundwork (newcameras), we are looking at exposing a similar degree of flexibility via the Canvas system by introducing new elements and/or additional "modes".For starters, this will probably only involve a new view-manager based Canvas view to render a slave scenery view to a Canvas texture. The next incarnation may include effects/shader support to customie such a slave view.Besides, it would also be possible to render a totally independent scene/osg::Node - which is something that we once prototyped to load 3D models from disk and rotate/transform those using an osg::PositionAttitudeTransformMatrix
[[Howto:Extending Canvas to support rendering 3D models]]
[[File:Canvas-model-element-rotated.png|right|250px]] <ref>{{cite web  |url    =  https://forum.flightgear.org/viewtopic.php?p=320105#p320105  |title  =  <nowiki> Re: getting started with RTT </nowiki>  |author =  <nowiki> Hooray </nowiki>  |date  =  Oct 7th, 2017  |added  =  Oct 7th, 2017  |script_version = 0.36  }}</ref>


== Implementation details / for the reviewer ==
== Implementation details / for the reviewer ==
Line 82: Line 153:
{{Note|This section provides a rough overview for the potentiaal reviewer, which is also intended to be used as the commit message when/if this should get committed}}
{{Note|This section provides a rough overview for the potentiaal reviewer, which is also intended to be used as the commit message when/if this should get committed}}


This set of patches (touching SimGear and fgdata) implements a new <code>Canvas::Element</code> by creating a sub-class named <code>Canvas::View</code>. The meat of it is in the constructor, i.e. <code>Canvas::View::View()</code>, where an off-screen camera (RTT/FBO) is set up, the FGCanvasSystemAdapter file has been extended to provide access to the FlightGear view manager to compute/obtain the view-specific view matrix, which is then used by the view element to update the offscreen camera in Canvas::View::update() accordingly.  
This set of patches (touching SimGear and fgdata) implements a new <code>Canvas::Element</code> by creating a sub-class named <code>Canvas::View</code>. The meat of it is in the constructor, i.e. <code>Canvas::View::View()</code>, where an off-screen camera (RTT/FBO) is set up, the FGCanvasSystemAdapter file has been extended to provide access to the FlightGear view manager to compute/obtain the view-specific view matrix, which is then used by this new canvas view element to update the offscreen camera in Canvas::View::update() accordingly.  


This is also a good way to stress-test the renderer, as new cameras can be easily added to the scene at runtime, so that the impact of doing so can be easily measured.
BTW: This is also a good way to stress-test the renderer, as new cameras can be easily added to the scene at runtime, so that the impact of doing so can be easily measured.


To actually test the new element using the [[Nasal Console]] and paste the following snippet of code into it:
the patch is experimental, it will basically look up a view and dynamically add a slave camera to the renderer that renders the whole thing to a Canvas, a Canvas is a fancy word for a RTT/FBO context in FlightGear that can be updated by using a property-based API built on top of the property tree in the form of events/signals that are represented via listeners.Which is to say each Canvas has a handful of well-defined property names (and types) that it is watching to handle "events" - think stuff like changing the sie/view port etc. And then there is a single top-level root group, which serves as the top-level element to keep other Canvas elements.A Canvas element is nothing more than a rendering primitive that the Canvas system can handle - e.g. stuff like a raster image can be added to a Canvas group, a text string/font, and 2D drawing primitives in the form of OpenVG instrutions mapped to ShivaVG. And that's basically about it (with a few exceptions that handle use-case specific stuff like 2D mapping/charts).Apart from that, the main thing to keep in mind is that a Canvas is really just a FBO - i.e. an invisible RTT context - to become actually visible, you need to add a so called "placement" - this tells the rendering engine to look up a certain canvas and add it to the scene/cockpit or the GUI (dialogs/windows).So far, all of this is handled using native code that watches the global /canvas tree in the property tree - there is a canvas manager that handles events and passes them onto the corresponding canvas instance and its child elements.Realistically, all Canvas textures are however instantiated/updated using scripting space hooks that end up writing to the corresponding properties in the global property tree, this makes it much easier to manipulate a canvas/element, because you don't need to do any low-level getprop/setprop stuff, but can directly use an element specific API.<ref>{{cite web  |url    =  https://forum.flightgear.org/viewtopic.php?p=320105#p320105  |title  =  <nowiki> Re: getting started with RTT </nowiki>  |author =  <nowiki> Hooray </nowiki>  |date  =  Oct 7th, 2017  |added  =  Oct 7th, 2017  |script_version = 0.36  }}</ref>
 
 
To actually test the new element using the [[Nasal Console]], paste the following snippet of code into it:


<syntaxhighlight lang="nasal">
<syntaxhighlight lang="nasal">
Line 138: Line 212:


== Troubleshooting ==
== Troubleshooting ==
* update the texture specific osg::StateSet to force an update of the texture
* compare the scenegraph generated for sc::View vs. sc::Image using http://wiki.flightgear.org/Canvas_Troubleshooting#Dumping_Canvas_scene_graphs_to_disk
* compare the scenegraph generated for sc::View vs. sc::Image using http://wiki.flightgear.org/Canvas_Troubleshooting#Dumping_Canvas_scene_graphs_to_disk
* check whether sub-classing sc::Image and  directly using its _texture makes any difference or not
* check whether sub-classing sc::Image and  directly using its _texture makes any difference or not
** if it does, we could just as well add support for a custom <code>view://by-number/</code> protocol to the src/filename handling helper
* if all else fails, use an osg::Image and assign it to the texture and invoke its dirty() method to update the whole thing
* if all else fails, use an osg::Image and assign it to the texture and invoke its dirty() method to update the whole thing


== Ideas ==
== Ideas ==
{{See also|Canvas_Troubleshooting#Adding_draw_masks_for_Canvas}}
* do we need support for looking up view numbers by name ?
* support draw-masks for scene graph features (terrain, skydome, models etc): " it'd be benefitial in the long run to let the user choose what part of the scene graph they want to show. This would be useful for deferred rendering schemes, where sometimes you don't need to display the whole scenery to save some precious frames. Canvas makes that easy with all of its property handling"
* support draw-masks for scene graph features (terrain, skydome, models etc): " it'd be benefitial in the long run to let the user choose what part of the scene graph they want to show. This would be useful for deferred rendering schemes, where sometimes you don't need to display the whole scenery to save some precious frames. Canvas makes that easy with all of its property handling"
* work out what is needed for [https://forum.flightgear.org/viewtopic.php?f=71&t=27985&p=264718&hilit=synthetic+terrain#p264718v synthetic terrain views] [http://www.aircraftspruce.eu/catalog/graphics/10-03932s.jpg]
* work out what is needed for [https://forum.flightgear.org/viewtopic.php?f=71&t=27985&p=264718&hilit=synthetic+terrain#p264718v synthetic terrain views] [http://www.aircraftspruce.eu/catalog/graphics/10-03932s.jpg]
Line 149: Line 227:


== Performance / Optimizations ==
== Performance / Optimizations ==
{{See also|Canvas_Troubleshooting#Hooking_up_Canvas_and_Osg_Stats}}
* Configurable refresh rate per view. 15 fps might be enough for a external camera, we don't need to draw at max rate.<ref>{{cite web
* Configurable refresh rate per view. 15 fps might be enough for a external camera, we don't need to draw at max rate.<ref>{{cite web
   |url    =  https://forum.flightgear.org/viewtopic.php?p=318028#p318028  
   |url    =  https://forum.flightgear.org/viewtopic.php?p=318028#p318028  
Line 176: Line 255:
   |script_version = 0.40  
   |script_version = 0.40  
   }}</ref>
   }}</ref>
 
* check if '''CullThreadPerCameraDrawThreadPerContext''' works, see [[Howto:Activate multi core and multi GPU support]]
== C++ ==
The following is the boilerplate code neede to add a new element to the Canvas system inheriting from the Canvas::Image class, i.e. an element having its own allocated Image/texture buffer, which is what we can use to render a camera into it:
 
<syntaxhighlight lang="diff">
 
diff --git a/simgear/canvas/elements/CMakeLists.txt b/simgear/canvas/elements/CMakeLists.txt
index 2b537c0..4eb1e95 100644
--- a/simgear/canvas/elements/CMakeLists.txt
+++ b/simgear/canvas/elements/CMakeLists.txt
@@ -7,6 +7,7 @@ set(HEADERS
  CanvasMap.hxx
  CanvasPath.hxx
  CanvasText.hxx
+  CanvasView.hxx
)
set(DETAIL_HEADERS
@@ -20,6 +21,7 @@ set(SOURCES
  CanvasMap.cxx
  CanvasPath.cxx
  CanvasText.cxx
+  CanvasView.cxx
)
simgear_scene_component(canvas-elements canvas/elements "${SOURCES}" "${HEADERS}")
@@ -28,4 +30,4 @@ simgear_component(canvas-elements/detail canvas/elements/detail "" "${DETAIL_HEA
add_boost_test(canvas_element
  SOURCES canvas_element_test.cpp
  LIBRARIES ${TEST_LIBS}
-)
\ No newline at end of file
+)
diff --git a/simgear/canvas/elements/CanvasGroup.cxx b/simgear/canvas/elements/CanvasGroup.cxx
index c753c2e..5d2585e 100644
--- a/simgear/canvas/elements/CanvasGroup.cxx
+++ b/simgear/canvas/elements/CanvasGroup.cxx
@@ -23,6 +23,8 @@
#include "CanvasMap.hxx"
#include "CanvasPath.hxx"
#include "CanvasText.hxx"
+#include "CanvasView.hxx"
+
#include <simgear/canvas/CanvasEventVisitor.hxx>
#include <simgear/canvas/events/MouseEvent.hxx>
@@ -66,6 +68,7 @@ namespace canvas
    add<Map  >(_child_factories);
    add<Path >(_child_factories);
    add<Text >(_child_factories);
+    add<View >(_child_factories);
  }
  //----------------------------------------------------------------------------
diff --git a/simgear/canvas/elements/CanvasView.cxx b/simgear/canvas/elements/CanvasView.cxx
new file mode 100644
index 0000000..edae7ab
--- /dev/null
+++ b/simgear/canvas/elements/CanvasView.cxx
@@ -0,0 +1,88 @@
+// This library is free software; you can redistribute it and/or
+// modify it under the terms of the GNU Library General Public
+// License as published by the Free Software Foundation; either
+// version 2 of the License, or (at your option) any later version.
+//
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+// Library General Public License for more details.
+//
+// You should have received a copy of the GNU Library General Public
+// License along with this library; if not, write to the Free Software
+// Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301, USA
+
+#include <simgear_config.h>
+#include "CanvasView.hxx"
+
+#include <cmath>
+
+#include <boost/algorithm/string/predicate.hpp>
+
+namespace simgear
+{
+namespace canvas
+{
+
+  //----------------------------------------------------------------------------
+  const std::string View::TYPE_NAME = "view-camera";
+
+  //----------------------------------------------------------------------------
+  void View::staticInit()
+  {
+    Image::staticInit();
+
+    if( isInit<View>() )
+      return;
+
+    // Do some initialization if needed...
+  }
+
+  //----------------------------------------------------------------------------
+  View::View( const CanvasWeakPtr& canvas,
+            const SGPropertyNode_ptr& node,
+            const Style& parent_style,
+            ElementWeakPtr parent ):
+    Image(canvas, node, parent_style, parent)
+  {
+    staticInit();
+  }
+
+  //----------------------------------------------------------------------------
+  View::~View()
+  {
+
+  }
+
+  //----------------------------------------------------------------------------
+  void View::update(double dt)
+  {
+    Image::update(dt);
+  }
+
+  //----------------------------------------------------------------------------
+  void View::childAdded(SGPropertyNode* parent, SGPropertyNode* child)
+  {
+      return Image::childAdded(parent, child);
+  }
+
+  //----------------------------------------------------------------------------
+  void View::childRemoved(SGPropertyNode* parent, SGPropertyNode* child)
+  {
+      return Image::childRemoved(parent, child);
+  }
+
+  //----------------------------------------------------------------------------
+  void View::valueChanged(SGPropertyNode* child)
+  {
+    return Image::valueChanged(child);
+  }
+
+  //----------------------------------------------------------------------------
+  void View::childChanged(SGPropertyNode* child)
+  {
+      return Image::childChanged(child);
+
+  }
+} // namespace canvas
+} // namespace simgear
diff --git a/simgear/canvas/elements/CanvasView.hxx b/simgear/canvas/elements/CanvasView.hxx
new file mode 100644
index 0000000..7b2ec4e
--- /dev/null
+++ b/simgear/canvas/elements/CanvasView.hxx
@@ -0,0 +1,59 @@
+// Based on https://forum.flightgear.org/viewtopic.php?f=71&t=23929
+//
+// This library is free software; you can redistribute it and/or
+// modify it under the terms of the GNU Library General Public
+// License as published by the Free Software Foundation; either
+// version 2 of the License, or (at your option) any later version.
+//
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+// Library General Public License for more details.
+//
+// You should have received a copy of the GNU Library General Public
+// License along with this library; if not, write to the Free Software
+// Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301, USA
+
+#ifndef CANVAS_VIEW_HXX_
+#define CANVAS_VIEW_HXX_
+
+#include "CanvasImage.hxx"
+
+#include <boost/shared_ptr.hpp>
+#include <boost/unordered_map.hpp>
+#include <boost/unordered_set.hpp>
+
+namespace simgear
+{
+namespace canvas
+{
+  class View:
+    public Image
+  {
+    public:
+      static const std::string TYPE_NAME;
+      static void staticInit();
+
+      View( const CanvasWeakPtr& canvas,
+          const SGPropertyNode_ptr& node,
+          const Style& parent_style,
+          ElementWeakPtr parent = 0 );
+      virtual ~View();
+
+      virtual void update(double dt);
+
+      virtual void childAdded( SGPropertyNode * parent,
+                              SGPropertyNode * child );
+      virtual void childRemoved( SGPropertyNode * parent,
+                                SGPropertyNode * child );
+      virtual void valueChanged(SGPropertyNode * child);
+
+    //protected:
+
+      virtual void childChanged(SGPropertyNode * child);
+  };
+
+} // namespace canvas
+} // namespace simgear
+
+#endif /* CANVAS_VIEW_HXX_ */
 
 
</syntaxhighlight>


== Base Package ==
== Base Package ==

Navigation menu