Canvas view camera element: Difference between revisions

From FlightGear wiki
Jump to navigation Jump to search
Line 56: Line 56:
{{Note|This section provides a rough overview for the potentiaal reviewer, which is also intended to be used as the commit message when/if this should get committed}}
{{Note|This section provides a rough overview for the potentiaal reviewer, which is also intended to be used as the commit message when/if this should get committed}}


This set of patches (touching SimGear and fgdata) implements a new Canvas::Element by creating a sub-class named Canvas::View. The meat of it is in the constructor, i.e. Canvas::View::View(), where an offscreen camera (RTT/FBO) is set up, the FGCanvasSystemAdapter file has been extended to provide access to the FlightGear view manager to compute/obtain the view specific view matrix, which is then used to update the offscreen camera in Canvas::View::update() accordingly.  
This set of patches (touching SimGear and fgdata) implements a new <code>Canvas::Element</code> by creating a sub-class named <code>Canvas::View</code>. The meat of it is in the constructor, i.e. <code>Canvas::View::View()</code>, where an off-screen camera (RTT/FBO) is set up, the FGCanvasSystemAdapter file has been extended to provide access to the FlightGear view manager to compute/obtain the view-specific view matrix, which is then used by the view element to update the offscreen camera in Canvas::View::update() accordingly.
 
This is also a good way to stress-test the renderer, as new cameras can be easily added to the scene at runtime, so that the impact of doing so can be easily measured.


To actually test the new element using the [[Nasal Console]] and paste the following snippet of code into it:
To actually test the new element using the [[Nasal Console]] and paste the following snippet of code into it:

Revision as of 16:18, 2 September 2017

This article is a stub. You can help the wiki by expanding it.
Canvas Camera Views
Canvas-view-element-prototype-by-icecode gl.png
Started in 08/2016 (prototyped by F-JJTH)
Description adds support for rendering slave scenery views to a FBO/RTT (texture)
Maintainer(s) none
Contributor(s) F-JJTH, Icecode GL, Hooray
Status experimental/known issues


Last updated in 09/2017

Summary

Several aircraft developers have manifested their interest in being able to render to a texture (RTT) Render target This is a link to a Wikipedia article and use it inside cockpits as mirrors, external cameras (so called tail cams) and other uses. Effects and shaders developers have also reached a point where ignoring RTT is a waste of resources and a limitating factor when creating new effects. Although this Canvas element is directed mainly towards the first need, it is a first step forward in terms of finally exposing Render To Texture capabilities to non-C++ space too.

Status

We have a basic prototype, provided by F-JJTH, who developed the whole thing in mid-2016. There are a few issues, and it's using the legacy approach for implementing OD_gauge based avionics, i.e. not yet using the Canvas system.

We also have a quick, but working, integration in the form of a custom Canvas element provided by Icecode GL. Originally, the image was mirrored because of different coordinate systems. Apparently the missing piece was to use FG's view manager, which is now fixed, thanks to Clement's contribution.

Who knows, with cleaner code and some tweaks we could have a deferred renderer and RTT capabilities in Canvas space, as long as the effect framework can access these RTT contexts, as Hooray proposed multiple times.[1]

Also, Stuart offered to help review any patches and get them committed. Please ping Stuart when you think the code is worth checking in. It doesn't have to be perfect - Particularly once the current release is out (2017.3). [2]

Gallery

Screenshot showing early experiments rendering a model to a Canvas view
An experimental Canvas element to render slaved scenery views to a custom Canvas texture, to be used for creating custom tail-cams/mirror textures and so on
Several Canvas dialogs with view manager views
Canvas gui dialog with a custom canvas element to display view manager views (based on code prototyped by F-JJTH)

Implementation details / for the reviewer

Note  This section provides a rough overview for the potentiaal reviewer, which is also intended to be used as the commit message when/if this should get committed

This set of patches (touching SimGear and fgdata) implements a new Canvas::Element by creating a sub-class named Canvas::View. The meat of it is in the constructor, i.e. Canvas::View::View(), where an off-screen camera (RTT/FBO) is set up, the FGCanvasSystemAdapter file has been extended to provide access to the FlightGear view manager to compute/obtain the view-specific view matrix, which is then used by the view element to update the offscreen camera in Canvas::View::update() accordingly.

This is also a good way to stress-test the renderer, as new cameras can be easily added to the scene at runtime, so that the impact of doing so can be easily measured.

To actually test the new element using the Nasal Console and paste the following snippet of code into it:

var (width,height) = (512,512);
var ELEMENT_NAME = "viewcamera";

var display_view = func(view=0) 
var title = 'Canvas test:' ~ ELEMENT_NAME;
var window = canvas.Window.new([width,height],"dialog").set('title',title);
var myCanvas = window.createCanvas().set("background", canvas.style.getColor("bg_color"));
var root = myCanvas.createGroup();

var child = root.createChild( ELEMENT_NAME );
child.set("view-number", view);
} # display_view()

var totalViews = props.getNode("sim").getChildren("view");
print(size(totalViews));
display_view(view: 3);

Aircraft developers can use the new Canvas view element simply by using the conventional approach to replace a static texture with a Canvas using a cockpit placement.

Roadmap

Note  There is a related patch available at https://forum.flightgear.org/viewtopic.php?f=71&t=23929#p317448


  • extend FGCanvasSystemAdapter to make the view manager available there [3]
  • come up with a new Canvas element inheriting from Canvas::Image 80}% completed
  • add Clement's camera setup routines
  • make the camera render into the texture used by the sub-class
  • this should be named viewmgr-camera or something like that to make it obvious what it is doing

Testing

Once we have a basic prototype working, the new camera element needs to be tested, specifically:

  • using different resolutions
  • displaying multiple/independent camera views
  • all supported Canvas placements
  • shaders and effects
  • ALS
  • Rembrandt
  • Reset/re-init
  • OSG threading modes

Known Issues

  • texture/view matrix updates don't currently take effect unless .update() is specifically invoked
  • skydome handling (probably missing effect/shader or wrong root node) Done Done (by Icecode_GL)
  • change DATA_VARIANCE for texture setup, analogous to the Canvas::Image::Image ctor
  • make the size of the texture re-configurable using properties

Ideas

  • support draw-masks for scene graph features (terrain, skydome, models etc): " it'd be benefitial in the long run to let the user choose what part of the scene graph they want to show. This would be useful for deferred rendering schemes, where sometimes you don't need to display the whole scenery to save some precious frames. Canvas makes that easy with all of its property handling"
  • work out what is needed for synthetic terrain views [1]
  • support effects/shaders: This is fairly straightforward to do, just create a EffectsGeode, fill it with a "compiled" .eff, and substitute the standard osg::Geode for it. But I don't think that's going to give us the flexibility we need. A shader won't be able to access several canvases at the same time if we do things that way.

Performance / Optimizations

  • provide a view-cache using a STL std::map<> ?
  • use PBOs [2]
  • consider hooking up the whole thing to the FGStatsManager (OSG stats) ?

C++

The following is the boilerplate code neede to add a new element to the Canvas system inheriting from the Canvas::Image class, i.e. an element having its own allocated Image/texture buffer, which is what we can use to render a camera into it:

diff --git a/simgear/canvas/elements/CMakeLists.txt b/simgear/canvas/elements/CMakeLists.txt
index 2b537c0..4eb1e95 100644
--- a/simgear/canvas/elements/CMakeLists.txt
+++ b/simgear/canvas/elements/CMakeLists.txt
@@ -7,6 +7,7 @@ set(HEADERS
   CanvasMap.hxx
   CanvasPath.hxx
   CanvasText.hxx
+  CanvasView.hxx
 )
 
 set(DETAIL_HEADERS
@@ -20,6 +21,7 @@ set(SOURCES
   CanvasMap.cxx
   CanvasPath.cxx
   CanvasText.cxx
+  CanvasView.cxx
 )
 
 simgear_scene_component(canvas-elements canvas/elements "${SOURCES}" "${HEADERS}")
@@ -28,4 +30,4 @@ simgear_component(canvas-elements/detail canvas/elements/detail "" "${DETAIL_HEA
 add_boost_test(canvas_element
   SOURCES canvas_element_test.cpp
   LIBRARIES ${TEST_LIBS}
-)
\ No newline at end of file
+)
diff --git a/simgear/canvas/elements/CanvasGroup.cxx b/simgear/canvas/elements/CanvasGroup.cxx
index c753c2e..5d2585e 100644
--- a/simgear/canvas/elements/CanvasGroup.cxx
+++ b/simgear/canvas/elements/CanvasGroup.cxx
@@ -23,6 +23,8 @@
 #include "CanvasMap.hxx"
 #include "CanvasPath.hxx"
 #include "CanvasText.hxx"
+#include "CanvasView.hxx"
+
 #include <simgear/canvas/CanvasEventVisitor.hxx>
 #include <simgear/canvas/events/MouseEvent.hxx>
 
@@ -66,6 +68,7 @@ namespace canvas
     add<Map  >(_child_factories);
     add<Path >(_child_factories);
     add<Text >(_child_factories);
+    add<View >(_child_factories);
   }
 
   //----------------------------------------------------------------------------
diff --git a/simgear/canvas/elements/CanvasView.cxx b/simgear/canvas/elements/CanvasView.cxx
new file mode 100644
index 0000000..edae7ab
--- /dev/null
+++ b/simgear/canvas/elements/CanvasView.cxx
@@ -0,0 +1,88 @@
+// This library is free software; you can redistribute it and/or
+// modify it under the terms of the GNU Library General Public
+// License as published by the Free Software Foundation; either
+// version 2 of the License, or (at your option) any later version.
+//
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+// Library General Public License for more details.
+//
+// You should have received a copy of the GNU Library General Public
+// License along with this library; if not, write to the Free Software
+// Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301, USA
+
+#include <simgear_config.h>
+#include "CanvasView.hxx"
+
+#include <cmath>
+
+#include <boost/algorithm/string/predicate.hpp>
+
+namespace simgear
+{
+namespace canvas
+{
+
+  //----------------------------------------------------------------------------
+  const std::string View::TYPE_NAME = "view-camera";
+
+  //----------------------------------------------------------------------------
+  void View::staticInit()
+  {
+    Image::staticInit();
+
+    if( isInit<View>() )
+      return;
+
+    // Do some initialization if needed...
+  }
+
+  //----------------------------------------------------------------------------
+  View::View( const CanvasWeakPtr& canvas,
+            const SGPropertyNode_ptr& node,
+            const Style& parent_style,
+            ElementWeakPtr parent ):
+    Image(canvas, node, parent_style, parent)
+  {
+    staticInit();
+  }
+
+  //----------------------------------------------------------------------------
+  View::~View()
+  {
+
+  }
+
+  //----------------------------------------------------------------------------
+  void View::update(double dt)
+  {
+    Image::update(dt);
+  }
+
+  //----------------------------------------------------------------------------
+  void View::childAdded(SGPropertyNode* parent, SGPropertyNode* child)
+  {
+      return Image::childAdded(parent, child);
+  }
+
+  //----------------------------------------------------------------------------
+  void View::childRemoved(SGPropertyNode* parent, SGPropertyNode* child)
+  {
+      return Image::childRemoved(parent, child);
+  }
+
+  //----------------------------------------------------------------------------
+  void View::valueChanged(SGPropertyNode* child)
+  {
+    return Image::valueChanged(child);
+  }
+
+  //----------------------------------------------------------------------------
+  void View::childChanged(SGPropertyNode* child)
+  {
+      return Image::childChanged(child);
+
+  }
+} // namespace canvas
+} // namespace simgear
diff --git a/simgear/canvas/elements/CanvasView.hxx b/simgear/canvas/elements/CanvasView.hxx
new file mode 100644
index 0000000..7b2ec4e
--- /dev/null
+++ b/simgear/canvas/elements/CanvasView.hxx
@@ -0,0 +1,59 @@
+// Based on https://forum.flightgear.org/viewtopic.php?f=71&t=23929
+//
+// This library is free software; you can redistribute it and/or
+// modify it under the terms of the GNU Library General Public
+// License as published by the Free Software Foundation; either
+// version 2 of the License, or (at your option) any later version.
+//
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+// Library General Public License for more details.
+//
+// You should have received a copy of the GNU Library General Public
+// License along with this library; if not, write to the Free Software
+// Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301, USA
+
+#ifndef CANVAS_VIEW_HXX_
+#define CANVAS_VIEW_HXX_
+
+#include "CanvasImage.hxx"
+
+#include <boost/shared_ptr.hpp>
+#include <boost/unordered_map.hpp>
+#include <boost/unordered_set.hpp>
+
+namespace simgear
+{
+namespace canvas
+{
+  class View:
+    public Image
+  {
+    public:
+      static const std::string TYPE_NAME;
+      static void staticInit();
+
+      View( const CanvasWeakPtr& canvas,
+           const SGPropertyNode_ptr& node,
+           const Style& parent_style,
+           ElementWeakPtr parent = 0 );
+      virtual ~View();
+
+      virtual void update(double dt);
+
+      virtual void childAdded( SGPropertyNode * parent,
+                               SGPropertyNode * child );
+      virtual void childRemoved( SGPropertyNode * parent,
+                                 SGPropertyNode * child );
+      virtual void valueChanged(SGPropertyNode * child);
+
+    //protected:
+
+      virtual void childChanged(SGPropertyNode * child);
+  };
+
+} // namespace canvas
+} // namespace simgear
+
+#endif /* CANVAS_VIEW_HXX_ */

Base Package

Nasal Console

After patching and rebuilding SimGear/FlightGear respectively, and applying the changes to to api.nas in the base package, the following can be pasted into the Nasal Console for testing purposes:

# TODO: this must match the TYPE_NAME used by the C++ code
var ELEMENT_NAME ="view-camera"; # to be adapted according to the C++ changes

var myElementTest = {
 ##
 # constructor
 new: func( camera ) {
  var m = { parents: [myElementTest] };
  m.dlg = canvas.Window.new([camera.width,camera.height],"dialog");
  m.canvas = m.dlg.createCanvas().setColorBackground(1,1,1,1);
  m.root = m.canvas.createGroup();

  ##
  # instantiate a new element
  m.myElement = m.root.createChild( ELEMENT_NAME );
  # set the view-number property
  m.myElement.set("view-number", camera.view);
  m.dlg.set("title", "view #"~camera.view); # TODO look up proper title/name

  return m;
 }, # new
 
}; # end of myElementTest

# TODO: get list of all views ?

var cameras = [
 {view: 0, width : 640, height: 480},
 {view: 1, width : 320, height: 160},
 {view: 2, width : 320, height: 160},
];

foreach(var cam; cameras) {
 var newCam = myElementTest.new( cam );
}

Related

References