Difference between revisions of "Canvas View Camera Element"

From FlightGear wiki
Jump to: navigation, search
(Ideas)
 
(66 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{Stub}}
 
 
 
{{infobox subsystem
 
{{infobox subsystem
|image      = Canvas-view-element-prototype-by-icecode gl.png
 
 
|name        = Canvas Camera Views
 
|name        = Canvas Camera Views
|started    = 08/2016 (prototyped by F-JJTH)
+
|description = Rendering arbitrary views to a texture/Canvas
|description = adds support for rendering slave scenery views to a FBO/RTT (texture)
+
|status      = planned/prototype
|status      = experimental/known issues
+
 
|maintainers = none
 
|maintainers = none
|developers  = F-JJTH, Icecode GL, Hooray
+
|developers  = none
 
}}
 
}}
  
 +
== Background ==
  
Last updated in 09/2017
+
Several aircraft developers have manifested their interest in being able to render scene views to a texture (RTT) {{Wikipedia|Render target}} and use it inside cockpits as mirrors, external cameras (so called tail cams) and other uses.
  
== Summary ==
+
With the [[Compositor]] framework it's possible to configure custom render pipelines in XML space, which together with Effect schemes allow complete control over how a rendered scene looks. A synthetic terrain can be rendered by assigning an Effect scheme to a <tt>scene</tt> type pass, which textures all terrain with a brown checkerboard pattern and paints the sky in a solid blue color. All other objects are ignored and aren't rendered, saving a lot of computation time. Custom render distances independent from the main LOD settings can also be used to further optimize the performance of the display.
  
Several aircraft developers have manifested their interest in being able to render to a texture and use it inside cockpits as mirrors, external cameras and other uses. Effects and shaders developers have also reached a point where ignoring RTT is a waste of resources and a limitating factor when creating new effects. Although this Canvas element is directed mainly towards the first need, it is a first step forward in terms of finally exposing Render To Texture capabilities to non-C++ space too.
+
== Use Cases ==
 
+
* Tail Cams (gear view)
== Status ==
+
* Mirrors
{{See also|View manager}}
+
* Dynamic in-sim view configuration
 
+
* On demand creation of views and windows (e.g. [[FGCamera]] previews)
We have a basic prototype, provided by F-JJTH, who developed the whole thing in mid-2016. There are a few issues, and it's using the legacy approach for implementing OD_gauge based avionics, i.e. not yet using the Canvas system.
+
* Missile/payload views
 
+
* Prototyping/testing HUDs or PFDs requiring synthetic terrain to work properly
We also have a quick, but working, integration into Canvas provided by Icecode GL. Originally, the image was mirrored because of different coordinate systems. Apparently the missing piece was to use FG's view manager, which is now fixed.
+
 
+
Who knows, with cleaner code and some tweaks we could have a deferred renderer and RTT capabilities in Canvas space, as long as the effect framework can access these RTT contexts, as Hooray proposed multiple times.<ref>{{cite web
+
  |url    =  https://forum.flightgear.org/viewtopic.php?p=317743#p317743
+
  |title  =  <nowiki> Re: Gear view in cockpit computer </nowiki>
+
  |author =  <nowiki> Icecode GL </nowiki>
+
  |date  =  Aug 28th, 2017
+
  |added  =  Aug 28th, 2017
+
  |script_version = 0.40
+
  }}</ref>
+
 
+
Also, Stuart offered to help review any patches and get them committed.  Please ping Stuart when you think the code is worth checking in. It doesn't have to be perfect - Particularly once the current release is out (2017.3). <ref>{{cite web
+
  |url    =  https://forum.flightgear.org/viewtopic.php?p=317967#p317967
+
  |title  =  <nowiki> Re: Gear view in cockpit computer </nowiki>
+
  |author =  <nowiki> stuart </nowiki>
+
  |date  =  Sep 1st, 2017
+
  |added  =  Sep 1st, 2017
+
  |script_version = 0.40
+
  }}</ref>
+
 
+
== Gallery ==
+
 
+
{|style="margin: 0 auto;"
+
| [[File:Icecode GL-Canvas-Camera-support.png|thumb|upright|An experimental [[Canvas]] element to render slaved scenery views to a custom Canvas texture, to be used for creating custom tail-cams/mirror textures and so on]]
+
| [[File:Icecodes slave cams 09-2017.png|thumb|upright|Several Canvas dialogs with view manager views]]
+
| [[File:Canvas-view-element-prototype-by-icecode gl.png|thumb|upright|[[Canvas]] gui dialog with a custom canvas element to display view manager views (based on code prototyped by F-JJTH)]]
+
|}
+
  
 
== Roadmap ==
 
== Roadmap ==
{{PatchAvailable|url=https://forum.flightgear.org/viewtopic.php?f=71&t=23929#p317448}}
 
  
 +
=== Use CompositeViewer {{Pending}} ===
 +
{{See also|CompositeViewer Support}}
  
* extend FGCanvasSystemAdapter to make the view manager available there <ref>https://forum.flightgear.org/viewtopic.php?f=71&t=23929&p=317798#p317798</ref>
+
Currently FlightGear uses only one instance of <tt>osg::Viewer</tt>, which is used by CameraGroup to manage the slave cameras. Supporting CompositeViewer would require modifying {{flightgear file|src/Viewer/fg_os_osgviewer.cxx}} and creating some kind of wrapper class that manages the CompositeViewer instance and assigns a CameraGroup to each <tt>osg::View</tt>. It's also important to note that currently all FG subsystems assume there is a single instance of [[Howto:CameraGroup talks|CameraGroup]].
* come up with a new Canvas element inheriting from Canvas::Image {{Progressbar|80}}
+
* add Clement's camera setup routines
+
* make the camera render into the texture used by the sub-class
+
* this should be named viewmgr-camera or something like that to make it obvious what it is doing
+
* look up why the skydome isn't added, probably effects/shader stuff - see the replace/visitor in Clement's patch which is overriding some effects
+
  
== Testing ==
+
=== Standardize the [[View manager]] {{Pending}} ===
Once we have a basic prototype working, the new camera element needs to be tested, specifically:
+
* using different resolutions
+
* displaying multiple/independent camera views
+
* all supported Canvas placements
+
* shaders and effects
+
* ALS
+
* Rembrandt
+
* Reset/re-init
+
* OSG threading modes
+
  
== Known Issues ==
+
The first step would be to port the view code to SimGear so it can be used and known by the Compositor and Canvas. The view manager ({{flightgear file|src/Viewer/viewmgr.cxx}}) currently has some hardcoded assumptions, so it would either need to be rewritten to remove them or a new interface for the Views could be created specifically for the Canvas Camera View.
* texture/view matric updates don't currently take effect unless .update() is specifically invoked
+
* <del>skydome handling (probably missing effect/shader or wrong root node)</del> {{Done}} (by Icecode_GL)
+
* change DATA_VARIANCE for texture setup, analogous to the Canvas::Image::Image ctor
+
* make the size of the texture re-configurable using properties
+
  
== Ideas ==
+
In FlightGear, the Canvas system (which resides in SimGear) is integrated using the equivalent of a FGCanvasSystemAdapter, which provides all FG APIs to SimGear and makes the Canvas system available inside FG: {{flightgear file|src/Canvas/FGCanvasSystemAdapter.cxx}}
* support draw-masks for scene graph features (terrain, skydome, models etc): " it'd be benefitial in the long run to let the user choose what part of the scene graph they want to show. This would be useful for deferred rendering schemes, where sometimes you don't need to display the whole scenery to save some precious frames. Canvas makes that easy with all of its property handling"
+
* work out what is needed for [https://forum.flightgear.org/viewtopic.php?f=71&t=27985&p=264718&hilit=synthetic+terrain#p264718v synthetic terrain views] [http://www.aircraftspruce.eu/catalog/graphics/10-03932s.jpg]
+
* support effects/shaders: This is fairly straightforward to do, just create a EffectsGeode, fill it with a "compiled" .eff, and substitute the standard osg::Geode for it. But I don't think that's going to give us the flexibility we need. A shader won't be able to access several canvases at the same time if we do things that way.
+
  
== Performance / Optimizations ==
+
Instead of using tied properties, the adapted view code would either used propertyObject<> or the propertyBasedMgr abstraction, so that the corresponding canvas element can continue to use well-known conventions to manipulate views: ({{flightgear file|src/Viewer/view.cxx}})
  
* use PBOs [http://forum.openscenegraph.org/viewtopic.php?t=10713&view=next]
+
=== Create a Canvas Element subclass {{Pending}} ===
* consider hooking up the whole thing to the FGStatsManager (OSG stats) ?
+
{{Main article|Canvas_Development#Adding_a_new_Element}}
  
== C++ ==
+
This derived class would require the following configuration:
The following is the boilerplate code neede to add a new element to the Canvas system inheriting from the Canvas::Image class, i.e. an element having its own allocated Image/texture buffer, which is what we can use to render a camera into it:
+
* [[Compositor]] to use.
 +
* [[View manager|View]] to use. This could be one of the "main" ones (i.e. the ones on the main property tree), or a locally-defined one that is only known to this Canvas Element (think FLIR).
 +
* Scene graph to use. By default the main scene graph would be used, but an arbitrary XML file can be loaded to [[Howto:Extending Canvas to support rendering 3D models|render a custom model]]. A typical use-case would be instruments that need to [[Shuttle ADI ball|manipulate/render a 3D object]] (for which we have working code, too)
 +
* Several optimization/miscellaneous parameters like framerate cap, etc.
  
<syntaxhighlight lang="diff">
+
Simplifying a lot, this Canvas Element would be an aggregation of a Compositor instance, a View and a pointer to a <tt>osg::Group</tt> representing the scene graph to render. This setup could be replicated in {{flightgear file|src/Viewer/fg_os_osgviewer.cxx}}, with the difference of ignoring Canvas and using native windowing features from OSG.
  
diff --git a/simgear/canvas/elements/CMakeLists.txt b/simgear/canvas/elements/CMakeLists.txt
+
== Related ==
index 2b537c0..4eb1e95 100644
+
--- a/simgear/canvas/elements/CMakeLists.txt
+
+++ b/simgear/canvas/elements/CMakeLists.txt
+
@@ -7,6 +7,7 @@ set(HEADERS
+
  CanvasMap.hxx
+
  CanvasPath.hxx
+
  CanvasText.hxx
+
+  CanvasView.hxx
+
)
+
+
set(DETAIL_HEADERS
+
@@ -20,6 +21,7 @@ set(SOURCES
+
  CanvasMap.cxx
+
  CanvasPath.cxx
+
  CanvasText.cxx
+
+  CanvasView.cxx
+
)
+
+
simgear_scene_component(canvas-elements canvas/elements "${SOURCES}" "${HEADERS}")
+
@@ -28,4 +30,4 @@ simgear_component(canvas-elements/detail canvas/elements/detail "" "${DETAIL_HEA
+
add_boost_test(canvas_element
+
  SOURCES canvas_element_test.cpp
+
  LIBRARIES ${TEST_LIBS}
+
-)
+
\ No newline at end of file
+
+)
+
diff --git a/simgear/canvas/elements/CanvasGroup.cxx b/simgear/canvas/elements/CanvasGroup.cxx
+
index c753c2e..5d2585e 100644
+
--- a/simgear/canvas/elements/CanvasGroup.cxx
+
+++ b/simgear/canvas/elements/CanvasGroup.cxx
+
@@ -23,6 +23,8 @@
+
#include "CanvasMap.hxx"
+
#include "CanvasPath.hxx"
+
#include "CanvasText.hxx"
+
+#include "CanvasView.hxx"
+
+
+
#include <simgear/canvas/CanvasEventVisitor.hxx>
+
#include <simgear/canvas/events/MouseEvent.hxx>
+
+
@@ -66,6 +68,7 @@ namespace canvas
+
    add<Map  >(_child_factories);
+
    add<Path >(_child_factories);
+
    add<Text >(_child_factories);
+
+    add<View >(_child_factories);
+
  }
+
+
  //----------------------------------------------------------------------------
+
diff --git a/simgear/canvas/elements/CanvasView.cxx b/simgear/canvas/elements/CanvasView.cxx
+
new file mode 100644
+
index 0000000..edae7ab
+
--- /dev/null
+
+++ b/simgear/canvas/elements/CanvasView.cxx
+
@@ -0,0 +1,88 @@
+
+// This library is free software; you can redistribute it and/or
+
+// modify it under the terms of the GNU Library General Public
+
+// License as published by the Free Software Foundation; either
+
+// version 2 of the License, or (at your option) any later version.
+
+//
+
+// This library is distributed in the hope that it will be useful,
+
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+
+// Library General Public License for more details.
+
+//
+
+// You should have received a copy of the GNU Library General Public
+
+// License along with this library; if not, write to the Free Software
+
+// Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301, USA
+
+
+
+#include <simgear_config.h>
+
+#include "CanvasView.hxx"
+
+
+
+#include <cmath>
+
+
+
+#include <boost/algorithm/string/predicate.hpp>
+
+
+
+namespace simgear
+
+{
+
+namespace canvas
+
+{
+
+
+
+  //----------------------------------------------------------------------------
+
+  const std::string View::TYPE_NAME = "view-camera";
+
+
+
+  //----------------------------------------------------------------------------
+
+  void View::staticInit()
+
+  {
+
+    Image::staticInit();
+
+
+
+    if( isInit<View>() )
+
+      return;
+
+
+
+    // Do some initialization if needed...
+
+  }
+
+
+
+  //----------------------------------------------------------------------------
+
+  View::View( const CanvasWeakPtr& canvas,
+
+            const SGPropertyNode_ptr& node,
+
+            const Style& parent_style,
+
+            ElementWeakPtr parent ):
+
+    Image(canvas, node, parent_style, parent)
+
+  {
+
+    staticInit();
+
+  }
+
+
+
+  //----------------------------------------------------------------------------
+
+  View::~View()
+
+  {
+
+
+
+  }
+
+
+
+  //----------------------------------------------------------------------------
+
+  void View::update(double dt)
+
+  {
+
+    Image::update(dt);
+
+  }
+
+
+
+  //----------------------------------------------------------------------------
+
+  void View::childAdded(SGPropertyNode* parent, SGPropertyNode* child)
+
+  {
+
+      return Image::childAdded(parent, child);
+
+  }
+
+
+
+  //----------------------------------------------------------------------------
+
+  void View::childRemoved(SGPropertyNode* parent, SGPropertyNode* child)
+
+  {
+
+      return Image::childRemoved(parent, child);
+
+  }
+
+
+
+  //----------------------------------------------------------------------------
+
+  void View::valueChanged(SGPropertyNode* child)
+
+  {
+
+    return Image::valueChanged(child);
+
+  }
+
+
+
+  //----------------------------------------------------------------------------
+
+  void View::childChanged(SGPropertyNode* child)
+
+  {
+
+      return Image::childChanged(child);
+
+
+
+  }
+
+} // namespace canvas
+
+} // namespace simgear
+
diff --git a/simgear/canvas/elements/CanvasView.hxx b/simgear/canvas/elements/CanvasView.hxx
+
new file mode 100644
+
index 0000000..7b2ec4e
+
--- /dev/null
+
+++ b/simgear/canvas/elements/CanvasView.hxx
+
@@ -0,0 +1,59 @@
+
+// Based on https://forum.flightgear.org/viewtopic.php?f=71&t=23929
+
+//
+
+// This library is free software; you can redistribute it and/or
+
+// modify it under the terms of the GNU Library General Public
+
+// License as published by the Free Software Foundation; either
+
+// version 2 of the License, or (at your option) any later version.
+
+//
+
+// This library is distributed in the hope that it will be useful,
+
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+
+// Library General Public License for more details.
+
+//
+
+// You should have received a copy of the GNU Library General Public
+
+// License along with this library; if not, write to the Free Software
+
+// Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301, USA
+
+
+
+#ifndef CANVAS_VIEW_HXX_
+
+#define CANVAS_VIEW_HXX_
+
+
+
+#include "CanvasImage.hxx"
+
+
+
+#include <boost/shared_ptr.hpp>
+
+#include <boost/unordered_map.hpp>
+
+#include <boost/unordered_set.hpp>
+
+
+
+namespace simgear
+
+{
+
+namespace canvas
+
+{
+
+  class View:
+
+    public Image
+
+  {
+
+    public:
+
+      static const std::string TYPE_NAME;
+
+      static void staticInit();
+
+
+
+      View( const CanvasWeakPtr& canvas,
+
+          const SGPropertyNode_ptr& node,
+
+          const Style& parent_style,
+
+          ElementWeakPtr parent = 0 );
+
+      virtual ~View();
+
+
+
+      virtual void update(double dt);
+
+
+
+      virtual void childAdded( SGPropertyNode * parent,
+
+                              SGPropertyNode * child );
+
+      virtual void childRemoved( SGPropertyNode * parent,
+
+                                SGPropertyNode * child );
+
+      virtual void valueChanged(SGPropertyNode * child);
+
+
+
+    //protected:
+
+
+
+      virtual void childChanged(SGPropertyNode * child);
+
+  };
+
+
+
+} // namespace canvas
+
+} // namespace simgear
+
+
+
+#endif /* CANVAS_VIEW_HXX_ */
+
  
 
+
=== Forum topics ===
</syntaxhighlight>
+
* {{forum link|t=32846|text=Canvas::View Development}}
 
+
* {{forum link|t=23929|text=Gear View in Cockpit}}
== Base Package ==
+
* {{forum link|t=20057|text=Rear-View mirror}}
{{collapsible script
+
* {{forum link|t=18905|text=Progress on synthetic terrain}}
| type  = view-camera
+
* {{forum link|t=17184|text=Instruments with heightmaps}}
| title  = view-camera  $FG_ROOT/Nasal/canvas/api.nas changes
+
| intro = Finally, we need to modify api.nas to provide Nasal wrappers
+
| script =  
+
diff --git a/Nasal/canvas/api.nas b/Nasal/canvas/api.nas
+
index 6d39d03..f98fdd9 100644
+
--- a/Nasal/canvas/api.nas
+
+++ b/Nasal/canvas/api.nas
+
@@ -1099,13 +1099,21 @@ var Image = {
+
  }
+
};
+
+
+var viewCamera = {
+
+  new: func(ghost)
+
{
+
+    return {parents: [viewCamera, Element.new(ghost)]};
+
+  },
+
+};
+
+
+
# Element factories used by #Group elements to create children
+
Group._element_factories = {
+
  "group": Group.new,
+
  "map": Map.new,
+
  "text": Text.new,
+
  "path": Path.new,
+
- "image": Image.new
+
+  "image": Image.new,
+
+  "view-camera": viewCamera.new,
+
};
+
| lang = diff
+
}}
+
 
+
== Nasal Console ==
+
 
+
After patching and rebuilding SimGear/FlightGear respectively, and applying the changes to to api.nas in the base package, the following can be pasted into the [[Nasal Console]] for testing purposes:
+
 
+
<syntaxhighlight lang="nasal">
+
# TODO: this must match the TYPE_NAME used by the C++ code
+
var ELEMENT_NAME ="view-camera"; # to be adapted according to the C++ changes
+
 
+
var myElementTest = {
+
##
+
# constructor
+
new: func( camera ) {
+
  var m = { parents: [myElementTest] };
+
  m.dlg = canvas.Window.new([camera.width,camera.height],"dialog");
+
  m.canvas = m.dlg.createCanvas().setColorBackground(1,1,1,1);
+
  m.root = m.canvas.createGroup();
+
 
+
  ##
+
  # instantiate a new element
+
  m.myElement = m.root.createChild( ELEMENT_NAME );
+
  # set the view-number property
+
  m.myElement.set("view-number", camera.view);
+
  m.dlg.set("title", "view #"~camera.view); # TODO look up proper title/name
+
 
+
  return m;
+
}, # new
+
+
}; # end of myElementTest
+
 
+
# TODO: get list of all views ?
+
 
+
var cameras = [
+
{view: 0, width : 640, height: 480},
+
{view: 1, width : 320, height: 160},
+
{view: 2, width : 320, height: 160},
+
];
+
 
+
foreach(var cam; cameras) {
+
var newCam = myElementTest.new( cam );
+
}
+
</syntaxhighlight>
+
 
+
== Related ==
+
{{Appendix}}
+

Latest revision as of 12:04, 10 April 2020

Canvas Camera Views
Description Rendering arbitrary views to a texture/Canvas
Maintainer(s) none
Contributor(s) none
Status planned/prototype

Background

Several aircraft developers have manifested their interest in being able to render scene views to a texture (RTT) Render target This is a link to a Wikipedia article and use it inside cockpits as mirrors, external cameras (so called tail cams) and other uses.

With the Compositor framework it's possible to configure custom render pipelines in XML space, which together with Effect schemes allow complete control over how a rendered scene looks. A synthetic terrain can be rendered by assigning an Effect scheme to a scene type pass, which textures all terrain with a brown checkerboard pattern and paints the sky in a solid blue color. All other objects are ignored and aren't rendered, saving a lot of computation time. Custom render distances independent from the main LOD settings can also be used to further optimize the performance of the display.

Use Cases

  • Tail Cams (gear view)
  • Mirrors
  • Dynamic in-sim view configuration
  • On demand creation of views and windows (e.g. FGCamera previews)
  • Missile/payload views
  • Prototyping/testing HUDs or PFDs requiring synthetic terrain to work properly

Roadmap

Use CompositeViewer Pending Pending

Currently FlightGear uses only one instance of osg::Viewer, which is used by CameraGroup to manage the slave cameras. Supporting CompositeViewer would require modifying flightgear/src/Viewer/fg_os_osgviewer.cxx and creating some kind of wrapper class that manages the CompositeViewer instance and assigns a CameraGroup to each osg::View. It's also important to note that currently all FG subsystems assume there is a single instance of CameraGroup.

Standardize the View manager Pending Pending

The first step would be to port the view code to SimGear so it can be used and known by the Compositor and Canvas. The view manager (flightgear/src/Viewer/viewmgr.cxx) currently has some hardcoded assumptions, so it would either need to be rewritten to remove them or a new interface for the Views could be created specifically for the Canvas Camera View.

In FlightGear, the Canvas system (which resides in SimGear) is integrated using the equivalent of a FGCanvasSystemAdapter, which provides all FG APIs to SimGear and makes the Canvas system available inside FG: flightgear/src/Canvas/FGCanvasSystemAdapter.cxx

Instead of using tied properties, the adapted view code would either used propertyObject<> or the propertyBasedMgr abstraction, so that the corresponding canvas element can continue to use well-known conventions to manipulate views: (flightgear/src/Viewer/view.cxx)

Create a Canvas Element subclass Pending Pending

1rightarrow.png See Canvas_Development#Adding_a_new_Element for the main article about this subject.

This derived class would require the following configuration:

  • Compositor to use.
  • View to use. This could be one of the "main" ones (i.e. the ones on the main property tree), or a locally-defined one that is only known to this Canvas Element (think FLIR).
  • Scene graph to use. By default the main scene graph would be used, but an arbitrary XML file can be loaded to render a custom model. A typical use-case would be instruments that need to manipulate/render a 3D object (for which we have working code, too)
  • Several optimization/miscellaneous parameters like framerate cap, etc.

Simplifying a lot, this Canvas Element would be an aggregation of a Compositor instance, a View and a pointer to a osg::Group representing the scene graph to render. This setup could be replicated in flightgear/src/Viewer/fg_os_osgviewer.cxx, with the difference of ignoring Canvas and using native windowing features from OSG.

Related

Forum topics