Canvas view camera element: Difference between revisions

From FlightGear wiki
Jump to navigation Jump to search
m (Johan G moved page Canvas View Camera Element to Canvas view camera element: Sentence case title)
 
(107 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{Stub}}
{{FGCquote
  |It would make sense to integrate all efforts ([[CompositeViewer Support|CompositeViewer]], [[Canvas]] and [[Compositor]]) to create a [[Canvas_Development#Elements|Canvas Element]] that can render an [[Canvas_Development#Supporting_Cameras|arbitrary view]].


{{infobox subsystem
Some time ago I wrote what I think are the required steps to get something like cockpit mirrors working: [[#Roadmap]]
|image      = Canvas-view-element-prototype-by-icecode gl.png
 
|name        = Canvas Camera Views
The main showstopper was having [[CompositeViewer Support]], but since that effort is underway, all the parts are there for someone willing to dive in.
|started    = 08/2016 (prototyped by F-JJTH)
  |{{cite web |url=https://sourceforge.net/p/flightgear/mailman/message/37089780/
|description = adds support for rendering slave scenery views to a FBO/RTT (texture)
    |title=<nowiki>Independent view windows</nowiki>
|status      = experimental/known issues
    |author=<nowiki>Fernando García Liñán </nowiki>
|maintainers = none
    |date=<nowiki>2020-08-20 14:27:58</nowiki>
|developers  = F-JJTH, Icecode GL, Hooray
  }}
}}
}}


{{FGCquote
  |It might be interesting to try to define the Shuttle arm view (apologies if that's the wrong name) as an [[SView]] in its own top-level window. Apart from anything else i'd like to make sure that Sview supports whatever requires; I'd imagine that we would have one Sview step for each hinge/link in the arm for example, and this might need a new Sview step type.


Last updated in 09/2017
It would also be a useful incentive to getting Canvas views working.
  |{{cite web |url=https://forum.flightgear.org/viewtopic.php?f=87&t=35076&p=383492#p383492
    |title=<nowiki>Space Shuttle - Development</nowiki>
    |author=<nowiki>cgdae </nowiki>
    |date=<nowiki>Mon Mar 22, 2021 10:49 pm </nowiki>
  }}
}}


== Summary ==
{{infobox subsystem
|image      = cvcanvas-777.jpeg|Demonstration of multiple view windows in flightgear
|started    = 11/2020


Several aircraft developers have manifested their interest in being able to render to a texture (RTT) {{Wikipedia|Render target}} and use it inside cockpits as mirrors, external cameras (so called tail cams) and other uses. Effects and shaders developers have also reached a point where ignoring RTT is a waste of resources and a limitating factor when creating new effects. Although this Canvas element is directed mainly towards the first need, it is a first step forward in terms of finally exposing Render To Texture capabilities to non-C++ space too.
|name        = Canvas Camera Views
|description = Rendering arbitrary scene views to a texture/Canvas
|status      = experimental prototype
|maintainers = none
|developers  = Cgdae,  Icecode
}}


== Status ==
== Background ==
{{See also|View manager}}


We have a basic prototype, provided by F-JJTH, who developed the whole thing in mid-2016. There are a few issues, and it's using the legacy approach for implementing OD_gauge based avionics, i.e. not yet using the Canvas system.
Several aircraft developers have manifested their interest in being able to render scene views to a texture (RTT) {{Wikipedia|Render target}} and use it inside cockpits as mirrors, external cameras (so called tail cams) and other uses.


We also have a quick, but working, integration in the form of a custom Canvas element provided by Icecode GL. Originally, the image was mirrored because of different coordinate systems. Apparently the missing piece was to use FG's view manager, which is now fixed, thanks to Clement's contribution.
With the [[Compositor]] framework it's possible to configure custom render pipelines in XML space, which together with Effect schemes allow complete control over how a rendered scene looks. A synthetic terrain can be rendered by assigning an Effect scheme to a <tt>scene</tt> type pass, which textures all terrain with a brown checkerboard pattern and paints the sky in a solid blue color. All other objects are ignored and aren't rendered, saving a lot of computation time. Custom render distances independent from the main LOD settings can also be used to further optimize the performance of the display.<ref>https://forum.flightgear.org/viewtopic.php?f=71&t=23929&p=318025&#p317946</ref>


Who knows, with cleaner code and some tweaks we could have a deferred renderer and RTT capabilities in Canvas space, as long as the effect framework can access these RTT contexts, as Hooray proposed multiple times.<ref>{{cite web
== Use Cases ==
  |url    =  https://forum.flightgear.org/viewtopic.php?p=317743#p317743
<!-- https://forum.flightgear.org/viewtopic.php?f=87&t=35076&p=384484#p384465 -->
  |title  =  <nowiki> Re: Gear view in cockpit computer </nowiki>
{{#ev:youtube|YcdGjSvij90}}
  |author =  <nowiki> Icecode GL </nowiki>
  |date  =  Aug 28th, 2017
  |added  =  Aug 28th, 2017
  |script_version = 0.40
  }}</ref>


Also, Stuart offered to help review any patches and get them committed. Please ping Stuart when you think the code is worth checking in. It doesn't have to be perfect - Particularly once the current release is out (2017.3). <ref>{{cite web
[[File:AFS-Panel-11-2020.png|thumb|MD11 Autopilot dialog implemented in Canvas to replace the existing PUI dialog.<ref>https://forum.flightgear.org/viewtopic.php?f=4&t=32822&p=377189#p377189</ref>]]
  |url    =  https://forum.flightgear.org/viewtopic.php?p=317967#p317967
* Tail Cams (gear view)
  |title  =  <nowiki> Re: Gear view in cockpit computer </nowiki>
* shuttle RMS cam
  |author =  <nowiki> stuart </nowiki>
* Mirrors
  |date  = Sep 1st, 2017
* Dynamic in-sim view configuration
  |added  = Sep 1st, 2017
* On demand creation of views and windows (e.g. [[FGCamera]] previews)
  |script_version = 0.40
* Missile/payload views
  }}</ref>
* Prototyping/testing HUDs or PFDs requiring synthetic terrain to work properly
* rendering an orthographic view to implement [[Canvas_Tile_Element|moving maps]] that render actual FlightGear terrain/DEM
* automated creation/rendering of cockpit GUI dialogs by rendering a perspective-corrected front-view of cockpit elements (MCP, AP, CDU etc) <ref>https://forum.flightgear.org/viewtopic.php?f=4&t=32822&p=377189&#p377189</ref>


== Gallery ==
== Proof of Concept ==
{{Main article|Hackathon Proposal: CompositeViewer and Canvas}}


{|style="margin: 0 auto;"
All the work we did is on branch topics/cvcanvas of the main flightgear, simgear and fgdata repositories:
| [[File:Early-canvas-view-screenshot.png|thumb|Screenshot showing early experiments rendering a model to a Canvas view]]
| [[File:Icecode GL-Canvas-Camera-support.png|thumb|upright|An experimental [[Canvas]] element to render slaved scenery views to a custom Canvas texture, to be used for creating custom tail-cams/mirror textures and so on]]
| [[File:Icecodes slave cams 09-2017.png|thumb|upright|Several Canvas dialogs with view manager views]]
| [[File:Canvas-view-element-prototype-by-icecode gl.png|thumb|upright|[[Canvas]] gui dialog with a custom canvas element to display view manager views (based on code prototyped by F-JJTH)]]
|}


== Implementation details / for the reviewer ==
* {{repo link
 
| site  = sf
{{Note|This section provides a rough overview for the potentiaal reviewer, which is also intended to be used as the commit message when/if this should get committed}}
| repo  = flightgear
 
| branch = topics/cvcanvas/~
This set of patches (touching SimGear and fgdata) implements a new <code>Canvas::Element</code> by creating a sub-class named <code>Canvas::View</code>. The meat of it is in the constructor, i.e. <code>Canvas::View::View()</code>, where an off-screen camera (RTT/FBO) is set up, the FGCanvasSystemAdapter file has been extended to provide access to the FlightGear view manager to compute/obtain the view-specific view matrix, which is then used by the view element to update the offscreen camera in Canvas::View::update() accordingly.
}}
 
* {{repo link
This is also a good way to stress-test the renderer, as new cameras can be easily added to the scene at runtime, so that the impact of doing so can be easily measured.
| site  = sf
 
| repo  = simgear
To actually test the new element using the [[Nasal Console]] and paste the following snippet of code into it:
| branch = topics/cvcanvas/~
 
}}
<syntaxhighlight lang="nasal">
* {{repo link
var (width,height) = (512,512);
| site  = sf
var ELEMENT_NAME = "viewcam";
| repo  = fgdata
 
| branch = topics/cvcanvas/~
var display_view = func(view=0) {
}}  
var title = 'Canvas test:' ~ ELEMENT_NAME;
var window = canvas.Window.new([width,height],"dialog").set('title',title);
var myCanvas = window.createCanvas().set("background", canvas.style.getColor("bg_color"));
var root = myCanvas.createGroup();
 
var child = root.createChild( ELEMENT_NAME );
# child.set("view-number", view);
} # display_view()
 
 
var totalViews = props.getNode("sim").getChildren("view");
forindex(var v;totalViews) {
display_view(view: v);
}</syntaxhighlight>
 
Aircraft developers can use the new Canvas '''view''' element simply by using the conventional approach to replace a static texture with a Canvas using a cockpit placement.
 
== Roadmap ==
{{PatchAvailable|url=https://forum.flightgear.org/viewtopic.php?f=71&t=23929#p317448}}




* extend FGCanvasSystemAdapter to make the view manager available there <ref>https://forum.flightgear.org/viewtopic.php?f=71&t=23929&p=317798#p317798</ref>
<!--
* come up with a new Canvas element inheriting from Canvas::Image {{Progressbar|80}}
{{Note|This is inspired by the long standing idea to render camera views to a texture for use as tailcams, gear views etc <ref>https://forum.flightgear.org/viewtopic.php?f=71&t=23929</ref>, it's based on code originally provided by F-JJTH<ref>https://forum.flightgear.org/viewtopic.php?f=71&t=23929#p317448</ref>, and subsequently reworked by Icecode GL and Hooray to integrate it with the Canvas system back in 2017<ref>https://forum.flightgear.org/viewtopic.php?f=47&t=32846&p=318046</ref>
* add Clement's camera setup routines
At the time, the main issue was that the new view element needed to be explicitly updated, but that should no longer be the case thanks to James' work on the setPixel() API <ref>https://sourceforge.net/p/flightgear/mailman/message/36926486/</ref> which also means that sc::Image now has a <code>dirtyPixels()</code> API which we can reuse <ref>https://sourceforge.net/p/flightgear/simgear/ci/ad3621e23b4056e1be7e8d9eb8dd4d513455add8/</ref>
* make the camera render into the texture used by the sub-class
Due to the adoption of the [[Compositor]] framework, this will probably need to reworked to be properly integrated.
* this should be named viewmgr-camera or something like that to make it obvious what it is doing
Furthermore, beginning in mid 2020 Julian Smith has started working on optional [[CompositeViewer Support]] so that independent scene views can be rendered.  }}
 
== Testing ==
Once we have a basic prototype working, the new camera element needs to be tested, specifically:
* using different resolutions (resizing at runtime?)
* displaying multiple/independent camera views
* all supported Canvas placements
* shaders and effects
* ALS
* Rembrandt
* Reset/re-init
* OSG threading modes
 
== Known Issues ==
* texture/view matrix updates don't currently take effect unless .update() is specifically invoked
* <del>skydome handling (probably missing effect/shader or wrong root node)</del> {{Done}} (by Icecode_GL)
* change DATA_VARIANCE for texture setup, analogous to the Canvas::Image::Image ctor
* make the size of the texture re-configurable using properties
 
== Ideas ==
* support draw-masks for scene graph features (terrain, skydome, models etc): " it'd be benefitial in the long run to let the user choose what part of the scene graph they want to show. This would be useful for deferred rendering schemes, where sometimes you don't need to display the whole scenery to save some precious frames. Canvas makes that easy with all of its property handling"
* work out what is needed for [https://forum.flightgear.org/viewtopic.php?f=71&t=27985&p=264718&hilit=synthetic+terrain#p264718v synthetic terrain views] [http://www.aircraftspruce.eu/catalog/graphics/10-03932s.jpg]
* support effects/shaders: This is fairly straightforward to do, just create a EffectsGeode, fill it with a "compiled" .eff, and substitute the standard osg::Geode for it. But I don't think that's going to give us the flexibility we need. A shader won't be able to access several canvases at the same time if we do things that way.
 
== Performance / Optimizations ==
* provide a view-cache using a STL std::map<> ?
* use PBOs [http://forum.openscenegraph.org/viewtopic.php?t=10713&view=next]
* consider hooking up the whole thing to the FGStatsManager (OSG stats) ?
 
== C++ ==
The following is the boilerplate code neede to add a new element to the Canvas system inheriting from the Canvas::Image class, i.e. an element having its own allocated Image/texture buffer, which is what we can use to render a camera into it:


SimGear:
<syntaxhighlight lang="diff">
<syntaxhighlight lang="diff">
 
diff --git a/simgear/canvas/CanvasSystemAdapter.hxx b/simgear/canvas/CanvasSystemAdapter.hxx
index 43b3f780..0fb62170 100644
--- a/simgear/canvas/CanvasSystemAdapter.hxx
+++ b/simgear/canvas/CanvasSystemAdapter.hxx
@@ -35,11 +35,13 @@ namespace canvas
      virtual ~SystemAdapter() {}
      virtual FontPtr getFont(const std::string& name) const = 0;
-      virtual void addCamera(osg::Camera* camera) const = 0;
+      virtual void addCamera(osg::Camera* camera, bool useSceneData = false) const = 0;
      virtual void removeCamera(osg::Camera* camera) const = 0;
      virtual osg::ref_ptr<osg::Image> getImage(const std::string& path) const = 0;
+      virtual osg::Node* getModel(const std::string& path) const = 0;
      virtual SGSubsystem* getSubsystem(const std::string& name) const = 0;
      virtual HTTP::Client* getHTTPClient() const = 0;
+      virtual osg::Matrix getViewMatrix(int view_number, double dt) const = 0;
  };
} // namespace canvas
diff --git a/simgear/canvas/canvas_fwd.hxx b/simgear/canvas/canvas_fwd.hxx
index 3b1b4646..97e9bd64 100644
--- a/simgear/canvas/canvas_fwd.hxx
+++ b/simgear/canvas/canvas_fwd.hxx
@@ -24,6 +24,7 @@
#include <osg/ref_ptr>
#include <osgText/Font>
+#include <osg/Node>
#include <boost/function.hpp>
#include <boost/shared_ptr.hpp>
diff --git a/simgear/canvas/elements/CMakeLists.txt b/simgear/canvas/elements/CMakeLists.txt
diff --git a/simgear/canvas/elements/CMakeLists.txt b/simgear/canvas/elements/CMakeLists.txt
index 2b537c0..4eb1e95 100644
index 2b537c0c..4eb1e955 100644
--- a/simgear/canvas/elements/CMakeLists.txt
--- a/simgear/canvas/elements/CMakeLists.txt
+++ b/simgear/canvas/elements/CMakeLists.txt
+++ b/simgear/canvas/elements/CMakeLists.txt
Line 154: Line 144:
+)
+)
diff --git a/simgear/canvas/elements/CanvasGroup.cxx b/simgear/canvas/elements/CanvasGroup.cxx
diff --git a/simgear/canvas/elements/CanvasGroup.cxx b/simgear/canvas/elements/CanvasGroup.cxx
index c753c2e..5d2585e 100644
index c753c2e2..6cec072f 100644
--- a/simgear/canvas/elements/CanvasGroup.cxx
--- a/simgear/canvas/elements/CanvasGroup.cxx
+++ b/simgear/canvas/elements/CanvasGroup.cxx
+++ b/simgear/canvas/elements/CanvasGroup.cxx
@@ -23,6 +23,8 @@
@@ -23,6 +23,7 @@
  #include "CanvasMap.hxx"
  #include "CanvasMap.hxx"
  #include "CanvasPath.hxx"
  #include "CanvasPath.hxx"
  #include "CanvasText.hxx"
  #include "CanvasText.hxx"
+#include "CanvasView.hxx"
+#include "CanvasView.hxx"
+
  #include <simgear/canvas/CanvasEventVisitor.hxx>
  #include <simgear/canvas/CanvasEventVisitor.hxx>
  #include <simgear/canvas/events/MouseEvent.hxx>
  #include <simgear/canvas/events/MouseEvent.hxx>
   
   
@@ -66,6 +68,7 @@ namespace canvas
@@ -66,6 +67,7 @@ namespace canvas
     add<Map  >(_child_factories);
     add<Map  >(_child_factories);
     add<Path >(_child_factories);
     add<Path >(_child_factories);
Line 176: Line 165:
diff --git a/simgear/canvas/elements/CanvasView.cxx b/simgear/canvas/elements/CanvasView.cxx
diff --git a/simgear/canvas/elements/CanvasView.cxx b/simgear/canvas/elements/CanvasView.cxx
new file mode 100644
new file mode 100644
index 0000000..edae7ab
index 00000000..502e82d7
--- /dev/null
--- /dev/null
+++ b/simgear/canvas/elements/CanvasView.cxx
+++ b/simgear/canvas/elements/CanvasView.cxx
@@ -0,0 +1,88 @@
@@ -0,0 +1,91 @@
+// This library is free software; you can redistribute it and/or
+#include <simgear/canvas/Canvas.hxx>
+// modify it under the terms of the GNU Library General Public
+#include <simgear/canvas/CanvasMgr.hxx>
+// License as published by the Free Software Foundation; either
+#include <simgear/canvas/CanvasSystemAdapter.hxx>
+// version 2 of the License, or (at your option) any later version.
+
+//
+#include <simgear/scene/util/OsgMath.hxx>
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// Library General Public License for more details.
+//
+// You should have received a copy of the GNU Library General Public
+// License along with this library; if not, write to the Free Software
+// Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301, USA
+
+
+#include <simgear_config.h>
+#include "CanvasView.hxx"
+#include "CanvasView.hxx"
+
+
+#include <cmath>
+#include <iostream>
+
+#include <osg/io_utils>
+#include <boost/algorithm/string/predicate.hpp>
+
+
+namespace simgear
+namespace simgear
Line 206: Line 185:
+{
+{
+
+
+ //----------------------------------------------------------------------------
+const std::string View::TYPE_NAME = "view";
const std::string View::TYPE_NAME = "view-camera";
+
+void View::staticInit()
+{
+    if ( isInit<View>() )
+        return;
+}
+
+
+ //----------------------------------------------------------------------------
+View::View( const CanvasWeakPtr& canvas,
+ void View::staticInit()
+              const SGPropertyNode_ptr& node,
+ {
+              const Style& parent_style,
+    Image::staticInit();
+              ElementWeakPtr parent ) :
+    Element(canvas, node, parent_style, parent),
+    _width(512),
+    _height(512)
+{
+    staticInit();
+
+    _texture = new osg::Texture2D;
+   _texture->setDataVariance(osg::Object::DYNAMIC);
+    _texture->setTextureSize(_width, _height);
+   _texture->setInternalFormat(GL_RGBA);
+    _texture->setFilter(osg::Texture2D::MIN_FILTER, osg::Texture2D::LINEAR);
+    _texture->setFilter(osg::Texture2D::MAG_FILTER, osg::Texture2D::LINEAR);
+
+
+    if( isInit<View>() )
+    _camera = new osg::Camera;
+     return;
+    _camera->setDataVariance(osg::Object::DYNAMIC);
+    _camera->setViewport(0, 0, _width, _height);
+    _camera->setClearColor(osg::Vec4(1.0f, 1.0f, 1.0f, 1.0f));
+    _camera->setClearMask(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
+   _camera->setRenderOrder(osg::Camera::PRE_RENDER);
+    _camera->setRenderTargetImplementation(osg::Camera::FRAME_BUFFER_OBJECT, osg::Camera::FRAME_BUFFER);
+    _camera->setReferenceFrame(osg::Transform::ABSOLUTE_RF);
+    double aspect_ratio = (double)_width / _height;
+    _camera->setProjectionMatrixAsPerspective(85.0, aspect_ratio, 0.1, 120000.0);
+
+
+    // Do some initialization if needed...
+    _camera->attach(osg::Camera::COLOR_BUFFER, _texture.get());
+  }
+
+
+ //----------------------------------------------------------------------------
+   if (Canvas::getSystemAdapter())
+  View::View( const CanvasWeakPtr& canvas,
+       Canvas::getSystemAdapter()->addCamera(_camera.get(), true);
+           const SGPropertyNode_ptr& node,
+            const Style& parent_style,
+            ElementWeakPtr parent ):
+    Image(canvas, node, parent_style, parent)
+  {
+    staticInit();
+  }
+
+
+ //----------------------------------------------------------------------------
+   _quad = osg::createTexturedQuadGeometry(osg::Vec3(0.0,    0.0,    0.0),
+ View::~View()
+                                            osg::Vec3(_width, 0.0,    0.0),
+ {
+                                           osg::Vec3(0.0,    _height, 0.0),
+                                           // In Canvas (0,0) is the top left corner, while in OSG/OpenGL
+                                            // it's the bottom left corner.
+                                            0.0, 1.0, 1.0, 0.0);
+    _quad->getOrCreateStateSet()->setTextureAttributeAndModes(0, _texture.get(), osg::StateAttribute::ON);
+
+
+ }
+   setDrawable(_quad.get());
+}
+
+
+ //----------------------------------------------------------------------------
+View::~View()
+  void View::update(double dt)
+{
+ {
+    if( _camera.valid() && Canvas::getSystemAdapter() )
+    Image::update(dt);
+        Canvas::getSystemAdapter()->removeCamera(_camera.get());
+ }
+    _camera.release();
+    _texture.release();
+}
+
+
+ //----------------------------------------------------------------------------
+void View::update(double dt)
void View::childAdded(SGPropertyNode* parent, SGPropertyNode* child)
+{
+ {
+   osg::Matrix view_matrix = Canvas::getSystemAdapter()->getViewMatrix(0, dt);
+     return Image::childAdded(parent, child);
+   _camera->setViewMatrix(view_matrix);
+ }
+
+
+ //----------------------------------------------------------------------------
+   Element::update(dt);
+  void View::childRemoved(SGPropertyNode* parent, SGPropertyNode* child)
+}
+  {
+      return Image::childRemoved(parent, child);
+ }
+
+
+ //----------------------------------------------------------------------------
+void View::resize(int width, int height)
void View::valueChanged(SGPropertyNode* child)
+{
+ {
+    return Image::valueChanged(child);
+  }
+
+
+ //----------------------------------------------------------------------------
+}
+  void View::childChanged(SGPropertyNode* child)
+  {
+      return Image::childChanged(child);
+
+
+  }
+} // namespace canvas
+} // namespace canvas
+} // namespace simgear
+} // namespace simgear
diff --git a/simgear/canvas/elements/CanvasView.hxx b/simgear/canvas/elements/CanvasView.hxx
diff --git a/simgear/canvas/elements/CanvasView.hxx b/simgear/canvas/elements/CanvasView.hxx
new file mode 100644
new file mode 100644
index 0000000..7b2ec4e
index 00000000..8cb3cc49
--- /dev/null
--- /dev/null
+++ b/simgear/canvas/elements/CanvasView.hxx
+++ b/simgear/canvas/elements/CanvasView.hxx
@@ -0,0 +1,59 @@
@@ -0,0 +1,39 @@
+// Based on https://forum.flightgear.org/viewtopic.php?f=71&t=23929
+//
+// This library is free software; you can redistribute it and/or
+// modify it under the terms of the GNU Library General Public
+// License as published by the Free Software Foundation; either
+// version 2 of the License, or (at your option) any later version.
+//
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+// Library General Public License for more details.
+//
+// You should have received a copy of the GNU Library General Public
+// License along with this library; if not, write to the Free Software
+// Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301, USA
+
+#ifndef CANVAS_VIEW_HXX_
+#ifndef CANVAS_VIEW_HXX_
+#define CANVAS_VIEW_HXX_
+#define CANVAS_VIEW_HXX_
+
+
+#include "CanvasImage.hxx"
+#include "CanvasElement.hxx"
+
+#include <boost/shared_ptr.hpp>
+#include <boost/unordered_map.hpp>
+#include <boost/unordered_set.hpp>
+
+
+namespace simgear
+namespace simgear
Line 303: Line 275:
+namespace canvas
+namespace canvas
+{
+{
+  class View:
+    public Image
+  {
+    public:
+      static const std::string TYPE_NAME;
+      static void staticInit();
+
+
+     View( const CanvasWeakPtr& canvas,
+class View : public Element {
+public:
+    static const std::string TYPE_NAME;
+    static void staticInit();
+
+    View( const CanvasWeakPtr& canvas,
+          const SGPropertyNode_ptr& node,
+          const SGPropertyNode_ptr& node,
+          const Style& parent_style,
+          const Style& parent_style = Style(),
+          ElementWeakPtr parent = 0 );
+          ElementWeakPtr parent = 0 );
+     virtual ~View();
+   virtual ~View();
+
+    virtual void update(double dt);
+
+    virtual void resize(int width, int height);
+
+
+     virtual void update(double dt);
+protected:
+
+
+     virtual void childAdded( SGPropertyNode * parent,
+   osg::ref_ptr<osg::Camera> _camera;
+                              SGPropertyNode * child );
+   osg::ref_ptr<osg::Texture2D> _texture;
+      virtual void childRemoved( SGPropertyNode * parent,
+                                SGPropertyNode * child );
+     virtual void valueChanged(SGPropertyNode * child);
+
+
+    //protected:
+    osg::ref_ptr<osg::Geometry> _quad;
+
+
+     virtual void childChanged(SGPropertyNode * child);
+   int _width, _height;
+ };
+};
+
+
+} // namespace canvas
+} // namespace canvas
+} // namespace simgear
+} // namespace simgear
+
+
+#endif /* CANVAS_VIEW_HXX_ */
+#endif // CANVAS_VIEW_HXX_
</syntaxhighlight>
 
FlightGear
<syntaxhighlight lang="diff">   
diff --git a/src/Canvas/FGCanvasSystemAdapter.cxx b/src/Canvas/FGCanvasSystemAdapter.cxx
index 72b20747e..9e04dbd14 100644
--- a/src/Canvas/FGCanvasSystemAdapter.cxx
+++ b/src/Canvas/FGCanvasSystemAdapter.cxx
@@ -23,9 +23,18 @@
#include <Network/HTTPClient.hxx>
#include <Viewer/renderer.hxx>
 
+
+#include <Viewer/view.hxx>
+#include <Viewer/viewmgr.hxx>
+#include <simgear/scene/util/OsgMath.hxx>
+
+
+
#include <osgDB/ReadFile>
#include <stdexcept>
 
+#include <simgear/scene/model/modellib.hxx>
+
namespace canvas
{
//----------------------------------------------------------------------------
@@ -64,9 +73,9 @@ namespace canvas
}
 
//----------------------------------------------------------------------------
-  void FGCanvasSystemAdapter::addCamera(osg::Camera* camera) const
+  void FGCanvasSystemAdapter::addCamera(osg::Camera* camera, bool useSceneData) const
{
-    globals->get_renderer()->addCamera(camera, false);
+    globals->get_renderer()->addCamera(camera, useSceneData);
}
 
//----------------------------------------------------------------------------
@@ -100,6 +109,46 @@ namespace canvas
return 0;
}
 
+    //----------------------------------------------------------------------------
+    // From http://wiki.flightgear.org/Howto:Extending_Canvas_to_support_rendering_3D_models#Extending_FGCanvasSystemAdapter
+    osg::Node* FGCanvasSystemAdapter::getModel(const std::string& path) const
+    {
+        const char *model_path = "Models/Geometry/glider.ac";
+        SGPath p(SGPath::fromUtf8(path));
+
+        if( p.isAbsolute() )
+        {
+            SGPath valid_path = fgValidatePath(p, false);
+            if( !valid_path.isNull() )
+            try {
+                std::string fullPath = simgear::SGModelLib::findDataFile(valid_path.local8BitStr());
+                osg::Node * object = simgear::SGModelLib::loadDeferredModel(fullPath, globals->get_props());
+                return object;
+            } catch (const sg_throwable& t) {
+                SG_LOG(SG_IO, SG_ALERT, "Error loading " << model_path << ":\n  " << t.getFormattedMessage() << t.getOrigin());
+                return 0;
+            } // error loading from absolute path
+            SG_LOG(SG_IO, SG_ALERT, "canvas::Model: reading '" << path << "' denied");
+        } // absolute path handling
+        else
+        {
+            SGPath tpath = globals->resolve_resource_path(path);
+            if( !tpath.isNull() )
+            try {
+                std::string fullPath = simgear::SGModelLib::findDataFile(path.c_str());
+                osg::Node * object = simgear::SGModelLib::loadDeferredModel(fullPath, globals->get_props());
+                return object;
+            } catch (const sg_throwable& t) {
+                SG_LOG(SG_IO, SG_ALERT, "Error loading " << model_path << ":\n  " << t.getFormattedMessage() << t.getOrigin());
+                return 0;
+            } // error loading from relative path
+
+            SG_LOG(SG_IO, SG_ALERT, "canvas::Model: No such model: '" << path << "'");
+        } // relative path handling
+
+        return 0;
+    }
+
//----------------------------------------------------------------------------
SGSubsystem*
FGCanvasSystemAdapter::getSubsystem(const std::string& name) const
@@ -121,4 +170,16 @@ namespace canvas
return 0;
}
 
+  osg::Matrix FGCanvasSystemAdapter::getViewMatrix(int view_number, double dt) const
+  {
+        flightgear::View *view = globals->get_viewmgr()->get_view(view_number);
+        view->update(dt);
+
+        osg::Vec3f position( toOsg(view->getViewPosition()) );
+        osg::Quat orientation( toOsg(view->getViewOrientation()) );
+        osg::Matrix viewMatrix( osg::Matrix::translate(-position) * osg::Matrix::rotate(orientation.inverse()) );
+
+        return viewMatrix;
+  }
+
}
diff --git a/src/Canvas/FGCanvasSystemAdapter.hxx b/src/Canvas/FGCanvasSystemAdapter.hxx
index 4c1fd6210..24ececb41 100644
--- a/src/Canvas/FGCanvasSystemAdapter.hxx
+++ b/src/Canvas/FGCanvasSystemAdapter.hxx
@@ -28,11 +28,13 @@ namespace canvas
{
public:
virtual simgear::canvas::FontPtr getFont(const std::string& name) const;
-      virtual void addCamera(osg::Camera* camera) const;
+      virtual void addCamera(osg::Camera* camera, bool useSceneData = false) const;
virtual void removeCamera(osg::Camera* camera) const;
virtual osg::ref_ptr<osg::Image> getImage(const std::string& path) const;
+      virtual osg::Node* getModel(const std::string& path) const;
virtual SGSubsystem* getSubsystem(const std::string& name) const;
virtual simgear::HTTP::Client* getHTTPClient() const;
+      virtual osg::Matrix getViewMatrix(int view_number, double dt) const;
};
}
 
diff --git a/src/Viewer/renderer.cxx b/src/Viewer/renderer.cxx
index d9e8f351f..b9b3fc6a0 100644
--- a/src/Viewer/renderer.cxx
+++ b/src/Viewer/renderer.cxx
@@ -1845,6 +1845,8 @@ FGRenderer::setEventHandler(FGEventHandler* eventHandler_)
void
FGRenderer::addCamera(osg::Camera* camera, bool useSceneData)
{
+    if (useSceneData)
+        camera->addChild(globals->get_scenery()->get_scene_graph());
_viewerSceneRoot->addChild(camera);
}




</syntaxhighlight>
</syntaxhighlight>


== Base Package ==
fgdata
{{collapsible script
<syntaxhighlight lang="diff">
| type  = view-camera
| title  = view-camera  $FG_ROOT/Nasal/canvas/api.nas changes
| intro = Finally, we need to modify api.nas to provide Nasal wrappers
| script =  
diff --git a/Nasal/canvas/api.nas b/Nasal/canvas/api.nas
diff --git a/Nasal/canvas/api.nas b/Nasal/canvas/api.nas
index 6d39d03..f98fdd9 100644
index 3f7f5a17..433a41f9 100644
--- a/Nasal/canvas/api.nas
--- a/Nasal/canvas/api.nas
+++ b/Nasal/canvas/api.nas
+++ b/Nasal/canvas/api.nas
@@ -1099,13 +1099,21 @@ var Image = {
@@ -1132,10 +1132,10 @@ var Image = {
   }
   }
  };
  };
   
   
+var viewCamera = {
-var Model = {
new: func(ghost)
+var View = {
{
  new: func(ghost)
+    return {parents: [viewCamera, Element.new(ghost)]};
  {
},
-    return {parents: [Model, Element.new(ghost)]};
+};
+    return {parents: [View, Element.new(ghost)]};
+
  },
  # Element factories used by #Group elements to create children
};
Group._element_factories = {
   
  "group": Group.new,
@@ -1146,7 +1146,7 @@ Group._element_factories = {
  "map": Map.new,
   "text": Text.new,
   "text": Text.new,
   "path": Path.new,
   "path": Path.new,
"image": Image.new
  "image": Image.new,
+ "image": Image.new,
- "model": Model.new
+  "view-camera": viewCamera.new,
+  "view": View.new
  };
  };
| lang = diff
}}
# Canvas
</syntaxhighlight>
-->
 
== Roadmap ==
 
=== Use CompositeViewer ===
{{Progressbar|80}} {{See also|CompositeViewer Support}}
<small>
Originally, FlightGear only used one instance of <tt>osg::Viewer</tt>, which is used by CameraGroup to manage the slave cameras. Supporting CompositeViewer did require modifying {{flightgear file|src/Viewer/fg_os_osgviewer.cxx}} to create some kind of wrapper class that manages the CompositeViewer instance and assign a CameraGroup to each <tt>osg::View</tt>. It's also important to note that currently all FG subsystems assume there is a single instance of [[Howto:CameraGroup talks|CameraGroup]].</small>
 
=== Standardize the View manager  ===
{{Note|As of 11/2020, Jules has already created a custom view manager implementation [[StepView]] that supports multiple independent instances, so this step might be obsolete by now.}}
 
<small>The first step would be to port the view code to SimGear so it can be used and known by the Compositor and Canvas. The view manager ({{flightgear file|src/Viewer/viewmgr.cxx}}) currently has some hardcoded assumptions, so it would either need to be rewritten to remove them or a new interface for the Views could be created specifically for the Canvas Camera View.
 
In FlightGear, the Canvas system (which resides in SimGear) is integrated using the equivalent of a FGCanvasSystemAdapter, which provides all FG APIs to SimGear and makes the Canvas system available inside FG: {{flightgear file|src/Canvas/FGCanvasSystemAdapter.cxx}}
 
Instead of using tied properties, the adapted view code would either used propertyObject<> or the propertyBasedMgr abstraction, so that the corresponding canvas element can continue to use well-known conventions to manipulate views: ({{flightgear file|src/Viewer/view.cxx}})
</small>


== Nasal Console ==
=== Create a Canvas Element subclass ===
[[File:CanvasView-Prototyping-UI.png|thumb|[[Canvas]] GUI dialog (code at [http://wiki.flightgear.org/File_talk:CanvasView-Prototyping-UI.png]) to provide a UI <ref>https://sourceforge.net/p/flightgear/mailman/message/37163870/</ref> for the experimental [[Canvas View Camera Element]] prototyped during the [[Virtual FSweekend Hackathon 2020]], for more details see: [[Hackathon Proposal: CompositeViewer and Canvas]]]]


After patching and rebuilding SimGear/FlightGear respectively, and applying the changes to to api.nas in the base package, the following can be pasted into the [[Nasal Console]] for testing purposes:
[[File:CompositeViewer-prototype-UI.png|thumb|[[Canvas]] based GUI dialog to create [[CompositeViewer Support|CompositeViewer views]] dynamically using a configurable [[Compositor]] pipeline.]]


<syntaxhighlight lang="nasal">
{{Progressbar|30}}
# TODO: this must match the TYPE_NAME used by the C++ code
{{See also|Talk:Hackathon Proposal: CompositeViewer and Canvas}}
var ELEMENT_NAME ="view-camera"; # to be adapted according to the C++ changes


var myElementTest = {
This derived class would require the following configuration:
##
* [[Compositor]] to use.
# constructor
* Scene graph to use. By default the main scene graph would be used, but an arbitrary XML file can be loaded to [[Howto:Extending Canvas to support rendering 3D models|render a custom model]]. A typical use-case would be instruments that need to [[Shuttle ADI ball|manipulate/render a 3D object]] (for which we have working code, too)
new: func( camera ) {
* [[View manager|View]] to use. This could be one of the "main" ones (i.e. the ones on the main property tree), or a locally-defined one that is only known to this Canvas Element (think FLIR).
  var m = { parents: [myElementTest] };
  m.dlg = canvas.Window.new([camera.width,camera.height],"dialog");
  m.canvas = m.dlg.createCanvas().setColorBackground(1,1,1,1);
  m.root = m.canvas.createGroup();


  ##
Several optimization/miscellaneous parameters exposed in the form of properties per element, like:
  # instantiate a new element
* texture size/resolution (see Fernando's comments here: [[Talk:Hackathon_Proposal:_CompositeViewer_and_Canvas#SceneGraph_Cameras_vs._Prerender_Cams]] )
  m.myElement = m.root.createChild( ELEMENT_NAME );
* OSG rendering mode (continuous vs. lazy/on demand): <code>setRunFrameScheme( osgViewer::ViewerBase::ON_DEMAND );</code> <ref>https://forum.flightgear.org/viewtopic.php?f=71&t=17650&hilit=knuq+taxiways&start=15#p166990</ref> <ref>https://stackoverflow.com/a/12259344</ref>
  # set the view-number property
* framerate cap via something like <code>view.setRunMaxFrameRate()</code> <ref>https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg26193.html</ref> [https://www.programmersought.com/article/23264806182/] [https://www.mail-archive.com/osg-users@openscenegraph.net/msg14033.html] (background: https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg26178.html )
  m.myElement.set("view-number", camera.view);
* [[Draw masks|node masks]]
  m.dlg.set("title", "view #"~camera.view); # TODO look up proper title/name
* [[Level Of Detail (LOD) Ranges|LOD]] ranges etc. <ref>https://forum.flightgear.org/viewtopic.php?f=71&t=32845&p=318027&hilit=canvas+draw+masks#p318028</ref>
* PagedLOD settings for <code>PIXEL_SIZE_ON_SCREEN</code> mode [https://sourceforge.net/p/flightgear/mailman/message/36438416/] [http://alphapixel.com/wp-content/uploads/2015/04/LOD-Level-of-detail-in-OpenSceneGraph-OSG.pdf] [https://osg-users.openscenegraph.narkive.com/DNuWtUiR/pagedlods-and-pixel-to-screen-size]
* whether or not to enable OSG StatsHandler per view, i.e. for troubleshooting per view
* whether or not to enable event handling: "Extra view windows don't yet handle events so one cannot change the view angle/zoom after creation." <ref>https://sourceforge.net/p/flightgear/mailman/message/37161269/</ref>


  return m;
Note, we can use the view's Camera's LODScale to adjust the which level of LOD child is selected for each view.<ref>https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg66861.html</ref>
  }, # new
Another common suggestion is to control the update rate of slave cameras by inserting them into the scene graph as needed, and then use a PostDrawCallback for the camera that removes the
   
camera again when it is no longer needed.<ref>https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg66868.html</ref>
}; # end of myElementTest
Robert (OSG) states that most apps should manage the frame for their own applications and data. The OSG is a general purpose scene graph rather than a domain specific IG.  It gives you the tools to do your
job, but it doesn't do it all for you.  The run() method stuff available in OSG/Viewer is really only for entry level app development.<ref>https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg26257.html</ref>
More sophisticated types of frame rate management are well beyond what  the viewer convenience methods like run are supposed to handle. People  should expect to roll your own frame management for this type of app.<ref>https://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg26184.html</ref>


# TODO: get list of all views ?


var cameras = [
Simplifying a lot, this Canvas Element would be an aggregation of:
{view: 0, width : 640, height: 480},
* a Compositor instance / effect scheme
{view: 1, width : 320, height: 160},
* a View
{view: 2, width : 320, height: 160},
* a pointer to a <tt>osg::Group</tt> representing the scene graph to render (e.g. alias or filename based).
];


foreach(var cam; cameras) {
This setup could be replicated in {{flightgear file|src/Viewer/fg_os_osgviewer.cxx}}, with the difference of ignoring Canvas and using native windowing features from OSG (see Jules' fgcommands to clone a view and open a dedicated GraphicsContext/window).
var newCam = myElementTest.new( cam );
}
</syntaxhighlight>


== Related ==
== Related ==
=== Forum topics ===
* {{forum link|t=6184|text=Mirrors}}
* {{forum link|t=32846|text=Canvas::View Development}}
* {{forum link|t=23929|text=Gear View in Cockpit}}
* {{forum link|t=32845|text=Canvas::View element: performance/optimizations}}
* {{forum link|t=20057|text=Rear-View mirror}}
* {{forum link|t=18905|text=Progress on synthetic terrain}}
* {{forum link|t=17184|text=Instruments with heightmaps}}
* {{forum link|t=38738|text=Compositor pipeline questions}}
== References ==
{{Appendix}}
{{Appendix}}
[[Category:Canvas Element Proposals]]

Latest revision as of 06:51, 10 May 2021

Cquote1.png It would make sense to integrate all efforts (CompositeViewer, Canvas and Compositor) to create a Canvas Element that can render an arbitrary view.

Some time ago I wrote what I think are the required steps to get something like cockpit mirrors working: #Roadmap

The main showstopper was having CompositeViewer Support, but since that effort is underway, all the parts are there for someone willing to dive in.


— Fernando García Liñán  (2020-08-20 14:27:58). Independent view windows.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png It might be interesting to try to define the Shuttle arm view (apologies if that's the wrong name) as an SView in its own top-level window. Apart from anything else i'd like to make sure that Sview supports whatever requires; I'd imagine that we would have one Sview step for each hinge/link in the arm for example, and this might need a new Sview step type.

It would also be a useful incentive to getting Canvas views working.


— cgdae  (Mon Mar 22, 2021 10:49 pm ). Space Shuttle - Development.
(powered by Instant-Cquotes)
Cquote2.png
Canvas Camera Views
Cvcanvas-777.jpeg
Started in 11/2020
Description Rendering arbitrary scene views to a texture/Canvas
Maintainer(s) none
Contributor(s) Cgdae, Icecode
Status experimental prototype

Background

Several aircraft developers have manifested their interest in being able to render scene views to a texture (RTT) Render target This is a link to a Wikipedia article and use it inside cockpits as mirrors, external cameras (so called tail cams) and other uses.

With the Compositor framework it's possible to configure custom render pipelines in XML space, which together with Effect schemes allow complete control over how a rendered scene looks. A synthetic terrain can be rendered by assigning an Effect scheme to a scene type pass, which textures all terrain with a brown checkerboard pattern and paints the sky in a solid blue color. All other objects are ignored and aren't rendered, saving a lot of computation time. Custom render distances independent from the main LOD settings can also be used to further optimize the performance of the display.[1]

Use Cases

MD11 Autopilot dialog implemented in Canvas to replace the existing PUI dialog.[2]
  • Tail Cams (gear view)
  • shuttle RMS cam
  • Mirrors
  • Dynamic in-sim view configuration
  • On demand creation of views and windows (e.g. FGCamera previews)
  • Missile/payload views
  • Prototyping/testing HUDs or PFDs requiring synthetic terrain to work properly
  • rendering an orthographic view to implement moving maps that render actual FlightGear terrain/DEM
  • automated creation/rendering of cockpit GUI dialogs by rendering a perspective-corrected front-view of cockpit elements (MCP, AP, CDU etc) [3]

Proof of Concept

1rightarrow.png See Hackathon Proposal: CompositeViewer and Canvas for the main article about this subject.

All the work we did is on branch topics/cvcanvas of the main flightgear, simgear and fgdata repositories:


Roadmap

Use CompositeViewer

80}% completed

Originally, FlightGear only used one instance of osg::Viewer, which is used by CameraGroup to manage the slave cameras. Supporting CompositeViewer did require modifying flightgear/src/Viewer/fg_os_osgviewer.cxx to create some kind of wrapper class that manages the CompositeViewer instance and assign a CameraGroup to each osg::View. It's also important to note that currently all FG subsystems assume there is a single instance of CameraGroup.

Standardize the View manager

Note  As of 11/2020, Jules has already created a custom view manager implementation StepView that supports multiple independent instances, so this step might be obsolete by now.

The first step would be to port the view code to SimGear so it can be used and known by the Compositor and Canvas. The view manager (flightgear/src/Viewer/viewmgr.cxx) currently has some hardcoded assumptions, so it would either need to be rewritten to remove them or a new interface for the Views could be created specifically for the Canvas Camera View.

In FlightGear, the Canvas system (which resides in SimGear) is integrated using the equivalent of a FGCanvasSystemAdapter, which provides all FG APIs to SimGear and makes the Canvas system available inside FG: flightgear/src/Canvas/FGCanvasSystemAdapter.cxx

Instead of using tied properties, the adapted view code would either used propertyObject<> or the propertyBasedMgr abstraction, so that the corresponding canvas element can continue to use well-known conventions to manipulate views: (flightgear/src/Viewer/view.cxx)

Create a Canvas Element subclass

Canvas GUI dialog (code at [1]) to provide a UI [4] for the experimental Canvas View Camera Element prototyped during the Virtual FSweekend Hackathon 2020, for more details see: Hackathon Proposal: CompositeViewer and Canvas
Canvas based GUI dialog to create CompositeViewer views dynamically using a configurable Compositor pipeline.

30}% completed

This derived class would require the following configuration:

  • Compositor to use.
  • Scene graph to use. By default the main scene graph would be used, but an arbitrary XML file can be loaded to render a custom model. A typical use-case would be instruments that need to manipulate/render a 3D object (for which we have working code, too)
  • View to use. This could be one of the "main" ones (i.e. the ones on the main property tree), or a locally-defined one that is only known to this Canvas Element (think FLIR).

Several optimization/miscellaneous parameters exposed in the form of properties per element, like:

Note, we can use the view's Camera's LODScale to adjust the which level of LOD child is selected for each view.[10] Another common suggestion is to control the update rate of slave cameras by inserting them into the scene graph as needed, and then use a PostDrawCallback for the camera that removes the camera again when it is no longer needed.[11] Robert (OSG) states that most apps should manage the frame for their own applications and data. The OSG is a general purpose scene graph rather than a domain specific IG. It gives you the tools to do your job, but it doesn't do it all for you. The run() method stuff available in OSG/Viewer is really only for entry level app development.[12] More sophisticated types of frame rate management are well beyond what the viewer convenience methods like run are supposed to handle. People should expect to roll your own frame management for this type of app.[13]


Simplifying a lot, this Canvas Element would be an aggregation of:

  • a Compositor instance / effect scheme
  • a View
  • a pointer to a osg::Group representing the scene graph to render (e.g. alias or filename based).

This setup could be replicated in flightgear/src/Viewer/fg_os_osgviewer.cxx, with the difference of ignoring Canvas and using native windowing features from OSG (see Jules' fgcommands to clone a view and open a dedicated GraphicsContext/window).

Related

Forum topics

References

References