Difference between revisions of "Canvas View Camera Element"

From FlightGear wiki
Jump to: navigation, search
m (Known Issues: pointer: https://sourceforge.net/p/flightgear/mailman/message/36925969/)
(32 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{Stub}}
+
{{FGCquote
 +
  |It would make sense to integrate all efforts ([[CompositeViewer Support|CompositeViewer]], [[Canvas]] and [[Compositor]]) to create a [[Canvas_Development#Elements|Canvas Element]] that can render an [[Canvas_Development#Supporting_Cameras|arbitrary view]].
 +
 
 +
Some time ago I wrote what I think are the required steps to get something like cockpit mirrors working: [[#Roadmap]]
 +
 
 +
The main showstopper was having [[CompositeViewer Support]], but since that effort is underway, all the parts are there for someone willing to dive in.
 +
  |{{cite web |url=https://sourceforge.net/p/flightgear/mailman/message/37089780/
 +
    |title=<nowiki>Independent view windows</nowiki>
 +
    |author=<nowiki>Fernando García Liñán </nowiki>
 +
    |date=<nowiki>2020-08-20 14:27:58</nowiki>
 +
  }}
 +
}}
  
 
{{infobox subsystem
 
{{infobox subsystem
|image      = Canvas-view-element-prototype-by-icecode gl.png
 
 
|name        = Canvas Camera Views
 
|name        = Canvas Camera Views
|started    = 08/2016 (prototyped by F-JJTH)
+
|description = Rendering arbitrary views to a texture/Canvas
|description = adds support for rendering slave scenery views to a FBO/RTT (offscreen-texture)
+
|status      = planned/prototype
|status      = experimental/known issues (being prepared for review/integration)
+
 
|maintainers = none
 
|maintainers = none
|developers  = F-JJTH, Icecode GL, Hooray, cyrfer <ref>{{cite web  |url    =  https://forum.flightgear.org/viewtopic.php?p=320012#p320012  |title  =  <nowiki> Re: getting started with RTT </nowiki>  |author =  <nowiki> cyrfer </nowiki>  |date  =  Oct 5th, 2017  |added  =  Oct 5th, 2017  |script_version = 0.36  }}</ref><ref>{{cite web  |url    =  https://forum.flightgear.org/viewtopic.php?p=324494#p324494  |title  =  <nowiki> Re: Canvas:View development </nowiki>  |author =  <nowiki> cyrfer </nowiki>  |date  =  Dec 14th, 2017  |added  =  Dec 14th, 2017  |script_version = 0.36  }}</ref>, Stuart (review <ref>{{cite web
+
|developers  = none
  |url    =  https://forum.flightgear.org/viewtopic.php?p=317967#p317967
+
  |title  =  <nowiki> Re: Gear view in cockpit computer </nowiki>
+
  |author =  <nowiki> stuart </nowiki>
+
  |date  =  Sep 1st, 2017
+
  |added  =  Sep 1st, 2017
+
  |script_version = 0.40
+
  }}</ref>)
+
<!--
+
|topic-sg = https://sourceforge.net/u/fgarlin/simgear/
+
-->
+
 
}}
 
}}
  
 +
== Background ==
  
Last updated in 09/2017
+
Several aircraft developers have manifested their interest in being able to render scene views to a texture (RTT) {{Wikipedia|Render target}} and use it inside cockpits as mirrors, external cameras (so called tail cams) and other uses.
  
== Summary ==
+
With the [[Compositor]] framework it's possible to configure custom render pipelines in XML space, which together with Effect schemes allow complete control over how a rendered scene looks. A synthetic terrain can be rendered by assigning an Effect scheme to a <tt>scene</tt> type pass, which textures all terrain with a brown checkerboard pattern and paints the sky in a solid blue color. All other objects are ignored and aren't rendered, saving a lot of computation time. Custom render distances independent from the main LOD settings can also be used to further optimize the performance of the display.
{{Main article|Compositor}}
+
 
+
Several aircraft developers have manifested their interest in being able to render to a texture (RTT) {{Wikipedia|Render target}} and use it inside cockpits as mirrors, external cameras (so called tail cams) and other uses. Effects and shaders developers have also reached a point where ignoring RTT is a waste of resources and a limitating factor when creating new effects. Although this Canvas element is directed mainly towards the first need, it is a first step forward in terms of finally exposing Render To Texture capabilities to non-C++ space too.
+
  
 
== Use Cases ==
 
== Use Cases ==
* Tail Cams
+
* Tail Cams (gear view)
 
* Mirrors
 
* Mirrors
* In-sim view configuration
+
* Dynamic in-sim view configuration
 
* On demand creation of views and windows (e.g. [[FGCamera]] previews)
 
* On demand creation of views and windows (e.g. [[FGCamera]] previews)
 +
* Missile/payload views
 
* Prototyping/testing HUDs or PFDs requiring synthetic terrain to work properly
 
* Prototyping/testing HUDs or PFDs requiring synthetic terrain to work properly
  
== Status ==
+
== Proof of Concept ==
{{See also|View manager}}
+
{{Note|This is inspired by the long standing idea to render camera views to a texture for use as tailcams, gear views etc <ref>https://forum.flightgear.org/viewtopic.php?f=71&t=23929</ref>, it's based on code originally provided by F-JJTH<ref>https://forum.flightgear.org/viewtopic.php?f=71&t=23929#p317448</ref>, and subsequently reworked by Icecode GL and Hooray to integrate it with the Canvas system back in 2017<ref>https://forum.flightgear.org/viewtopic.php?f=47&t=32846&p=318046</ref>
 +
At the time, the main issue was that the new view element needed to be explicitly updated, but that should no longer be the case thanks to James' work on the setPixel() API <ref>https://sourceforge.net/p/flightgear/mailman/message/36926486/</ref> which also means that sc::Image now has a <code>dirtyPixels()</code> API which we can reuse <ref>https://sourceforge.net/p/flightgear/simgear/ci/ad3621e23b4056e1be7e8d9eb8dd4d513455add8/</ref>
 +
Due to the adoption of the [[Compositor]] framework, this will probably need to reworked to be properly integrated.
 +
Furthermore, beginning in mid 2020 Julian Smith has started working on optional [[CompositeViewer Support]] so that independent scene views can be rendered.  }}
  
We have a basic prototype, provided by F-JJTH, who developed the whole thing in mid-2016. There are a few issues, and it's using the legacy approach for implementing OD_gauge based avionics, i.e. not yet using the Canvas system.
+
SimGear:
 +
<syntaxhighlight lang="diff">
 +
diff --git a/simgear/canvas/CanvasSystemAdapter.hxx b/simgear/canvas/CanvasSystemAdapter.hxx
 +
index 43b3f780..0fb62170 100644
 +
--- a/simgear/canvas/CanvasSystemAdapter.hxx
 +
+++ b/simgear/canvas/CanvasSystemAdapter.hxx
 +
@@ -35,11 +35,13 @@ namespace canvas
 +
 +
      virtual ~SystemAdapter() {}
 +
      virtual FontPtr getFont(const std::string& name) const = 0;
 +
-     virtual void addCamera(osg::Camera* camera) const = 0;
 +
+      virtual void addCamera(osg::Camera* camera, bool useSceneData = false) const = 0;
 +
      virtual void removeCamera(osg::Camera* camera) const = 0;
 +
      virtual osg::ref_ptr<osg::Image> getImage(const std::string& path) const = 0;
 +
+      virtual osg::Node* getModel(const std::string& path) const = 0;
 +
      virtual SGSubsystem* getSubsystem(const std::string& name) const = 0;
 +
      virtual HTTP::Client* getHTTPClient() const = 0;
 +
+      virtual osg::Matrix getViewMatrix(int view_number, double dt) const = 0;
 +
  };
 +
 +
} // namespace canvas
 +
diff --git a/simgear/canvas/canvas_fwd.hxx b/simgear/canvas/canvas_fwd.hxx
 +
index 3b1b4646..97e9bd64 100644
 +
--- a/simgear/canvas/canvas_fwd.hxx
 +
+++ b/simgear/canvas/canvas_fwd.hxx
 +
@@ -24,6 +24,7 @@
 +
 +
#include <osg/ref_ptr>
 +
#include <osgText/Font>
 +
+#include <osg/Node>
 +
 +
#include <boost/function.hpp>
 +
#include <boost/shared_ptr.hpp>
 +
diff --git a/simgear/canvas/elements/CMakeLists.txt b/simgear/canvas/elements/CMakeLists.txt
 +
index 2b537c0c..4eb1e955 100644
 +
--- a/simgear/canvas/elements/CMakeLists.txt
 +
+++ b/simgear/canvas/elements/CMakeLists.txt
 +
@@ -7,6 +7,7 @@ set(HEADERS
 +
  CanvasMap.hxx
 +
  CanvasPath.hxx
 +
  CanvasText.hxx
 +
+  CanvasView.hxx
 +
)
 +
 +
set(DETAIL_HEADERS
 +
@@ -20,6 +21,7 @@ set(SOURCES
 +
  CanvasMap.cxx
 +
  CanvasPath.cxx
 +
  CanvasText.cxx
 +
+  CanvasView.cxx
 +
)
 +
 +
simgear_scene_component(canvas-elements canvas/elements "${SOURCES}" "${HEADERS}")
 +
@@ -28,4 +30,4 @@ simgear_component(canvas-elements/detail canvas/elements/detail "" "${DETAIL_HEA
 +
add_boost_test(canvas_element
 +
  SOURCES canvas_element_test.cpp
 +
  LIBRARIES ${TEST_LIBS}
 +
-)
 +
\ No newline at end of file
 +
+)
 +
diff --git a/simgear/canvas/elements/CanvasGroup.cxx b/simgear/canvas/elements/CanvasGroup.cxx
 +
index c753c2e2..6cec072f 100644
 +
--- a/simgear/canvas/elements/CanvasGroup.cxx
 +
+++ b/simgear/canvas/elements/CanvasGroup.cxx
 +
@@ -23,6 +23,7 @@
 +
#include "CanvasMap.hxx"
 +
#include "CanvasPath.hxx"
 +
#include "CanvasText.hxx"
 +
+#include "CanvasView.hxx"
 +
#include <simgear/canvas/CanvasEventVisitor.hxx>
 +
#include <simgear/canvas/events/MouseEvent.hxx>
 +
 +
@@ -66,6 +67,7 @@ namespace canvas
 +
    add<Map  >(_child_factories);
 +
    add<Path >(_child_factories);
 +
    add<Text >(_child_factories);
 +
+    add<View >(_child_factories);
 +
  }
 +
 +
  //----------------------------------------------------------------------------
 +
diff --git a/simgear/canvas/elements/CanvasView.cxx b/simgear/canvas/elements/CanvasView.cxx
 +
new file mode 100644
 +
index 00000000..502e82d7
 +
--- /dev/null
 +
+++ b/simgear/canvas/elements/CanvasView.cxx
 +
@@ -0,0 +1,91 @@
 +
+#include <simgear/canvas/Canvas.hxx>
 +
+#include <simgear/canvas/CanvasMgr.hxx>
 +
+#include <simgear/canvas/CanvasSystemAdapter.hxx>
 +
+
 +
+#include <simgear/scene/util/OsgMath.hxx>
 +
+
 +
+#include "CanvasView.hxx"
 +
+
 +
+#include <iostream>
 +
+#include <osg/io_utils>
 +
+
 +
+namespace simgear
 +
+{
 +
+namespace canvas
 +
+{
 +
+
 +
+const std::string View::TYPE_NAME = "view";
 +
+
 +
+void View::staticInit()
 +
+{
 +
+    if ( isInit<View>() )
 +
+        return;
 +
+}
 +
+
 +
+View::View( const CanvasWeakPtr& canvas,
 +
+              const SGPropertyNode_ptr& node,
 +
+              const Style& parent_style,
 +
+              ElementWeakPtr parent ) :
 +
+    Element(canvas, node, parent_style, parent),
 +
+    _width(512),
 +
+    _height(512)
 +
+{
 +
+    staticInit();
 +
+
 +
+    _texture = new osg::Texture2D;
 +
+    _texture->setDataVariance(osg::Object::DYNAMIC);
 +
+    _texture->setTextureSize(_width, _height);
 +
+    _texture->setInternalFormat(GL_RGBA);
 +
+    _texture->setFilter(osg::Texture2D::MIN_FILTER, osg::Texture2D::LINEAR);
 +
+    _texture->setFilter(osg::Texture2D::MAG_FILTER, osg::Texture2D::LINEAR);
 +
+
 +
+    _camera = new osg::Camera;
 +
+    _camera->setDataVariance(osg::Object::DYNAMIC);
 +
+    _camera->setViewport(0, 0, _width, _height);
 +
+    _camera->setClearColor(osg::Vec4(1.0f, 1.0f, 1.0f, 1.0f));
 +
+    _camera->setClearMask(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
 +
+    _camera->setRenderOrder(osg::Camera::PRE_RENDER);
 +
+    _camera->setRenderTargetImplementation(osg::Camera::FRAME_BUFFER_OBJECT, osg::Camera::FRAME_BUFFER);
 +
+    _camera->setReferenceFrame(osg::Transform::ABSOLUTE_RF);
 +
+    double aspect_ratio = (double)_width / _height;
 +
+    _camera->setProjectionMatrixAsPerspective(85.0, aspect_ratio, 0.1, 120000.0);
 +
+
 +
+    _camera->attach(osg::Camera::COLOR_BUFFER, _texture.get());
 +
+
 +
+    if (Canvas::getSystemAdapter())
 +
+        Canvas::getSystemAdapter()->addCamera(_camera.get(), true);
 +
+
 +
+    _quad = osg::createTexturedQuadGeometry(osg::Vec3(0.0,    0.0,    0.0),
 +
+                                            osg::Vec3(_width, 0.0,    0.0),
 +
+                                            osg::Vec3(0.0,    _height, 0.0),
 +
+                                            // In Canvas (0,0) is the top left corner, while in OSG/OpenGL
 +
+                                            // it's the bottom left corner.
 +
+                                            0.0, 1.0, 1.0, 0.0);
 +
+    _quad->getOrCreateStateSet()->setTextureAttributeAndModes(0, _texture.get(), osg::StateAttribute::ON);
 +
+
 +
+    setDrawable(_quad.get());
 +
+}
 +
+
 +
+View::~View()
 +
+{
 +
+    if( _camera.valid() && Canvas::getSystemAdapter() )
 +
+        Canvas::getSystemAdapter()->removeCamera(_camera.get());
 +
+    _camera.release();
 +
+    _texture.release();
 +
+}
 +
+
 +
+void View::update(double dt)
 +
+{
 +
+    osg::Matrix view_matrix = Canvas::getSystemAdapter()->getViewMatrix(0, dt);
 +
+    _camera->setViewMatrix(view_matrix);
 +
+
 +
+    Element::update(dt);
 +
+}
 +
+
 +
+void View::resize(int width, int height)
 +
+{
 +
+
 +
+}
 +
+
 +
+} // namespace canvas
 +
+} // namespace simgear
 +
diff --git a/simgear/canvas/elements/CanvasView.hxx b/simgear/canvas/elements/CanvasView.hxx
 +
new file mode 100644
 +
index 00000000..8cb3cc49
 +
--- /dev/null
 +
+++ b/simgear/canvas/elements/CanvasView.hxx
 +
@@ -0,0 +1,39 @@
 +
+#ifndef CANVAS_VIEW_HXX_
 +
+#define CANVAS_VIEW_HXX_
 +
+
 +
+#include "CanvasElement.hxx"
 +
+
 +
+namespace simgear
 +
+{
 +
+namespace canvas
 +
+{
 +
+
 +
+class View : public Element {
 +
+public:
 +
+    static const std::string TYPE_NAME;
 +
+    static void staticInit();
 +
+
 +
+    View( const CanvasWeakPtr& canvas,
 +
+          const SGPropertyNode_ptr& node,
 +
+          const Style& parent_style = Style(),
 +
+          ElementWeakPtr parent = 0 );
 +
+    virtual ~View();
 +
+
 +
+    virtual void update(double dt);
 +
+
 +
+    virtual void resize(int width, int height);
 +
+
 +
+protected:
 +
+
 +
+    osg::ref_ptr<osg::Camera> _camera;
 +
+    osg::ref_ptr<osg::Texture2D> _texture;
 +
+
 +
+    osg::ref_ptr<osg::Geometry> _quad;
 +
+
 +
+    int _width, _height;
 +
+};
 +
+
 +
+} // namespace canvas
 +
+} // namespace simgear
 +
+
 +
+#endif // CANVAS_VIEW_HXX_
 +
</syntaxhighlight>
  
We also have a quick, but working, integration in the form of a custom Canvas element provided by Icecode GL. Originally, the image was mirrored because of different coordinate systems. Apparently the missing piece was to use FG's view manager, which is now fixed, thanks to Clement's contribution.
+
FlightGear
 +
<syntaxhighlight lang="diff">   
 +
diff --git a/src/Canvas/FGCanvasSystemAdapter.cxx b/src/Canvas/FGCanvasSystemAdapter.cxx
 +
index 72b20747e..9e04dbd14 100644
 +
--- a/src/Canvas/FGCanvasSystemAdapter.cxx
 +
+++ b/src/Canvas/FGCanvasSystemAdapter.cxx
 +
@@ -23,9 +23,18 @@
 +
#include <Network/HTTPClient.hxx>
 +
#include <Viewer/renderer.hxx>
  
Who knows, with cleaner code and some tweaks we could have a deferred renderer and RTT capabilities in Canvas space, as long as the effect framework can access these RTT contexts, as Hooray proposed multiple times.<ref>{{cite web
+
+
  |url    =  https://forum.flightgear.org/viewtopic.php?p=317743#p317743
+
+#include <Viewer/view.hxx>
  |title  =  <nowiki> Re: Gear view in cockpit computer </nowiki>  
+
+#include <Viewer/viewmgr.hxx>
  |author =  <nowiki> Icecode GL </nowiki>  
+
+#include <simgear/scene/util/OsgMath.hxx>
  |date  =  Aug 28th, 2017
+
+
  |added  =  Aug 28th, 2017
+
+
  |script_version = 0.40
+
+
  }}</ref>
+
#include <osgDB/ReadFile>
 +
#include <stdexcept>
  
Also, Stuart offered to help review any patches and get them committed.  Please ping Stuart when you think the code is worth checking in. It doesn't have to be perfect - Particularly once the current release is out (2017.3). <ref>{{cite web
+
+#include <simgear/scene/model/modellib.hxx>
  |url    =  https://forum.flightgear.org/viewtopic.php?p=317967#p317967
+
+
  |title  =  <nowiki> Re: Gear view in cockpit computer </nowiki>  
+
namespace canvas
  |author =  <nowiki> stuart </nowiki>
+
{
  |date  =  Sep 1st, 2017
+
//----------------------------------------------------------------------------
  |added  =  Sep 1st, 2017
+
@@ -64,9 +73,9 @@ namespace canvas
  |script_version = 0.40
+
}
  }}</ref>
+
  
== The Future ==
+
//----------------------------------------------------------------------------
for now we are emphasizing in implementing camera views in Canvas. We also keep in mind that this might be an important groundwork for what might be RTT support for effects/shaders. For that we obviously need a link between  Canvas and Effects. It's kind of a good thing that both systems are completely separated and don't know of each other, we are given more freedom when it comes to joining them. This link is kind of delicate and has to be well planned out.<ref>{{cite web
+
- void FGCanvasSystemAdapter::addCamera(osg::Camera* camera) const
  |url    = https://forum.flightgear.org/viewtopic.php?p=318059#p318059
+
+ void FGCanvasSystemAdapter::addCamera(osg::Camera* camera, bool useSceneData) const
  |title =  <nowiki> Re: Canvas:View development </nowiki>
+
{
  |author =  <nowiki> Icecode GL </nowiki>  
+
-    globals->get_renderer()->addCamera(camera, false);
  |date  =  Sep 2nd, 2017
+
+    globals->get_renderer()->addCamera(camera, useSceneData);
  |added  =  Sep 2nd, 2017
+
}
  |script_version = 0.40
+
  }}</ref>
+
  
Shaders can recieve many input textures and modern shaders support MRT (Multiple Render Targets) too. It makes sense to come up with some kind of well defined link between Canvas and Effects that allows for inputting/outputting canvases via effects. This would allow the same things that per-element effects would allow but with more flexibility and less overlapping functionality. <ref>{{cite web
+
//----------------------------------------------------------------------------
  |url    =  https://forum.flightgear.org/viewtopic.php?p=318059#p318059
+
@@ -100,6 +109,46 @@ namespace canvas
  |title  =  <nowiki> Re: Canvas:View development </nowiki>
+
return 0;
  |author =  <nowiki> Icecode GL </nowiki>
+
}
  |date  =  Sep 2nd, 2017
+
  |added  =  Sep 2nd, 2017
+
  |script_version = 0.40
+
  }}</ref>
+
  
it would be possible to also come up with an XML-configurable rendering pipeline - in fact, Zan once came up with a modified CameraGroup design that would move the whole hard-coded pipeline to XML space: [[Canvas Development#Supporting Cameras]]<ref>{{cite web
+
+    //----------------------------------------------------------------------------
  |url   =  https://forum.flightgear.org/viewtopic.php?p=275228#p275228
+
+   // From http://wiki.flightgear.org/Howto:Extending_Canvas_to_support_rendering_3D_models#Extending_FGCanvasSystemAdapter
  |title  =  <nowiki> Re: Review of FG on reddit: xpost </nowiki>  
+
+    osg::Node* FGCanvasSystemAdapter::getModel(const std::string& path) const
  |author =  <nowiki> Hooray </nowiki>
+
+    {
  |date  =  Feb 6th, 2016
+
+        const char *model_path = "Models/Geometry/glider.ac";
  |added  =  Feb 6th, 2016
+
+        SGPath p(SGPath::fromUtf8(path));
  |script_version = 0.40
+
+
  }}</ref>
+
+        if( p.isAbsolute() )
 +
+        {
 +
+            SGPath valid_path = fgValidatePath(p, false);
 +
+            if( !valid_path.isNull() )
 +
+            try {
 +
+                std::string fullPath = simgear::SGModelLib::findDataFile(valid_path.local8BitStr());
 +
+                osg::Node * object = simgear::SGModelLib::loadDeferredModel(fullPath, globals->get_props());
 +
+                return object;
 +
+            } catch (const sg_throwable& t) {
 +
+                SG_LOG(SG_IO, SG_ALERT, "Error loading " << model_path << ":\n " << t.getFormattedMessage() << t.getOrigin());
 +
+                return 0;
 +
+            } // error loading from absolute path
 +
+            SG_LOG(SG_IO, SG_ALERT, "canvas::Model: reading '" << path << "' denied");
 +
+        } // absolute path handling
 +
+        else
 +
+        {
 +
+            SGPath tpath = globals->resolve_resource_path(path);
 +
+            if( !tpath.isNull() )
 +
+            try {
 +
+                std::string fullPath = simgear::SGModelLib::findDataFile(path.c_str());
 +
+                osg::Node * object = simgear::SGModelLib::loadDeferredModel(fullPath, globals->get_props());
 +
+                return object;
 +
+            } catch (const sg_throwable& t) {
 +
+                SG_LOG(SG_IO, SG_ALERT, "Error loading " << model_path << ":\n " << t.getFormattedMessage() << t.getOrigin());
 +
+                return 0;
 +
+            } // error loading from relative path
 +
+
 +
+            SG_LOG(SG_IO, SG_ALERT, "canvas::Model: No such model: '" << path << "'");
 +
+        } // relative path handling
 +
+
 +
+        return 0;
 +
+    }
 +
+
 +
//----------------------------------------------------------------------------
 +
SGSubsystem*
 +
FGCanvasSystemAdapter::getSubsystem(const std::string& name) const
 +
@@ -121,4 +170,16 @@ namespace canvas
 +
return 0;
 +
}
  
Ideally, something like this would be integrated with the existing view manager, i.e. using the same property names (via property objects), and then hooked up to CanvasImage, e.g. as a custom '''camera://''' protocol (we already support canvas:// and http(s)://)
+
+  osg::Matrix FGCanvasSystemAdapter::getViewMatrix(int view_number, double dt) const
So some kind of dedicated CanvasCamera element would make sense, possibly inheriting from CanvasImage.
+
+  {
 +
+        flightgear::View *view = globals->get_viewmgr()->get_view(view_number);
 +
+        view->update(dt);
 +
+
 +
+        osg::Vec3f position( toOsg(view->getViewPosition()) );
 +
+        osg::Quat orientation( toOsg(view->getViewOrientation()) );
 +
+        osg::Matrix viewMatrix( osg::Matrix::translate(-position) * osg::Matrix::rotate(orientation.inverse()) );
 +
+
 +
+        return viewMatrix;
 +
+  }
 +
+
 +
}
 +
diff --git a/src/Canvas/FGCanvasSystemAdapter.hxx b/src/Canvas/FGCanvasSystemAdapter.hxx
 +
index 4c1fd6210..24ececb41 100644
 +
--- a/src/Canvas/FGCanvasSystemAdapter.hxx
 +
+++ b/src/Canvas/FGCanvasSystemAdapter.hxx
 +
@@ -28,11 +28,13 @@ namespace canvas
 +
{
 +
public:
 +
virtual simgear::canvas::FontPtr getFont(const std::string& name) const;
 +
-      virtual void addCamera(osg::Camera* camera) const;
 +
+      virtual void addCamera(osg::Camera* camera, bool useSceneData = false) const;
 +
virtual void removeCamera(osg::Camera* camera) const;
 +
virtual osg::ref_ptr<osg::Image> getImage(const std::string& path) const;
 +
+      virtual osg::Node* getModel(const std::string& path) const;
 +
virtual SGSubsystem* getSubsystem(const std::string& name) const;
 +
virtual simgear::HTTP::Client* getHTTPClient() const;
 +
+      virtual osg::Matrix getViewMatrix(int view_number, double dt) const;
 +
};
 +
}
  
And it would also make sense to look at Zan's new-cameras patches, because those add tons of features to CameraGroup.cxx
+
diff --git a/src/Viewer/renderer.cxx b/src/Viewer/renderer.cxx
This would already allow arbitrary views slaved to the main view (camera)  
+
index d9e8f351f..b9b3fc6a0 100644
 +
--- a/src/Viewer/renderer.cxx
 +
+++ b/src/Viewer/renderer.cxx
 +
@@ -1845,6 +1845,8 @@ FGRenderer::setEventHandler(FGEventHandler* eventHandler_)
 +
void
 +
FGRenderer::addCamera(osg::Camera* camera, bool useSceneData)
 +
{
 +
+    if (useSceneData)
 +
+        camera->addChild(globals->get_scenery()->get_scene_graph());
 +
_viewerSceneRoot->addChild(camera);
 +
}
  
<ref>{{cite web
 
  |url    =  https://forum.flightgear.org/viewtopic.php?p=261665#p261665
 
  |title  =  <nowiki> Re: WINDOW IN WINDOW </nowiki>
 
  |author =  <nowiki> Hooray </nowiki>
 
  |date  =  Oct 25th, 2015
 
  |added  =  Oct 25th, 2015
 
  |script_version = 0.40
 
  }}</ref>
 
  
As has been said previously, the proper way to support "cameras" via Canvas is using CompositeViewer, which does require a re-architecting of several parts of FG: [[CompositeViewer Support]]
+
</syntaxhighlight>
Given the current state of things, that seems at least another 3-4 release cycles away.
+
So, short of that, the only thing that we can currently support with reasonable effort is "slaved views" (as per $FG_ROOT/Docs/README.multiscreen).
+
That would not require too much in terms of coding, because the code is already there - in fact, CameraGroup.cxx already contains a RTT/FBO (render-to-texture) implementation that renders slaved views to an offscreen context. This is also how Rembrandt buffers are set up behind the scenes.
+
So basically, the code is there, it would need to be extracted/genralied and turned into a CanvasElement, and possibly integrated with the existing view manager code.
+
And then, there also is Zan's newcameras branch, which exposes rendering stages (passes) to XML/property tree space, so that individual stages are made accessible to shaders/effects.
+
Thus, most of the code is there, it is mainly a matter of integrating things, i.e. that would require someone able to build SG/FG from source, familiar with C++ and willing/able to work through some OSG tutorials/docs to make this work<ref>{{cite web
+
  |url    =  https://forum.flightgear.org/viewtopic.php?p=260810#p260810
+
  |title  =  <nowiki> Re: WINDOW IN WINDOW </nowiki>
+
  |author =  <nowiki> Hooray </nowiki>
+
  |date  =  Oct 17th, 2015
+
  |added  =  Oct 17th, 2015
+
  |script_version = 0.40
+
  }}</ref>
+
  
Canvas is/was primarily about exposing 2D rendering to fgdata space, so that fgdata developers could incorporatedevelop and maintain 2D rendering related features without having to be core developers (core development being an obvious bottleneck, as well as having  significant barrier to entry).
+
fgdata
In other words, people would need to be convinced that they want to let Canvas evolve beyond the 2D use-case, i.e. by allowing effects/shaders per element, but also to let Cameras be created/controlled easily.
+
<syntaxhighlight lang="diff">
Personally, I do believe that this is a worthwhile thing to aim for, as it would help unify (and simplify) most RTT/FBO handling in SG/FG, and make this available to people like Thorsten who have a track record of doing really fancy, unprecedented stuff, with this flexibility.
+
Equally, there are tons of use-cases where aircraft/scenery developers may want to set up custom cameras (A380 tail cam, space shuttle) and render those to an offscreen texture (e.g. GUI dialog and/or MFD screen).
+
It is true that "slaved views" are kinda limited at the moment, but they are also comparatively easy to set up, so I think that supporting slaved camera views via Canvas could be a good way to bootstrap/boost this development and pave the way for CompositeViewer adoption/integration in the future. <ref>{{cite web
+
  |url    =  https://forum.flightgear.org/viewtopic.php?p=260810#p260810
+
  |title  =  <nowiki> Re: WINDOW IN WINDOW </nowiki>
+
  |author =  <nowiki> Hooray </nowiki>
+
  |date  =  Oct 17th, 2015
+
  |added  =  Oct 17th, 2015
+
  |script_version = 0.40
+
  }}</ref>
+
 
+
== Gallery ==
+
 
+
{|style="margin: 0 auto;"
+
| [[File:Early-canvas-view-screenshot.png|thumb|Screenshot showing early experiments rendering a model to a Canvas view]]
+
| [[File:Icecode GL-Canvas-Camera-support.png|thumb|upright|An experimental [[Canvas]] element to render slaved scenery views to a custom Canvas texture, to be used for creating custom tail-cams/mirror textures and so on]]
+
| [[File:Icecodes slave cams 09-2017.png|thumb|upright|Several Canvas dialogs with view manager views]]
+
| [[File:Canvas-view-element-prototype-by-icecode gl.png|thumb|upright|[[Canvas]] gui dialog with a custom canvas element to display view manager views (based on code prototyped by F-JJTH)]]
+
|}
+
 
+
== Getting involved ==
+
Icecode GL has been quite interested in this subject lately. [...] For shader support it'd use the current effects framework, and to display stuff on the screen it'd either use Canvas or a OSG window directly.If you are interested feel free to contact him so he can share with you some code and pointers/ideas that he's been collecting along the way. I guess that'd be better than starting from zero.<ref>{{cite web  |url    =  https://forum.flightgear.org/viewtopic.php?p=320025#p320025  |title  =  <nowiki> Re: getting started with RTT </nowiki>  |author =  <nowiki> Icecode GL </nowiki>  |date  =  Oct 5th, 2017  |added  =  Oct 5th, 2017  |script_version = 0.36  }}</ref>
+
 
+
The Canvas system is currently constrained to being 2D specific, i.e. to be useful for any of the "3d scene" stuff you're interested in, you'd need to read up on adding new/custom elements - e.g. those useful for 3d stuff, and effects/shaders specifically.As Icecode GL mentioned, this is something that he's been tinkering with lately. Like he also mentioned, based on Zan's groundwork (newcameras), we are looking at exposing a similar degree of flexibility via the Canvas system by introducing new elements and/or additional "modes".For starters, this will probably only involve a new view-manager based Canvas view to render a slave scenery view to a Canvas texture. The next incarnation may include effects/shader support to customie such a slave view.Besides, it would also be possible to render a totally independent scene/osg::Node - which is something that we once prototyped to load 3D models from disk and rotate/transform those using an osg::PositionAttitudeTransformMatrix
+
[[Howto:Extending Canvas to support rendering 3D models]]
+
 
+
[[File:Canvas-model-element-rotated.png|right|250px]] <ref>{{cite web  |url    =  https://forum.flightgear.org/viewtopic.php?p=320105#p320105  |title  =  <nowiki> Re: getting started with RTT </nowiki>  |author =  <nowiki> Hooray </nowiki>  |date  =  Oct 7th, 2017  |added  =  Oct 7th, 2017  |script_version = 0.36  }}</ref>
+
 
+
== Implementation details / for the reviewer ==
+
 
+
{{Note|This section provides a rough overview for the potentiaal reviewer, which is also intended to be used as the commit message when/if this should get committed}}
+
 
+
This set of patches (touching SimGear and fgdata) implements a new <code>Canvas::Element</code> by creating a sub-class named <code>Canvas::View</code>. The meat of it is in the constructor, i.e. <code>Canvas::View::View()</code>, where an off-screen camera (RTT/FBO) is set up, the FGCanvasSystemAdapter file has been extended to provide access to the FlightGear view manager to compute/obtain the view-specific view matrix, which is then used by this new canvas view element to update the offscreen camera in Canvas::View::update() accordingly.
+
 
+
BTW: This is also a good way to stress-test the renderer, as new cameras can be easily added to the scene at runtime, so that the impact of doing so can be easily measured.
+
 
+
the patch is experimental, it will basically look up a view and dynamically add a slave camera to the renderer that renders the whole thing to a Canvas, a Canvas is a fancy word for a RTT/FBO context in FlightGear that can be updated by using a property-based API built on top of the property tree in the form of events/signals that are represented via listeners.Which is to say each Canvas has a handful of well-defined property names (and types) that it is watching to handle "events" - think stuff like changing the sie/view port etc. And then there is a single top-level root group, which serves as the top-level element to keep other Canvas elements.A Canvas element is nothing more than a rendering primitive that the Canvas system can handle - e.g. stuff like a raster image can be added to a Canvas group, a text string/font, and 2D drawing primitives in the form of OpenVG instrutions mapped to ShivaVG. And that's basically about it (with a few exceptions that handle use-case specific stuff like 2D mapping/charts).Apart from that, the main thing to keep in mind is that a Canvas is really just a FBO - i.e. an invisible RTT context - to become actually visible, you need to add a so called "placement" - this tells the rendering engine to look up a certain canvas and add it to the scene/cockpit or the GUI (dialogs/windows).So far, all of this is handled using native code that watches the global /canvas tree in the property tree - there is a canvas manager that handles events and passes them onto the corresponding canvas instance and its child elements.Realistically, all Canvas textures are however instantiated/updated using scripting space hooks that end up writing to the corresponding properties in the global property tree, this makes it much easier to manipulate a canvas/element, because you don't need to do any low-level getprop/setprop stuff, but can directly use an element specific API.<ref>{{cite web  |url    =  https://forum.flightgear.org/viewtopic.php?p=320105#p320105  |title  =  <nowiki> Re: getting started with RTT </nowiki>  |author =  <nowiki> Hooray </nowiki>  |date  =  Oct 7th, 2017  |added  =  Oct 7th, 2017  |script_version = 0.36  }}</ref>
+
 
+
 
+
To actually test the new element using the [[Nasal Console]], paste the following snippet of code into it:
+
 
+
<syntaxhighlight lang="nasal">
+
var (width,height) = (512,512);
+
var ELEMENT_NAME = "viewcam";
+
 
+
var display_view = func(view=0) {
+
var title = 'Canvas test:' ~ ELEMENT_NAME;
+
var window = canvas.Window.new([width,height],"dialog").set('title',title);
+
var myCanvas = window.createCanvas().set("background", canvas.style.getColor("bg_color"));
+
var root = myCanvas.createGroup();
+
 
+
var child = root.createChild( ELEMENT_NAME );
+
# child.set("view-number", view);
+
} # display_view()
+
 
+
 
+
var totalViews = props.getNode("sim").getChildren("view");
+
forindex(var v;totalViews) {
+
display_view(view: v);
+
}</syntaxhighlight>
+
 
+
Aircraft developers can use the new Canvas '''view''' element simply by using the conventional approach to replace a static texture with a Canvas using a cockpit placement.
+
 
+
== Roadmap ==
+
{{PatchAvailable|url=https://forum.flightgear.org/viewtopic.php?f=71&t=23929#p317448}}
+
 
+
 
+
* extend FGCanvasSystemAdapter to make the view manager available there <ref>https://forum.flightgear.org/viewtopic.php?f=71&t=23929&p=317798#p317798</ref>
+
* come up with a new Canvas element inheriting from Canvas::Image {{Progressbar|80}}
+
* add Clement's camera setup routines
+
* make the camera render into the texture used by the sub-class
+
* this should be named viewmgr-camera or something like that to make it obvious what it is doing
+
 
+
== Testing ==
+
Once we have a basic prototype working, the new camera element needs to be tested, specifically:
+
* using different resolutions (resizing at runtime?)
+
* displaying multiple/independent camera views
+
* all supported Canvas placements
+
* shaders and effects
+
* ALS
+
* Rembrandt
+
* Reset/re-init
+
* OSG threading modes
+
 
+
== Known Issues ==
+
* texture/view matrix updates don't currently take effect unless .update() is specifically invoked, see [https://sourceforge.net/p/flightgear/mailman/message/36925969/]
+
* <del>skydome handling (probably missing effect/shader or wrong root node)</del> {{Done}} (by Icecode_GL)
+
* change DATA_VARIANCE for texture setup, analogous to the Canvas::Image::Image ctor
+
* make the size of the texture re-configurable using properties
+
 
+
== Troubleshooting ==
+
* update the texture specific osg::StateSet to force an update of the texture
+
* compare the scenegraph generated for sc::View vs. sc::Image using http://wiki.flightgear.org/Canvas_Troubleshooting#Dumping_Canvas_scene_graphs_to_disk
+
* check whether sub-classing sc::Image and  directly using its _texture makes any difference or not
+
** if it does, we could just as well add support for a custom <code>view://by-number/</code> protocol to the src/filename handling helper
+
* if all else fails, use an osg::Image and assign it to the texture and invoke its dirty() method to update the whole thing
+
 
+
== Ideas ==
+
{{See also|Canvas_Troubleshooting#Adding_draw_masks_for_Canvas}}
+
* do we need support for looking up view numbers by name ?
+
* support draw-masks for scene graph features (terrain, skydome, models etc): " it'd be benefitial in the long run to let the user choose what part of the scene graph they want to show. This would be useful for deferred rendering schemes, where sometimes you don't need to display the whole scenery to save some precious frames. Canvas makes that easy with all of its property handling"
+
* work out what is needed for [https://forum.flightgear.org/viewtopic.php?f=71&t=27985&p=264718&hilit=synthetic+terrain#p264718v synthetic terrain views] [http://www.aircraftspruce.eu/catalog/graphics/10-03932s.jpg]
+
* support effects/shaders: This is fairly straightforward to do, just create a EffectsGeode, fill it with a "compiled" .eff, and substitute the standard osg::Geode for it. But I don't think that's going to give us the flexibility we need. A shader won't be able to access several canvases at the same time if we do things that way.
+
* introduce some sort of STL cache using something like std::map<std::string, osg::Texture2D> to only ever update/draw a view once, no matter how often it is shown by different elements/placements ?
+
 
+
== Performance / Optimizations ==
+
{{See also|Canvas_Troubleshooting#Hooking_up_Canvas_and_Osg_Stats}}
+
* Configurable refresh rate per view. 15 fps might be enough for a external camera, we don't need to draw at max rate.<ref>{{cite web
+
  |url    =  https://forum.flightgear.org/viewtopic.php?p=318028#p318028
+
  |title  =  <nowiki> Re: Canvas::View element: performance/optimizations </nowiki>
+
  |author =  <nowiki> Icecode GL </nowiki>
+
  |date  =  Sep 2nd, 2017
+
  |added  =  Sep 2nd, 2017
+
  |script_version = 0.40
+
  }}</ref>
+
* provide a view-cache using a STL std::map<> ?
+
* use PBOs [http://forum.openscenegraph.org/viewtopic.php?t=10713&view=next]
+
* consider hooking up the whole thing to the FGStatsManager (OSG stats) ?
+
* Torsten implemented some [[PagedLOD]] scheme to only render AI entities/objects if they're actually visible in terms of pixels on the screen <ref>{{cite web
+
  |url    =  https://sourceforge.net/p/flightgear/mailman/message/33057570/
+
  |title  =  <nowiki> [Flightgear-devel] Improved LOD handling for AI Models </nowiki>
+
  |author =  <nowiki> Torsten Dreyer </nowiki>
+
  |date  =  Nov 19th, 2014
+
  |added  =  Nov 19th, 2014
+
  |script_version = 0.40
+
  }}</ref> <ref>{{cite web
+
  |url    =  https://sourceforge.net/p/flightgear/mailman/message/33271362/
+
  |title  =  <nowiki> Re: [Flightgear-devel] Flightgear next (3.5.0) Does not render MP
+
aircraft </nowiki>
+
  |author =  <nowiki> Torsten Dreyer </nowiki>
+
  |date  =  Jan 26th, 2015
+
  |added  =  Jan 26th, 2015
+
  |script_version = 0.40
+
  }}</ref>
+
* check if '''CullThreadPerCameraDrawThreadPerContext''' works, see [[Howto:Activate multi core and multi GPU support]]
+
 
+
== Base Package ==
+
{{collapsible script
+
| type  = view-camera
+
| title  = view-camera  $FG_ROOT/Nasal/canvas/api.nas changes
+
| intro = Finally, we need to modify api.nas to provide Nasal wrappers
+
| script =
+
 
diff --git a/Nasal/canvas/api.nas b/Nasal/canvas/api.nas
 
diff --git a/Nasal/canvas/api.nas b/Nasal/canvas/api.nas
index 6d39d03..f98fdd9 100644
+
index 3f7f5a17..433a41f9 100644
 
--- a/Nasal/canvas/api.nas
 
--- a/Nasal/canvas/api.nas
 
+++ b/Nasal/canvas/api.nas
 
+++ b/Nasal/canvas/api.nas
@@ -1099,13 +1099,21 @@ var Image = {
+
@@ -1132,10 +1132,10 @@ var Image = {
 
   }
 
   }
 
  };
 
  };
 
   
 
   
+var viewCamera = {
+
-var Model = {
+ new: func(ghost)
+
+var View = {
{
+
  new: func(ghost)
+   return {parents: [viewCamera, Element.new(ghost)]};
+
  {
+ },
+
-   return {parents: [Model, Element.new(ghost)]};
+};
+
+   return {parents: [View, Element.new(ghost)]};
+
+
  },
  # Element factories used by #Group elements to create children
+
  };
  Group._element_factories = {
+
   
  "group": Group.new,
+
@@ -1146,7 +1146,7 @@ Group._element_factories = {
  "map": Map.new,
+
 
   "text": Text.new,
 
   "text": Text.new,
 
   "path": Path.new,
 
   "path": Path.new,
"image": Image.new
+
  "image": Image.new,
+ "image": Image.new,
+
- "model": Model.new
+  "view-camera": viewCamera.new,
+
+  "view": View.new
 
  };
 
  };
| lang = diff
+
}}
+
# Canvas
 +
</syntaxhighlight>
  
== Nasal Console ==
+
== Roadmap ==
  
After patching and rebuilding SimGear/FlightGear respectively, and applying the changes to to api.nas in the base package, the following can be pasted into the [[Nasal Console]] for testing purposes:
+
=== Use CompositeViewer {{Pending}} ===
 +
{{See also|CompositeViewer Support}}
  
<syntaxhighlight lang="nasal">
+
Currently FlightGear uses only one instance of <tt>osg::Viewer</tt>, which is used by CameraGroup to manage the slave cameras. Supporting CompositeViewer would require modifying {{flightgear file|src/Viewer/fg_os_osgviewer.cxx}} and creating some kind of wrapper class that manages the CompositeViewer instance and assigns a CameraGroup to each <tt>osg::View</tt>. It's also important to note that currently all FG subsystems assume there is a single instance of [[Howto:CameraGroup talks|CameraGroup]].
# TODO: this must match the TYPE_NAME used by the C++ code
+
var ELEMENT_NAME ="view-camera"; # to be adapted according to the C++ changes
+
  
var myElementTest = {
+
=== Standardize the [[View manager]] {{Pending}} ===
##
+
# constructor
+
new: func( camera ) {
+
  var m = { parents: [myElementTest] };
+
  m.dlg = canvas.Window.new([camera.width,camera.height],"dialog");
+
  m.canvas = m.dlg.createCanvas().setColorBackground(1,1,1,1);
+
  m.root = m.canvas.createGroup();
+
  
  ##
+
The first step would be to port the view code to SimGear so it can be used and known by the Compositor and Canvas. The view manager ({{flightgear file|src/Viewer/viewmgr.cxx}}) currently has some hardcoded assumptions, so it would either need to be rewritten to remove them or a new interface for the Views could be created specifically for the Canvas Camera View.
  # instantiate a new element
+
  m.myElement = m.root.createChild( ELEMENT_NAME );
+
  # set the view-number property
+
  m.myElement.set("view-number", camera.view);
+
  m.dlg.set("title", "view #"~camera.view); # TODO look up proper title/name
+
  
  return m;
+
In FlightGear, the Canvas system (which resides in SimGear) is integrated using the equivalent of a FGCanvasSystemAdapter, which provides all FG APIs to SimGear and makes the Canvas system available inside FG: {{flightgear file|src/Canvas/FGCanvasSystemAdapter.cxx}}
}, # new
+
+
}; # end of myElementTest
+
  
# TODO: get list of all views ?
+
Instead of using tied properties, the adapted view code would either used propertyObject<> or the propertyBasedMgr abstraction, so that the corresponding canvas element can continue to use well-known conventions to manipulate views: ({{flightgear file|src/Viewer/view.cxx}})
  
var cameras = [
+
=== Create a Canvas Element subclass {{Pending}} ===
{view: 0, width : 640, height: 480},
+
{{Main article|Canvas_Development#Adding_a_new_Element}}
{view: 1, width : 320, height: 160},
+
{view: 2, width : 320, height: 160},
+
];
+
  
foreach(var cam; cameras) {
+
This derived class would require the following configuration:
var newCam = myElementTest.new( cam );
+
* [[Compositor]] to use.
}
+
* [[View manager|View]] to use. This could be one of the "main" ones (i.e. the ones on the main property tree), or a locally-defined one that is only known to this Canvas Element (think FLIR).
</syntaxhighlight>
+
* Scene graph to use. By default the main scene graph would be used, but an arbitrary XML file can be loaded to [[Howto:Extending Canvas to support rendering 3D models|render a custom model]]. A typical use-case would be instruments that need to [[Shuttle ADI ball|manipulate/render a 3D object]] (for which we have working code, too)
 +
* Several optimization/miscellaneous parameters like framerate cap, etc.
 +
 
 +
Simplifying a lot, this Canvas Element would be an aggregation of a Compositor instance, a View and a pointer to a <tt>osg::Group</tt> representing the scene graph to render. This setup could be replicated in {{flightgear file|src/Viewer/fg_os_osgviewer.cxx}}, with the difference of ignoring Canvas and using native windowing features from OSG.
  
 
== Related ==
 
== Related ==
{{Appendix}}
+
 
 +
=== Forum topics ===
 +
* {{forum link|t=32846|text=Canvas::View Development}}
 +
* {{forum link|t=23929|text=Gear View in Cockpit}}
 +
* {{forum link|t=20057|text=Rear-View mirror}}
 +
* {{forum link|t=18905|text=Progress on synthetic terrain}}
 +
* {{forum link|t=17184|text=Instruments with heightmaps}}
 +
 
 +
[[Category:Canvas Element Proposals]]

Revision as of 12:49, 23 August 2020

Cquote1.png It would make sense to integrate all efforts (CompositeViewer, Canvas and Compositor) to create a Canvas Element that can render an arbitrary view.

Some time ago I wrote what I think are the required steps to get something like cockpit mirrors working: #Roadmap

The main showstopper was having CompositeViewer Support, but since that effort is underway, all the parts are there for someone willing to dive in.


— Fernando García Liñán  (2020-08-20 14:27:58). Independent view windows.
(powered by Instant-Cquotes)
Cquote2.png
Canvas Camera Views
Description Rendering arbitrary views to a texture/Canvas
Maintainer(s) none
Contributor(s) none
Status planned/prototype

Background

Several aircraft developers have manifested their interest in being able to render scene views to a texture (RTT) Render target This is a link to a Wikipedia article and use it inside cockpits as mirrors, external cameras (so called tail cams) and other uses.

With the Compositor framework it's possible to configure custom render pipelines in XML space, which together with Effect schemes allow complete control over how a rendered scene looks. A synthetic terrain can be rendered by assigning an Effect scheme to a scene type pass, which textures all terrain with a brown checkerboard pattern and paints the sky in a solid blue color. All other objects are ignored and aren't rendered, saving a lot of computation time. Custom render distances independent from the main LOD settings can also be used to further optimize the performance of the display.

Use Cases

  • Tail Cams (gear view)
  • Mirrors
  • Dynamic in-sim view configuration
  • On demand creation of views and windows (e.g. FGCamera previews)
  • Missile/payload views
  • Prototyping/testing HUDs or PFDs requiring synthetic terrain to work properly

Proof of Concept

Note  This is inspired by the long standing idea to render camera views to a texture for use as tailcams, gear views etc [1], it's based on code originally provided by F-JJTH[2], and subsequently reworked by Icecode GL and Hooray to integrate it with the Canvas system back in 2017[3]

At the time, the main issue was that the new view element needed to be explicitly updated, but that should no longer be the case thanks to James' work on the setPixel() API [4] which also means that sc::Image now has a dirtyPixels() API which we can reuse [5] Due to the adoption of the Compositor framework, this will probably need to reworked to be properly integrated. Furthermore, beginning in mid 2020 Julian Smith has started working on optional CompositeViewer Support so that independent scene views can be rendered.

SimGear:

diff --git a/simgear/canvas/CanvasSystemAdapter.hxx b/simgear/canvas/CanvasSystemAdapter.hxx
index 43b3f780..0fb62170 100644
--- a/simgear/canvas/CanvasSystemAdapter.hxx
+++ b/simgear/canvas/CanvasSystemAdapter.hxx
@@ -35,11 +35,13 @@ namespace canvas
 
       virtual ~SystemAdapter() {}
       virtual FontPtr getFont(const std::string& name) const = 0;
-      virtual void addCamera(osg::Camera* camera) const = 0;
+      virtual void addCamera(osg::Camera* camera, bool useSceneData = false) const = 0;
       virtual void removeCamera(osg::Camera* camera) const = 0;
       virtual osg::ref_ptr<osg::Image> getImage(const std::string& path) const = 0;
+      virtual osg::Node* getModel(const std::string& path) const = 0;
       virtual SGSubsystem* getSubsystem(const std::string& name) const = 0;
       virtual HTTP::Client* getHTTPClient() const = 0;
+      virtual osg::Matrix getViewMatrix(int view_number, double dt) const = 0;
   };
 
 } // namespace canvas
diff --git a/simgear/canvas/canvas_fwd.hxx b/simgear/canvas/canvas_fwd.hxx
index 3b1b4646..97e9bd64 100644
--- a/simgear/canvas/canvas_fwd.hxx
+++ b/simgear/canvas/canvas_fwd.hxx
@@ -24,6 +24,7 @@
 
 #include <osg/ref_ptr>
 #include <osgText/Font>
+#include <osg/Node>
 
 #include <boost/function.hpp>
 #include <boost/shared_ptr.hpp>
diff --git a/simgear/canvas/elements/CMakeLists.txt b/simgear/canvas/elements/CMakeLists.txt
index 2b537c0c..4eb1e955 100644
--- a/simgear/canvas/elements/CMakeLists.txt
+++ b/simgear/canvas/elements/CMakeLists.txt
@@ -7,6 +7,7 @@ set(HEADERS
   CanvasMap.hxx
   CanvasPath.hxx
   CanvasText.hxx
+  CanvasView.hxx
 )
 
 set(DETAIL_HEADERS
@@ -20,6 +21,7 @@ set(SOURCES
   CanvasMap.cxx
   CanvasPath.cxx
   CanvasText.cxx
+  CanvasView.cxx
 )
 
 simgear_scene_component(canvas-elements canvas/elements "${SOURCES}" "${HEADERS}")
@@ -28,4 +30,4 @@ simgear_component(canvas-elements/detail canvas/elements/detail "" "${DETAIL_HEA
 add_boost_test(canvas_element
   SOURCES canvas_element_test.cpp
   LIBRARIES ${TEST_LIBS}
-)
\ No newline at end of file
+)
diff --git a/simgear/canvas/elements/CanvasGroup.cxx b/simgear/canvas/elements/CanvasGroup.cxx
index c753c2e2..6cec072f 100644
--- a/simgear/canvas/elements/CanvasGroup.cxx
+++ b/simgear/canvas/elements/CanvasGroup.cxx
@@ -23,6 +23,7 @@
 #include "CanvasMap.hxx"
 #include "CanvasPath.hxx"
 #include "CanvasText.hxx"
+#include "CanvasView.hxx"
 #include <simgear/canvas/CanvasEventVisitor.hxx>
 #include <simgear/canvas/events/MouseEvent.hxx>
 
@@ -66,6 +67,7 @@ namespace canvas
     add<Map  >(_child_factories);
     add<Path >(_child_factories);
     add<Text >(_child_factories);
+    add<View >(_child_factories);
   }
 
   //----------------------------------------------------------------------------
diff --git a/simgear/canvas/elements/CanvasView.cxx b/simgear/canvas/elements/CanvasView.cxx
new file mode 100644
index 00000000..502e82d7
--- /dev/null
+++ b/simgear/canvas/elements/CanvasView.cxx
@@ -0,0 +1,91 @@
+#include <simgear/canvas/Canvas.hxx>
+#include <simgear/canvas/CanvasMgr.hxx>
+#include <simgear/canvas/CanvasSystemAdapter.hxx>
+
+#include <simgear/scene/util/OsgMath.hxx>
+
+#include "CanvasView.hxx"
+
+#include <iostream>
+#include <osg/io_utils>
+
+namespace simgear
+{
+namespace canvas
+{
+
+const std::string View::TYPE_NAME = "view";
+
+void View::staticInit()
+{
+    if ( isInit<View>() )
+        return;
+}
+
+View::View( const CanvasWeakPtr& canvas,
+              const SGPropertyNode_ptr& node,
+              const Style& parent_style,
+              ElementWeakPtr parent ) :
+    Element(canvas, node, parent_style, parent),
+    _width(512),
+    _height(512)
+{
+    staticInit();
+
+    _texture = new osg::Texture2D;
+    _texture->setDataVariance(osg::Object::DYNAMIC);
+    _texture->setTextureSize(_width, _height);
+    _texture->setInternalFormat(GL_RGBA);
+    _texture->setFilter(osg::Texture2D::MIN_FILTER, osg::Texture2D::LINEAR);
+    _texture->setFilter(osg::Texture2D::MAG_FILTER, osg::Texture2D::LINEAR);
+
+    _camera = new osg::Camera;
+    _camera->setDataVariance(osg::Object::DYNAMIC);
+    _camera->setViewport(0, 0, _width, _height);
+    _camera->setClearColor(osg::Vec4(1.0f, 1.0f, 1.0f, 1.0f));
+    _camera->setClearMask(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
+    _camera->setRenderOrder(osg::Camera::PRE_RENDER);
+    _camera->setRenderTargetImplementation(osg::Camera::FRAME_BUFFER_OBJECT, osg::Camera::FRAME_BUFFER);
+    _camera->setReferenceFrame(osg::Transform::ABSOLUTE_RF);
+    double aspect_ratio = (double)_width / _height;
+    _camera->setProjectionMatrixAsPerspective(85.0, aspect_ratio, 0.1, 120000.0);
+
+    _camera->attach(osg::Camera::COLOR_BUFFER, _texture.get());
+
+    if (Canvas::getSystemAdapter())
+        Canvas::getSystemAdapter()->addCamera(_camera.get(), true);
+
+    _quad = osg::createTexturedQuadGeometry(osg::Vec3(0.0,    0.0,     0.0),
+                                            osg::Vec3(_width, 0.0,     0.0),
+                                            osg::Vec3(0.0,    _height, 0.0),
+                                            // In Canvas (0,0) is the top left corner, while in OSG/OpenGL
+                                            // it's the bottom left corner.
+                                            0.0, 1.0, 1.0, 0.0);
+    _quad->getOrCreateStateSet()->setTextureAttributeAndModes(0, _texture.get(), osg::StateAttribute::ON);
+
+    setDrawable(_quad.get());
+}
+
+View::~View()
+{
+    if( _camera.valid() && Canvas::getSystemAdapter() )
+        Canvas::getSystemAdapter()->removeCamera(_camera.get());
+    _camera.release();
+    _texture.release();
+}
+
+void View::update(double dt)
+{
+    osg::Matrix view_matrix = Canvas::getSystemAdapter()->getViewMatrix(0, dt);
+    _camera->setViewMatrix(view_matrix);
+
+    Element::update(dt);
+}
+
+void View::resize(int width, int height)
+{
+
+}
+
+} // namespace canvas
+} // namespace simgear
diff --git a/simgear/canvas/elements/CanvasView.hxx b/simgear/canvas/elements/CanvasView.hxx
new file mode 100644
index 00000000..8cb3cc49
--- /dev/null
+++ b/simgear/canvas/elements/CanvasView.hxx
@@ -0,0 +1,39 @@
+#ifndef CANVAS_VIEW_HXX_
+#define CANVAS_VIEW_HXX_
+
+#include "CanvasElement.hxx"
+
+namespace simgear
+{
+namespace canvas
+{
+
+class View : public Element {
+public:
+    static const std::string TYPE_NAME;
+    static void staticInit();
+
+    View( const CanvasWeakPtr& canvas,
+           const SGPropertyNode_ptr& node,
+           const Style& parent_style = Style(),
+           ElementWeakPtr parent = 0 );
+    virtual ~View();
+
+    virtual void update(double dt);
+
+    virtual void resize(int width, int height);
+
+protected:
+
+    osg::ref_ptr<osg::Camera> _camera;
+    osg::ref_ptr<osg::Texture2D> _texture;
+
+    osg::ref_ptr<osg::Geometry> _quad;
+
+    int _width, _height;
+};
+
+} // namespace canvas
+} // namespace simgear
+
+#endif // CANVAS_VIEW_HXX_

FlightGear

    
diff --git a/src/Canvas/FGCanvasSystemAdapter.cxx b/src/Canvas/FGCanvasSystemAdapter.cxx
index 72b20747e..9e04dbd14 100644
--- a/src/Canvas/FGCanvasSystemAdapter.cxx
+++ b/src/Canvas/FGCanvasSystemAdapter.cxx
@@ -23,9 +23,18 @@
#include <Network/HTTPClient.hxx>
#include <Viewer/renderer.hxx>

+
+#include <Viewer/view.hxx>
+#include <Viewer/viewmgr.hxx>
+#include <simgear/scene/util/OsgMath.hxx>
+
+
+
#include <osgDB/ReadFile>
#include <stdexcept>

+#include <simgear/scene/model/modellib.hxx>
+
namespace canvas
{
//----------------------------------------------------------------------------
@@ -64,9 +73,9 @@ namespace canvas
}

//----------------------------------------------------------------------------
-  void FGCanvasSystemAdapter::addCamera(osg::Camera* camera) const
+  void FGCanvasSystemAdapter::addCamera(osg::Camera* camera, bool useSceneData) const
{
-    globals->get_renderer()->addCamera(camera, false);
+    globals->get_renderer()->addCamera(camera, useSceneData);
}

//----------------------------------------------------------------------------
@@ -100,6 +109,46 @@ namespace canvas
return 0;
}

+    //----------------------------------------------------------------------------
+    // From http://wiki.flightgear.org/Howto:Extending_Canvas_to_support_rendering_3D_models#Extending_FGCanvasSystemAdapter
+    osg::Node* FGCanvasSystemAdapter::getModel(const std::string& path) const
+    {
+        const char *model_path = "Models/Geometry/glider.ac";
+        SGPath p(SGPath::fromUtf8(path));
+
+        if( p.isAbsolute() )
+        {
+            SGPath valid_path = fgValidatePath(p, false);
+            if( !valid_path.isNull() )
+            try {
+                std::string fullPath = simgear::SGModelLib::findDataFile(valid_path.local8BitStr());
+                osg::Node * object = simgear::SGModelLib::loadDeferredModel(fullPath, globals->get_props());
+                return object;
+            } catch (const sg_throwable& t) {
+                SG_LOG(SG_IO, SG_ALERT, "Error loading " << model_path << ":\n  " << t.getFormattedMessage() << t.getOrigin());
+                return 0;
+            } // error loading from absolute path
+            SG_LOG(SG_IO, SG_ALERT, "canvas::Model: reading '" << path << "' denied");
+        } // absolute path handling
+        else
+        {
+            SGPath tpath = globals->resolve_resource_path(path);
+            if( !tpath.isNull() )
+            try {
+                std::string fullPath = simgear::SGModelLib::findDataFile(path.c_str());
+                osg::Node * object = simgear::SGModelLib::loadDeferredModel(fullPath, globals->get_props());
+                return object;
+            } catch (const sg_throwable& t) {
+                SG_LOG(SG_IO, SG_ALERT, "Error loading " << model_path << ":\n  " << t.getFormattedMessage() << t.getOrigin());
+                return 0;
+            } // error loading from relative path
+
+            SG_LOG(SG_IO, SG_ALERT, "canvas::Model: No such model: '" << path << "'");
+        } // relative path handling
+
+        return 0;
+    }
+
//----------------------------------------------------------------------------
SGSubsystem*
FGCanvasSystemAdapter::getSubsystem(const std::string& name) const
@@ -121,4 +170,16 @@ namespace canvas
return 0;
}

+  osg::Matrix FGCanvasSystemAdapter::getViewMatrix(int view_number, double dt) const
+  {
+        flightgear::View *view = globals->get_viewmgr()->get_view(view_number);
+        view->update(dt);
+
+        osg::Vec3f position( toOsg(view->getViewPosition()) );
+        osg::Quat orientation( toOsg(view->getViewOrientation()) );
+        osg::Matrix viewMatrix( osg::Matrix::translate(-position) * osg::Matrix::rotate(orientation.inverse()) );
+
+        return viewMatrix;
+  }
+
}
diff --git a/src/Canvas/FGCanvasSystemAdapter.hxx b/src/Canvas/FGCanvasSystemAdapter.hxx
index 4c1fd6210..24ececb41 100644
--- a/src/Canvas/FGCanvasSystemAdapter.hxx
+++ b/src/Canvas/FGCanvasSystemAdapter.hxx
@@ -28,11 +28,13 @@ namespace canvas
{
public:
virtual simgear::canvas::FontPtr getFont(const std::string& name) const;
-      virtual void addCamera(osg::Camera* camera) const;
+      virtual void addCamera(osg::Camera* camera, bool useSceneData = false) const;
virtual void removeCamera(osg::Camera* camera) const;
virtual osg::ref_ptr<osg::Image> getImage(const std::string& path) const;
+      virtual osg::Node* getModel(const std::string& path) const;
virtual SGSubsystem* getSubsystem(const std::string& name) const;
virtual simgear::HTTP::Client* getHTTPClient() const;
+      virtual osg::Matrix getViewMatrix(int view_number, double dt) const;
};
}

diff --git a/src/Viewer/renderer.cxx b/src/Viewer/renderer.cxx
index d9e8f351f..b9b3fc6a0 100644
--- a/src/Viewer/renderer.cxx
+++ b/src/Viewer/renderer.cxx
@@ -1845,6 +1845,8 @@ FGRenderer::setEventHandler(FGEventHandler* eventHandler_)
void
FGRenderer::addCamera(osg::Camera* camera, bool useSceneData)
{
+    if (useSceneData)
+        camera->addChild(globals->get_scenery()->get_scene_graph());
_viewerSceneRoot->addChild(camera);
}

fgdata

diff --git a/Nasal/canvas/api.nas b/Nasal/canvas/api.nas
index 3f7f5a17..433a41f9 100644
--- a/Nasal/canvas/api.nas
+++ b/Nasal/canvas/api.nas
@@ -1132,10 +1132,10 @@ var Image = {
   }
 };
 
-var Model = {
+var View = {
   new: func(ghost)
   {
-    return {parents: [Model, Element.new(ghost)]};
+    return {parents: [View, Element.new(ghost)]};
   },
 };
 
@@ -1146,7 +1146,7 @@ Group._element_factories = {
   "text": Text.new,
   "path": Path.new,
   "image": Image.new,
-  "model": Model.new
+  "view": View.new
 };
 
 # Canvas

Roadmap

Use CompositeViewer Pending Pending

Currently FlightGear uses only one instance of osg::Viewer, which is used by CameraGroup to manage the slave cameras. Supporting CompositeViewer would require modifying flightgear/src/Viewer/fg_os_osgviewer.cxx and creating some kind of wrapper class that manages the CompositeViewer instance and assigns a CameraGroup to each osg::View. It's also important to note that currently all FG subsystems assume there is a single instance of CameraGroup.

Standardize the View manager Pending Pending

The first step would be to port the view code to SimGear so it can be used and known by the Compositor and Canvas. The view manager (flightgear/src/Viewer/viewmgr.cxx) currently has some hardcoded assumptions, so it would either need to be rewritten to remove them or a new interface for the Views could be created specifically for the Canvas Camera View.

In FlightGear, the Canvas system (which resides in SimGear) is integrated using the equivalent of a FGCanvasSystemAdapter, which provides all FG APIs to SimGear and makes the Canvas system available inside FG: flightgear/src/Canvas/FGCanvasSystemAdapter.cxx

Instead of using tied properties, the adapted view code would either used propertyObject<> or the propertyBasedMgr abstraction, so that the corresponding canvas element can continue to use well-known conventions to manipulate views: (flightgear/src/Viewer/view.cxx)

Create a Canvas Element subclass Pending Pending

1rightarrow.png See Canvas_Development#Adding_a_new_Element for the main article about this subject.

This derived class would require the following configuration:

  • Compositor to use.
  • View to use. This could be one of the "main" ones (i.e. the ones on the main property tree), or a locally-defined one that is only known to this Canvas Element (think FLIR).
  • Scene graph to use. By default the main scene graph would be used, but an arbitrary XML file can be loaded to render a custom model. A typical use-case would be instruments that need to manipulate/render a 3D object (for which we have working code, too)
  • Several optimization/miscellaneous parameters like framerate cap, etc.

Simplifying a lot, this Canvas Element would be an aggregation of a Compositor instance, a View and a pointer to a osg::Group representing the scene graph to render. This setup could be replicated in flightgear/src/Viewer/fg_os_osgviewer.cxx, with the difference of ignoring Canvas and using native windowing features from OSG.

Related

Forum topics

  • https://forum.flightgear.org/viewtopic.php?f=71&t=23929
  • https://forum.flightgear.org/viewtopic.php?f=71&t=23929#p317448
  • https://forum.flightgear.org/viewtopic.php?f=47&t=32846&p=318046
  • https://sourceforge.net/p/flightgear/mailman/message/36926486/
  • https://sourceforge.net/p/flightgear/simgear/ci/ad3621e23b4056e1be7e8d9eb8dd4d513455add8/