Howto:Extending Canvas to support rendering 3D models

From FlightGear wiki
Jump to navigation Jump to search
This article describes content/features that may not yet be available in the latest stable version of FlightGear (2020.3).
You may need to install some extra components, use the latest development (Git) version or even rebuild FlightGear from source, possibly from a custom topic branch using special build settings: .
This feature is scheduled for FlightGear 2023.3. 10}% completed

If you'd like to learn more about getting your own ideas into FlightGear, check out Implementing new features for FlightGear.

This article is a stub. You can help the wiki by expanding it.

This howto will demonstrate how the FlightGear Canvas system can be extended to load 3D models from disk and render them to a texture on demand, as well as how to position the object arbitrarily using properties.

Screen shot showing a custom canvas element for loading/displaying arbitrary 3D models
Screen shot showing a custom canvas element for loading/displaying arbitrary 3D models (rotated perspective)
FlightGear logo.png The FlightGear forum has a
subforum related to: Canvas
Canvas - the FlightGear 2D drawing API
RFC
Frameworks
Tutorials
Reference
Related
To be ported
Deprecated by Canvas
(click on headings to toggle)

Background

See also Shuttle ADI ball.

There are several scenarios where we cannot currently use Canvas because we're lacking a means for rendering a 3D model to a texture - omega95 once mentioned that some modern avionics are increasingly using 3D representations[1][2]. Equally, a typical FlightGear GUI front-end will usually require a way for previewing aircraft 3D models.

Currently, we cannot use Canvas for any of these use-cases because there's no way to load 3D models. Given the increasing focus on extending the Canvas-based Aircraft Center, supporting 3D models via Canvas simply makes sense at some point:

Cquote1.png As we move forward with FlightGear development and future versions, we will be expanding the "in app" aircraft center. This dialog inside flightgear lets you select, download, and switch to any of the aircraft in the library.
— Curtis Olson (2015-02-17). Re: [Flightgear-devel] Citation II for base package?.
(powered by Instant-Cquotes)
 Cquote2.png

Approach

  • We'll be extending FGCanvasSystemAdapter to add a new method for loading a 3D model form $FG_ROOT using fgValidatePath() properly
  • Next, we'll be creating a new Canvas Element using the tutorial at Canvas Development#Adding a new Element
  • The new element will serve as a container for an Osg::PositionAttitudeTransform (PAT) for positioning the 3D model
  • as a child node, we'll add an Osg::ProxyNode
  • the new Canvas element will be monitoring the property tree for filename events and update the loaded 3D model accordingly.
  • equally, we'll need to expose a few PAT methods to the property tree, so that the model can be transformed/rotated by updating a few properties

The osg::ProxyNode node will reduce the start time of the viewer if there are huge numbers of models to be loaded and displayed in the scene graph. It is able to function as the interface of external files, help applications to start up as soon as possible, and then read those waiting models by using an independent data thread. It uses setFileName() rather than addChild() to set a model file and dynamically load it as a child.[3]

By looking at existing code, we can learn more about the SimGear/FlightGear APIs for loading 3D models from the base package. A commonly-used idiom can be seen in the model-manager (flightgear/src/Model/modelmgr.cxx (line 72)) (slightly modified for clarity): 

try {
    const char *model_path = "Models/Geometry/glider.ac";
    std::string fullPath = simgear::SGModelLib::findDataFile(model_path);
    osg::Node * object = SGModelLib::loadDeferredModel(fullPath, globals->get_props());
} catch (const sg_throwable& t) {
    SG_LOG(SG_AIRCRAFT, SG_ALERT, "Error loading " << model_path << ":\n  "
        << t.getFormattedMessage() << t.getOrigin());
    return;
}

Next, we'll need to ensure that read/write permissions (as per IORules) are properly handled by our new code, which is commonly done using fgValidatePath(): 

if (!fgValidatePath(file.c_str(), false)) {
    SG_LOG(SG_IO, SG_ALERT, "load: reading '" << file << "' denied "
        "(unauthorized access)");
    return false;
}

Once those two snippets are integrated, we end up with something along these lines: 

const char *model_path = "Models/Geometry/glider.ac";

if (!fgValidatePath(mode_path, false)) {
    SG_LOG(SG_IO, SG_ALERT, "load: reading '" << file << "' denied "
        "(unauthorized access)");
    return false;
}

try {
    std::string fullPath = simgear::SGModelLib::findDataFile(model_path);
    osg::Node * object = SGModelLib::loadDeferredModel(fullPath, globals->get_props());
} catch (const sg_throwable& t) {
    SG_LOG(SG_AIRCRAFT, SG_ALERT, "Error loading " << model_path << ":\n  "
        << t.getFormattedMessage() << t.getOrigin());
    return;
}

Extending FGCanvasSystemAdapter

These are already the main building blocks required for loading a 3D model from disk ($FG_ROOT) and getting an osg::Node* in return. Next, we need to add this as a method to FGCanvasSystemAdapter to expose this FlightGear-specific API to the Canvas subsystem living in SimGear: 

diff --git a/simgear/canvas/canvas_fwd.hxx b/simgear/canvas/canvas_fwd.hxx
index 3b1b464..15ccd57 100644
--- a/simgear/canvas/canvas_fwd.hxx
+++ b/simgear/canvas/canvas_fwd.hxx
@@ -23,8 +23,10 @@
 #include <simgear/structure/SGWeakPtr.hxx>
 
 #include <osg/ref_ptr>
+#include <osg/Node>
 #include <osgText/Font>
 
+
 #include <boost/function.hpp>
 #include <boost/shared_ptr.hpp>
 #include <boost/weak_ptr.hpp>

diff --git a/simgear/canvas/CanvasSystemAdapter.hxx b/simgear/canvas/CanvasSystemAdapter.hxx
index 2492bb9..48f2894 100644
--- a/simgear/canvas/CanvasSystemAdapter.hxx
+++ b/simgear/canvas/CanvasSystemAdapter.hxx
@@ -38,6 +38,7 @@ namespace canvas
       virtual void addCamera(osg::Camera* camera) const = 0;
       virtual void removeCamera(osg::Camera* camera) const = 0;
       virtual osg::Image* getImage(const std::string& path) const = 0;
+      virtual osg::Node* getModel(const std::string& path) const = 0;
       virtual SGSubsystem* getSubsystem(const std::string& name) const = 0;
       virtual HTTP::Client* getHTTPClient() const = 0;
   };

diff --git a/src/Canvas/FGCanvasSystemAdapter.hxx b/src/Canvas/FGCanvasSystemAdapter.hxx
index c43f793..6b7ad82 100644
--- a/src/Canvas/FGCanvasSystemAdapter.hxx
+++ b/src/Canvas/FGCanvasSystemAdapter.hxx
@@ -21,6 +21,8 @@
 
 #include <simgear/canvas/CanvasSystemAdapter.hxx>
 
+#include <simgear/scene/model/modellib.hxx>
+
 namespace canvas
 {
   class FGCanvasSystemAdapter:
@@ -31,6 +33,7 @@ namespace canvas
       virtual void addCamera(osg::Camera* camera) const;
       virtual void removeCamera(osg::Camera* camera) const;
       virtual osg::Image* getImage(const std::string& path) const;
+      virtual osg::Node* getModel(const std::string& path) const;
       virtual SGSubsystem* getSubsystem(const std::string& name) const;
       virtual simgear::HTTP::Client* getHTTPClient() const;
   };

diff --git a/src/Canvas/FGCanvasSystemAdapter.cxx b/src/Canvas/FGCanvasSystemAdapter.cxx
index 95b6b65..627ab18 100644
--- a/src/Canvas/FGCanvasSystemAdapter.cxx
+++ b/src/Canvas/FGCanvasSystemAdapter.cxx
@@ -75,7 +75,44 @@ namespace canvas
     if( globals->get_renderer() )
       globals->get_renderer()->removeCamera(camera);
   }
+  //----------------------------------------------------------------------------
+  osg::Node* FGCanvasSystemAdapter::getModel(const std::string& path) const
+  {
+    const char *model_path = "Models/Geometry/glider.ac";
 
+    if( SGPath(path).isAbsolute() )
+    {
+      const char* valid_path = fgValidatePath(path.c_str(), false);
+      if( valid_path )
+   try {
+      std::string fullPath = simgear::SGModelLib::findDataFile(valid_path);
+      osg::Node * object = simgear::SGModelLib::loadDeferredModel(fullPath, globals->get_props());
+      return object;
+  } catch (const sg_throwable& t) {
+    SG_LOG(SG_IO, SG_ALERT, "Error loading " << model_path << ":\n  " << t.getFormattedMessage() << t.getOrigin());
+    return 0;
+  } // error loading from absolute path
+      SG_LOG(SG_IO, SG_ALERT, "canvas::Model: reading '" << path << "' denied");
+    } // absolute path handling
+    else
+    {
+      SGPath tpath = globals->resolve_resource_path(path);
+      if( !tpath.isNull() )
+   try {
+      std::string fullPath = simgear::SGModelLib::findDataFile(path.c_str());
+      osg::Node * object = simgear::SGModelLib::loadDeferredModel(fullPath, globals->get_props());
+      return object;
+  } catch (const sg_throwable& t) {
+    SG_LOG(SG_IO, SG_ALERT, "Error loading " << model_path << ":\n  " << t.getFormattedMessage() << t.getOrigin());
+    return 0;
+  } // error loading from relative path
+
+      SG_LOG(SG_IO, SG_ALERT, "canvas::Model: No such model: '" << path << "'");
+    } // relative path handling
+
+    return 0;
+ 
+  }

To see if our new SystemAdapter method is working properly, we can simply add it to the constructor of an existing element, e.g.: 

osg::Node* model = Canvas::getSystemAdapter()->getModel("Models/Geometry/glider.ac");
if (!model) {
    SG_LOG(SG_GL, SG_ALERT, "Adapter not working: getModel()");
} else {
    SG_LOG(SG_GL, SG_ALERT, "Success, model loaded !");
}

While this will not yet add the model to the scene graph (canvas), we should at least be seeing success/failure messages in the console suggesting if our function works properly (that is, once the corresponding element is added). Next, we only need to coax our new Canvas element to use said osg::Node object, which will usually involve adding it as a child to the _transform member of the corresponding CanvasElement sub-class. However, instead of adding this directly, we should add the PAT first:

Turning the filename into a property

So far, we really only have a proof-of-concept loading a hard-coded 3D model from disk and adding this to a Canvas scene-graph. However, to be really useful, the file name should not be hard-coded, but should be just a property living in the sub-tree of the corresponding Canvas element, which is what we're going to implement next. Again, it makes sense to look at existing/similar use-cases in FlightGear, and specifically the corresponding subsystem, i.e. Canvas in this case - for that, we can simply look at the implementation of CanvasImage, which already supports the notion of a "filename", which will update the whole element once modified:

simgear/simgear/canvas/elements/CanvasImage.cxx (line 615) 

if(name == "file"){
    SG_LOG(SG_GL, SG_WARN, "'file' is deprecated. Use 'src' instead");
}

As can be seen here, the general idea is to use the ::childChanged(SGPropertyNode* child) method to look for supported nodes and process updates accordingly. The main thing to keep in mind here is that Canvas/SimGear don't know anything about FlightGear-specific APIs, which is why the CanvasSystemAdapter needs to be used, and called, to expose/access the corresponding APIs:

simgear/simgear/canvas/elements/CanvasImage.cxx (line 680) 

Canvas::getSystemAdapter()
    ->getHTTPClient()
    ->load(url)
    // TODO handle capture of 'this'
    ->done(this, &Image::handleImageLoadDone);

Using loadPagedModel() instead

(see flightgear/AI/Aircraft/AIBase.cxx, Nasal animated models)

Using ProxyNode

Adding a PositionAttitudeTransform

Proof-of-Concept

See also Howto:Use_a_Camera_View_in_an_Instrument#Part_1:_Rendering_a_Camera_View_to_Texture.


You will want to add a new Canvas::Element subclass whenever you want to add support for features which cannot be currently expressed easily (or efficiently) using existing means/canvas drawing primitives (i.e. via existing elements and scripting space frameworks).

For example, this may involve projects requiring camera support, i.e. rendering scenery views to a texture, rendering 3D models to a texture or doing a complete moving map with terrain elevations/height maps (even though the latter could be implemented by sub-classing Canvas::Image to some degree).

Another good example for implementing new elements is rendering file formats like PDF, 3d models or ESRI shape files.

To create a new element, you need to create a new child class which inherits from Canvas::Element base class (or any of its child-classes, e.g. Canvas::Image) and implement the interface of the parent class by providing/overriding the correspond virtual methods.

To add a new element, these are the main steps:

  • Set up a working build environment (including simgear): Building FlightGear
  • update/pull simgear,flightgear and fgdata
  • check out a new set of topic branches for each repo: git checkout -b topic/canvas-Model
  • Navigate to $SG_SRC/canvas/elements
  • Create a new set of files Model.cxx/.hxx (as per Adding a new Canvas element)
  • add them to $SG_SRC/canvas/elements/CMakeLists.txt (as per Developing using CMake)
  • edit $SG_SRC/canvas/elements/CanvasGroup.cxx to register your new element (header and staticInit)
  • begin replacing the stubs with your own C++ code
  • map the corresponding OSG/library APIs to properties/events understood by the Canvas element (see the valueChanged() and update() methods)
  • alternatively, consider using dedicated Nasal/CppBind bindings

Below, you can find patches illustrating how to approach each of these steps using boilerplate code, which you will need to customize/replace accordingly:



Model: Canvas 3D model viewer (required cmake changes)


Model: Canvas 3D model viewer (minimal C++ changes)


Model: Canvas 3D model viewer (registering the new element)


Model: Canvas 3D model viewer $FG_ROOT/Nasal/canvas/api.nas changes


Model: Canvas 3D model viewer (testing the new element using the Nasal Console)
testing a custom canvas element

Next, we need to set up a new camera for rendering our model to a texture - the usual way to accomplish this is to set up a FBO camera (analogous to how the ODGauge code works). First we will set up a texture: 

int tex_width = 512, tex_height = 512;
osg::ref_ptr<osg::Texture2D> texture = new osg::Texture2D;
texture->setTextureSize( tex_width, tex_height );
texture->setInternalFormat( GL_RGBA );
texture->setFilter( osg::Texture2D::MIN_FILTER,
osg::Texture2D::LINEAR );
texture->setFilter( osg::Texture2D::MAG_FILTER,osg::Texture2D::LINEAR );

Now we can set up a new camera and attach it to the texture for rendering: 

osg::Camera* camera = new osg::Camera;
// set up the background color and clear mask.
camera->setClearColor(osg::Vec4(1.0f,1.0f,1.0f,0.0f));
camera->setClearMask(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

// set viewport
camera->setViewport(0,0,tex_width,tex_height);
        
// set the camera to render before the main camera.
camera->setRenderOrder(osg::Camera::PRE_RENDER);
// use FBOs
camera->setRenderTargetImplementation(osg::Camera::FRAME_BUFFER_OBJECT );

// attach the texture we set up to use for rendering
camera->attach(osg::Camera::COLOR_BUFFER, texture.get() );

camera->setReferenceFrame(osg::Transform::ABSOLUTE_RF);

References

References
Retrieved from "https://wiki.flightgear.org/w/index.php?title=Howto:Extending_Canvas_to_support_rendering_3D_models&oldid=136071"