Nasal/CppBind

From FlightGear wiki
Jump to: navigation, search
This article is a stub. You can help the wiki by expanding it.

FlightGear's built-in Nasal scripting language comes with a set of standard libraries, and can be extended using FlightGear specific APIs.

Exposing simulator internals to scripting space is a fairly common and useful thing, because it enables base package developers to access these internals without having to build FlightGear from source, so the barrier to entry is significantly lower and we've seen an increasing number of novel features purely implemented in scripting space, due to powerful APIs being available to aircraft developers and other base package developers.

Until FlightGear 2.8, the Nasal scripting engine only provided a C API to expose such hooks/bindings to scripting space or to expose scripting space data structures back to C/C++.

Unlike the core Nasal engine itself (which is C), FlightGear however is mostly written and being developed in C++. For quite a while, that meant that the Nasal APIs were a bit low-level, and sometimes also awkward to use when making functions, data structures or objects accessible between C++ and Nasal.

Thanks to development on Tom's Canvas system, there's now a new bindings framework to be found in $SG_SRC/simgear/nasal/cppbind. This is fully object oriented and supports modern C++ features by operating through classes and methods with full STL support, abstracting most common operations away.

You will find that most of the "old" code in $FG_SRC/Scripting still uses those old C-APIs for interacting with the Nasal engine. Only the new code, those that #include <simgear/nasal/cppbind> use boost templates to hide low level details.

Most of the code in the Nasal subsystem itself (FGNasalSys) also still uses the legacy C APIs. You will find the old, low-level APIs explained at Howto:Extend Nasal - this is just to explain the two approaches, to avoid unnecessary confusion.

The cppbind framework is much more generic and high level than the bare C APIs, cppbind includes unit testing support and makes use of modern C++ features like templates and STL support, including SimGear specific types like SGPath/SGGeod etc, its overhead is fairly small (not just performance, but also LoC to create new bindings). The cppbind framework is already extensively used by the Canvas system and the NasalPositioned_cppbind bindings, both of which are a good place to look for code examples.

Meanwhile, we suggest to favor cppbind over the old, low-level, approach, it isn't only much more elegant, but also saves you tons of typing, too - and will do certain error-checking automatically that you would otherwise have to implement manually.

Worthwhile Targets

Note: While this article is based on considerable community feedback, there's nobody working on this currently.
So if you'd like to help in one way or another, please get in touch or just help improve the article in the meantime!
Useful Skills:
Building Flightgear, Developing using CMake, C++, some Nasal


People:

Mentors: Hooray (get in touch to learn more)
It's possible that this article hasn't been updated in a while, so to catch up with the latest developments, you are advised not to start working on anything directly related to this without first coordinating your ideas with fellow FlightGear contributors using the FlightGear developers mailing list or the FlightGear forums. See also the talk page.


FlightGear's built-in Nasal scripting language comes with a set of standard libraries, and can be extended using FlightGear specific APIs.

Exposing simulator internals to scripting space is a fairly common and useful thing, because it enables base package developers to access these internals without having to build FlightGear from source, so the barrier to entry is significantly lower and we've seen an increasing number of novel features purely implemented in scripting space, due to powerful APIs being available to aircraft developers and other base package developers.

Until FlightGear 2.8, the Nasal scripting engine only provided a C API to expose such hooks/bindings to scripting space or to expose scripting space data structures back to C/C++.

Unlike the core Nasal engine itself (which is C), FlightGear however is mostly written and being developed in C++. For quite a while, that meant that the Nasal APIs were a bit low-level, and sometimes also awkward to use when making functions, data structures or objects accessible between C++ and Nasal.

Thanks to development on Tom's Canvas system, there's now a new bindings framework to be found in $SG_SRC/simgear/nasal/cppbind. This is fully object oriented and supports modern C++ features by operating through classes and methods with full STL support, abstracting most common operations away.


After working through the Nasal/CppBind article, some of the more useful things to play with in the beginning, would be exposing additional SG/FG classes to Nasal space, such as for example:

Done

Work in Progress

  • SGPropertyChangeListener Pending Pending (suggested by Zakalawe & TheTom) [1]
Cquote1.png This and using maketimer instead of settimer should reduce the number of leaked resources a lot, because you would not be able to accidentally leak listeners/timers anymore.
— Thomas Geymayer (2014-11-22). [Flightgear-devel] RFC: Nasal ghosts and garbage collection.
(powered by Instant-Cquotes)
Cquote2.png

Autopilot/Property Rules

Note  This is a summary of all discussions about exposing the autopilot/property-rule system (there are certain Nasal GC issues, so that we ask people not to implement FDM-coupled Nasal code like autopilots): this would be the best way to decrease the amount of Canvas-related Nasal code, i.e. by using property-rules for animation purposes, as per Torsten's RBAR EFIS [2] and TheTom's system-modeling plans.
Cquote1.png The quantity of details and system modeling that goes in to covering all the aircraft of the world is impossibly complex. The idea is to put as much support for common/shared systems in C++ as we can and then make it possible to stitch these systems and details together and configure them with xml in a wide variety of ways to create aircraft. But we can never anticipate every system in use, and we can't anticipate the level of detail or feature set that every aircraft developer might want to implement or experiment with, and even if we could there would be no way to model everything in the world in a single application. Nasal gives a lot of flexibility to cover those unanticipated gaps and it allows aircraft developers to push in new areas ahead of the C++ coverage. Now in many cases, aircraft developers were happy with the nasal implementation and called it good enough. Many aircraft developers have become proficient and comfortable in nasal and prefer doing their work there.
— curt (Dec 16th, 2015). Re: Military simulation (from Su-15 Screenshots).
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png Nasal does have an advantage in that it is easier to tailor to specific requirements. So, providing that the CPU overhead is acceptable, this may be a preferable method for many aircraft.A C++ coded module is fixed in stone, but nasal and xml modules are far easier to modify/overload on a per-aircraft basis.As I am modelling a 1960´s military bomber I have quite a number of (very) aircraft specific requirements which are not met by the current RM/GPS code which is targeted at present day commercial usage.
— Alant (Aug 19th, 2015). Re: Route manager leg modes.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png I would prefer to do this in an XML filter in the generic autopilot helpers - definitely not in Nasal. It can be done in C++ if strictly required but then we need way to disable it for people who want different filtering.
— James Turner (2015-04-03). Re: [Flightgear-devel] Route manager: waypoint smoothing.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png I vote for #3: avoid *any* Nasal in the fast simulation [FDM] loop. Nasal execution is slow and non-deterministic. Running it in the fast simulation loop is the last thing we want.
Cquote2.png
Cquote1.png Concerning your original issue on implementing an autopilot: a much better way to do it is to avoid Nasal for the actual autopilot controller elements (numeric computation). Instead, use XML "autopilot" rules for the filter, gain, damper, integrator elements: http://wiki.flightgear.org/Autopilot_Configuration_Reference

You can then use Nasal for the high level stuff, and enable/disable/switch the individual controller elements (e.g. in order to automatically switch the autopilot mode when capturing the ILS).


Cquote2.png
Cquote1.png This is also how such things are done in the real world: controllers aren't implemented in imperative programming languages these days - especially not in scripting languages. People use model-based design and connect controller elements - using graphical tools like MATLAB/Simulink. Obviously, FG is missing a graphical interface to specify the controller rules - but the idea of specifying through XML is the same and specification is straight forward.
Cquote2.png
Cquote1.png I agree with your main point that xml-configured hard-coded filters are the right way to implement and autopilot, and I also agree that in general low-level multi-purpose workhorse code should be C++ whereas Nasal is more suitable for the numerically cheap high-level specific functions.
— Renk Thorsten (2012-08-30). Re: [Flightgear-devel] Running Nasal at simulation rate.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png it should basically resemble the C++ 3D animation system and be invisible (enough) that it could easily be replaced with more C++ should we need more performance (or just 'cause). Exposing SGExpression to Nasal would be helpful soon but not necessary yet.
— Philosopher (Sat Aug 16).  animations.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png Given that a very simple animation system would mainly need to deal with "events/triggers" (i.e. timers & listeners) and "actions", we might even be able to reuse some of galvedro's sophisticated failure management modules, because those already deal with both concepts via the property tree (sent a heads-up to him for some feedback).
— Hooray (Sat Aug 16). Re: .
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png I'd strongly agree with Thorsten here. It's nothing against Nasal from me - I've not even used it - but creating an autopilot (or any GNC or system model, for that matter) can be done very effectively with discrete objects such as summers, gains, controllers, filters, switches, etc., much as JSBSim has done with the system components. This is a standard approach in industry as Thorsten mentions as exemplified by Mathwork's $imulink product.

Scilab/Scicos is similar in concept. Control system topologies are often diagrammed in a way that can lead to a one-to-one correspondence between a block and a control system object that can be referenced in an XML file, if the control system component library has been defined properly. This, again, is the way that JSBSim has approached the solution. Some benefits to such an approach include (IMHO) better testability, more predictability, and easier interface (someday) with a GUI tool, should one materialize. The downside is that XML can be verbose, but it's a price I've come to accept.


— Jon S. Berndt (2012-08-30). Re: [Flightgear-devel] Running Nasal at simulation rate.
(powered by Instant-Cquotes)
Cquote2.png


Cquote1.png I have recently committed some code to allow runtime loading of property rules and have a Nasal binding for that in mind.
— Torsten (Thu Feb 02). Re: 2 Questions: vacuum & electrical.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png The more I think about it, the more I am leaning towards unifying the different system modeling blocks in the C++ core under a generic interface that is exposed (or linked in some way) to Nasal. Think the PID controller, the different filters, flip-flops, etc. They are not substantially different to the basic bricks I am writing...The basic idea would be to detach those blocks from their specific application (autopilot, for example) and refactor them into an independent library with bindings in Nasal and a similar interface to what I have been showing so far. The end result would be quite simulinkish in flavour. It is already starting to smell a bit to that actually... :D

An architecture like that would eventually enable three possible approaches to system modeling: low level C++, static xml driving C++ underneath and fully scripted Nasal.


— galvedro (Tue Nov 05). Re: A general approach to systems modeling in Nasal.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png Regarding things like the PID controller code, its developer/maintainer (Torsten) was actually planning on making this stuff accessible from Nasal, just to prevent scripters from implementing APs in Nasal (due to garbage collection issues) - so that should be a no-brainer actually, and such work should be appreciated
Cquote2.png
Cquote1.png there's an extremely powerful and flexible autopilot system in FG that is entirely XML configurable: Autopilot

There's also a very powerful route manager. Please note however, that there's currently no support for AI traffic to directly make use of the autopilot system or the route manager, so you need to come up with your own infrastructure in scripting space.


— Hooray (Thu Jan 05). Re: Multiple intelligent flyers.
(powered by Instant-Cquotes)
Cquote2.png


Cquote1.png Note however that scripted AI traffic cannot currently make use of any hard-coded FDMs/AP/RM functionality (JSBSim/YaSim), instead you need to come up with your own "pseudo systems" in scripting space unfortunately.
— Hooray (Tue Jan 03). Re: Multiple intelligent flyers.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png The AI traffic system has its own "route manager" system which is currently not yet compatible with the rest of FG unfortunately. But there are plans in place to fix this eventually: http://flightgear.org/forums/viewtopic. ... 0&#p134970
— Hooray (Thu Jan 05). Re: Multiple intelligent flyers.
(powered by Instant-Cquotes)
Cquote2.png


Cquote1.png At the moment, Durk has already implemented his own "custom" AI FDM logic, exactly like David predicted a decade ago
— Hooray (Thu Mar 15). Re: [SUGGESTION] Multi-core FlightGear support.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png You can probably find 50+ postings by long term contributors suggesting that AI traffic with FDM support would be a good idea: An_Integrated_AI_Traffic_System#FDM_driven_AI_Traffic
— Hooray (Thu Mar 15). Re: [SUGGESTION] Multi-core FlightGear support.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png When you then take into account that you will probably not need to run the AI traffic FDM at a rate of 120 hz (the FlightGear default), but much more likely at 1-5 hz (at most), you should be able to multiplex at least 20-30 different aircraft onto one dedicated FDM thread (or process).


Thus, the problem lies in the integration part and not the lack of computing power: AI traffic is already the component that accounts for many reported performance bottlenecks and unfortunately the "solution" has often be to disable AI traffic or at least reduce traffic complexity.
Before these issues are resolved, we are unlikely to see real FDMs being supported to simulate AI traffic, even though that would simplify many things in one go (e.g. any aircraft could be used as an AI aircraft and so AI aircraft could also benefit from other systems such as the autopilot or route manager).


— Hooray (Wed Feb 24). Re: "Sophisticated AI aircraft" ... ?.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png you would need to hack the autopilot code so that autopilot configurations can be loaded and created dynamically. At the moment, the autopilot code makes the fixed assumption that there's only a single "main aircraft", so it doesn't know about more than one aircraft.
— Hooray (Thu Jan 05). Re: Multiple intelligent flyers.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png if you wanted to equip your AI traffic with a working route manager, autopilot or even FDM, you would also need to instantiate these subsystems dynamically
— Hooray (Thu Jan 05). Re: Multiple intelligent flyers.
(powered by Instant-Cquotes)
Cquote2.png

props.nas

Cquote1.png ne thing I thing I want to achieve with this changes is to make the Nasal props API more similar to its C++ counterpart as this makes it easier to use if you are using both the C++ and the Nasal API. Also someday I want to refactor nasal-props.cpp to use cppbind, where I want to export as much methods as possible with exactly the same signature than in C++. Especially if using properties seldom (eg. only for initialiation) the relative versions are probably even faster, as the Nasal overhead is lower. Eg. consider the following Nasal code used to initialize some module: var cfg = props.globals.getNode("/my/config/root", 1); var x = cfg.getDoubleValue("x"); var do_it = cfg.getBoolValue("do_it"); Using getprop on the one hand does not allow getting a property converted to a given type and on the other hand is tedious to use for more than one property, as one has to assemble the according property paths (which is definitely less efficient than using a relative method).
— Thomas Geymayer (Apr 14th, 2013). Re: [Flightgear-devel] Nasal props API relative path support.
(powered by Instant-Cquotes)
Cquote2.png

Candidates

Cquote1.png When we have vector road data at runtime, we can do the following:
  • render it in moving-map displays - either real ones (Garmin G1000 / G430) or the map dialog
  • use it to drive building outline creation as Thomas was suggesting
  • animate traffic on the roads, as point lights at night or even meshes in daytime
    (the important one...)
  • generate tile overlay textures dynamically (and based on view distance) to show the roads on the terrain surface. (And the detail of this texture can include road markings / borders based on distance from the viewer / view angle / etc)

— James Turner (2014-11-21). Re: [Flightgear-devel] Future city terrain strategy.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png Specifically, there are some C++ data structures that still need to be exposed to Nasal via cppbind so that we can implement features available in the Map dialog and the hard-coded ND

Canvas_GUI#PUI_Widgets


— Hooray (Tue Jun 24). Phasing out MapWidget post 3.2.
(powered by Instant-Cquotes)
Cquote2.png
Note  Before working on anything related, please do get in touch with other contributors to ensure that this list is still up-to-date.

For more technical Nasal questions (C API, internals etc), you'll probably want to refer to Philosopher, TheTom, Zakalawe or Hooray on the forum - TheTom and Zakalawe can also provide help on using cppbind, having both used it extensively during the last months.


Prerequisites

Objective

Provide a fully annotated step-by-step introduction to Nasal's cppbind framework. This is mostly based on existing code in SimGear/FlightGear. The cppbind framework itself is to be found $SG_SRC/nasal/cppbind and it's pretty well commented, and makes use of Doxygen strings. If you are already familiar with C++ and SG/FG,, you'll want to check out the unit tests in cppbind_test.cxx.

This write-up should get you started with the basics of the framework.

Next, there are more sophisticated examples to be found in $FG_SRC/Scripting, you'll want to look at the following sources that make use of cppbind (listed in ascending complexity):

  • NasalString.?xx
  • NasalHTTP.?xx
  • NasalCanvas.?xx
  • NasalPositioned_cppbind.?xx

Getting started

Open $FG_SRC/Scripting/CMakeLists.txt and add these entries to the SOURCES/HEADER section respectively:

  • NasalDemo.cxx (SOURCES)
  • NasalDemo.hxx (HEADERS)

Let's create the NasalDemo.hxx header file first:

// NasalDemo.hxx
#ifndef SCRIPTING_NASAL_DEMO_HXX
#define SCRIPTING_NASAL_DEMO_HXX
#include <simgear/nasal/nasal.h>
 
naRef initNasalDemo(naRef globals, naContext c);
#endif // of SCRIPTING_NASAL_DEMO_HXX

Next, open $FG_SRC/Scripting/NasalSys.cxx and locate the FGNasalSys::init() method, to call the new initNasalDemo() function, add this to the bottom of the function:

initNasalDemo(_globals, _context);

Next, we can create the NasalDemo.cxx file

#include <Main/globals.hxx>
#include <Main/util.hxx>

// $FG_SRC/Scripting/NasalDemo.cxx
 
#include <simgear/nasal/cppbind/from_nasal.hxx>
#include <simgear/nasal/cppbind/to_nasal.hxx>
#include <simgear/nasal/cppbind/NasalHash.hxx>
#include <simgear/nasal/cppbind/Ghost.hxx>
 
//the struct we want to expose to Nasal (could also be a class obviously)
struct Test {
 int value;
 void hello() {
  std::cout << "Hello World from CppBind!\nValue is:" << value ; 
 }
 void setValue(const int val) {value=val;}
 int getValue() const {return value;}
};
 
// cppbind manages all objects as shared pointers
// use boost::shared_ptr or SGReferenced objects
// typically, you'll want to provide two helper
// functions for each of your classes

typedef boost::shared_ptr<Test> Test_ptr;
typedef nasal::Ghost< Test_ptr > NasalTest;
 
// next, two helper functions that tell cppbind how to
// convert our objects when passing them between C++  <-> Nasal

// this will be used whenever you want to turn a C++ object into a Nasal Ghost 
naRef to_nasal_helper(naContext c, Test *obj)
  { 
    Test_ptr ptr(obj); // set up a smart pointer wrapping obj
    return NasalTest::create(c, ptr ); // return the smart pointer wrapped in a naGhost
  }
 

// and this will be used whenever you want to turn a Nasal Ghost
// into a C++ object
Test*
from_nasal_helper(naContext c, naRef ref, const Test*)
  { 
      return (Test*) naGhost_ptr(ref);
  }
 
 
// create a new Test object and returns it to Nasal
// as a naRef, wrapped in an naGhost
// 
static naRef f_newtest(const nasal::CallContext& ctx)
{
  Test* t = new Test();
  // we can do some initial state setup now
  t->value=100;
  // and now return the new object to Nasal 
  return ctx.to_nasal( t ); // NOTE: this only calls to_nasal - the to_nasal_helper we provided above is internally used
}
 
 
// this will register our bindings in the Nasal engine
// it should  be called at the end of $FG_SRC/Scripting/NasalSys.cxx::FGNasalSys::init()
// 
// to call the code, add this to FGNasalSys::init():
/*
 	initNasalDemo(_globals, _context);
*/

naRef initNasalDemo(naRef globals, naContext c)
{
   if(NasalTest::isInit() ) return naNil(); // avoid re-init during reset/re-init

    // This only needs to be called once for each ghost, so make sure to use the ::isInit() check in FGNasalSys::init()
    NasalTest::init("Test") // this is the ghost's symbol used in error messages/diagnostics (it is NOT the namespace/symbol used by nasal code!)
                .method("hello", &Test::hello) // add a method to the ghost and map it to the method in the struct/class
		.member("value", &Test::getValue, &Test::setValue); 
    // set up a  new namespace for our functions, named test
    nasal::Hash globals_module(globals, c),
              test = globals_module.createHash("test"); // this is the namespace we'll see and use in Nasal
 
   // add an allocator to the test namespace for creating new test objects
   // which will be accessible as test.new()
   test.set("new", &f_newtest);
 
 
    return naNil(); //we already did all the namespace setup, so we can simply return naNil() here, it's discarded anyways 
}

Testing the whole thing

Rebuild & run FlightGear, and then fire up the Nasal Console, and run this:

# inspect our new test namespace
debug.dump( test );

# create a test object:
var obj = test.new();

# inspect the result of running our allocator function
debug.dump( obj );
 
# run the hello method
obj.hello();

# print the value
print( obj.value );

# change the value
obj.value = 4444;

# print it again
print( obj.value );

Exposing HLA classes to Nasal

Here's another stub, this time exposing the simgear::HLAFederate class and a handful of its methods to Nasal space as a Nasal Ghost. Note that the following snippet assumes, that:

  • You configured, built & installed OpenRTI (see HLA)
  • you configured, built & installed SimGear with -DENABLE_RTI=ON
  • you configured, built FlightGear with -DENABLE_RTI=ON

In and of itself, this snippet isn't yet particularly useful, especially because we don't want HLA federates to be running inside the main process, but the same HLA bindings could obviously also be used by a standalone Nasal interpreter at some point. So this just serves as an example. To learn more about HLA, please also see Developing with HLA and Nasal HLA standalone.

// $FG_SRC/Scripting/NasalHLA.cxx

#ifdef HAVE_CONFIG_H
#  include "config.h"
#endif

#include <Main/globals.hxx>
#include <Main/util.hxx>
 
#include <simgear/nasal/cppbind/from_nasal.hxx>
#include <simgear/nasal/cppbind/to_nasal.hxx>
#include <simgear/nasal/cppbind/NasalHash.hxx>
#include <simgear/nasal/cppbind/Ghost.hxx>

#include <simgear/hla/HLAFederate.hxx>
 
typedef boost::shared_ptr<simgear::HLAFederate> HLAFederate_ptr;
typedef nasal::Ghost< HLAFederate_ptr > NasalHLAFederate;

naRef to_nasal_helper(naContext c, simgear::HLAFederate* obj)
{
	HLAFederate_ptr ptr(obj);
	return NasalHLAFederate::create(c, ptr);
}
 
simgear::HLAFederate*
from_nasal_helper(naContext c, naRef ref, const simgear::HLAFederate*)
{
	return (simgear::HLAFederate*) naGhost_ptr(ref);
}

typedef boost::shared_ptr<simgear::HLAObjectClass> HLAObjectClass_ptr;
typedef nasal::Ghost< HLAObjectClass_ptr > NasalHLAObjectClass;

naRef to_nasal_helper(naContext c, simgear::HLAObjectClass* obj)
{
	HLAObjectClass_ptr ptr(obj);
	return NasalHLAObjectClass::create(c, ptr);
}
 
simgear::HLAObjectClass*
from_nasal_helper(naContext c, naRef ref, const simgear::HLAObjectClass*)
{
	return (simgear::HLAObjectClass*) naGhost_ptr(ref);
}

static naRef f_new_federate(const nasal::CallContext& ctx)
{
  
  return ctx.to_nasal( new  simgear::HLAFederate );
}
 

naRef initNasalHLA(naRef globals, naContext c)
{
    nasal::Hash globals_module(globals, c);
 
   using namespace simgear;
   NasalHLAFederate::init("hla.federate")
	.method("init", &HLAFederate::init) 
	.method("update", &HLAFederate::update)
	.method("shutdown", &HLAFederate::shutdown)
	.method("createObjectClass", &HLAFederate::createObjectClass)
	.method("setFederateType", &HLAFederate::setFederateType)
	.method("setFederationExecutionName", &HLAFederate::setFederationExecutionName);

   nasal::Hash hla = globals_module.createHash("hla");
   hla.set("new", &f_new_federate); 
   return naNil();
}

CallContext

  • isNumeric()
  • isString()
  • isHash()
  • isVector()
  • isGhost()
  • requireArg<type>(index)
  • getArg()
  • popFront()
  • popBack()

Exposing C++ Classes

Argument processing

Returning Nasal classes to C++

Inheritance

Exposing static members

Supporting custom Types

  • to_nasal
  • from_nasal