Howto:Extend Nasal: Difference between revisions

From FlightGear wiki
Jump to navigation Jump to search
Line 283: Line 283:
  naRef naStringValue(naContext c, naRef n);
  naRef naStringValue(naContext c, naRef n);


= Wrapping Classes as Nasal Objects (ghosts) =
= Wrapping C++ Classes as Nasal Objects (ghosts) =
'''To be written by Hooray & Zakalawe'''
'''To be written by Hooray & Zakalawe'''



Revision as of 17:47, 27 June 2012

This article is a stub. You can help the wiki by expanding it.

This article is dedicated to describing how to write custom C/C++ extension functions in order to extend the Nasal scripting interpreter in FlightGear, for example in order to expose new or existing FlightGear APIs to the Nasal scripting engine, so that Nasal scripts can access additional FlightGear internals.

Some interesting ideas for extending Nasal this way have been collected at Proposals:Nasal related.

Pre-Requisites

In order to work through this article, you will likely need to be interested in FlightGear core development, need to be somewhat familiar with C/C++, as well as with some basic Nasal (given that Nasal itself is implemented in ANSI C, basic C knowledge will mostly do for starters-C++ knowledge will only be required in order to understand the FlightGear side of things).

Also, you should have experience compiling FlightGear (see Building FlightGear for Linux instructions or Building Flightgear - Windows for Windows specific instructions).

In addition, you may want to check out the FlightGear Integration Docs.

Nasal

Introductory information on the Nasal programming language itself can be found at Nasal scripting language. Information on writing simple Nasal scripts can be found at Howto: Write simple scripts in Nasal. Useful Nasal snippets can be found at Nasal Snippets. A Nasal Style Guide is available at Nasal Style Guide (Note that as of 05/2009, this is work in progress).

If you have any Nasal specific questions, you will want to check out the Nasal FAQ, feel free to ask new questions or help answer and refine existing ones.

All Nasal related articles can be found in the Nasal category.

Note: FlightGear's version of the Nasal interpreter is maintained in the SimGear git repository, inside the [gitorious.org/fg/simgear/trees/next/simgear/nasal] folder, the most important header file detailing the internal Nasal API is "nasal.h", you will want to check this out for the latest changes and information.

You will probably also want to check out the [gitorious.org/fg/simgear/trees/next/simgear/nasal] folder for specific examples on using the various Nasal APIs that are not yet covered here completely.

Important: As of 05/2009, this article is work in progress, and none of the examples have so far been tested/compiled. Your help in improving and updating this article is appreciated, thanks!

Intro

In FlightGear, the simplest way to add new extension functions is to look at the existing functions in $FG_SRC/Scripting/NasalSys.cxx(src/Scripting/NasalSys.cxx).

There is a static table of function pointers (named funcs[]) referencing extension functions, along with their corresponding names in Nasal: http://gitorious.org/fg/flightgear/blobs/next/src/Scripting/NasalSys.cxx#line482 The following is a copy of the extension function list, taken in 05/2009:

  // Table of extension functions.  Terminate with zeros.
  static struct { const char* name; naCFunction func; } funcs[] = {
    { "getprop",   f_getprop },
    { "setprop",   f_setprop },
    { "print",     f_print },
    { "_fgcommand", f_fgcommand },
    { "settimer",  f_settimer },
    { "_setlistener", f_setlistener },
    { "removelistener", f_removelistener },
    { "_cmdarg",  f_cmdarg },
    { "_interpolate",  f_interpolate },
    { "rand",  f_rand },
    { "srand",  f_srand },
    { "abort", f_abort },
    { "directory", f_directory },
    { "parsexml", f_parsexml },
    { "systime", f_systime },
    { "carttogeod", f_carttogeod },
    { "geodtocart", f_geodtocart },
    { "geodinfo", f_geodinfo },
    { "airportinfo", f_airportinfo },
    { 0, 0 }
  };

So, the basic format is "name" (string), function_pointer - whereas "name" refers to the internal name used by Nasal and its scripts, and "function_pointer" has to use the right function signature and is a pointer to the implementation of the Nasal function in C/C++ code space:

 // The function signature for an extension function:
 typedef naRef (*naCFunction)(naContext ctx, naRef me, int argc, naRef* args);

You will need to add your new extension function to this list of static functions, preferably following the existing naming convention (i.e. "f_" prefix).

If your extension functions are likely to be fairly low level, and will thus be provided with a more abstract wrapper in Nasal space, these functions should use a prepended undercore ("_"), such as the _fgcommand, _setlistener, _cmdarg and _interpolate functions.

Extension Function Signature

These extension functions look like:

  static naRef f_your_function(naContext c, naRef me, int argc, naRef* args)
  {
      // ...
  }

So, this is basically the boilerplate that you'll need to use in order to expose a new function (i.e. you can just copy & paste this into the NasalSys.cxx file and change the function's name accordingly).

If you don't have anything else to return, you can just return naNil(), so that a function named "f_cool" looks like this:

  static naRef f_cool (naContext c, naRef me, int argc, naRef* args)
  {
      SG_LOG(SG_GENERAL, SG_ALERT, "Nasal:cool() got executed!");
      return naNil();
  }

In order to make this function known to the Nasal interpreter, you'll want to extend the previously mentioned list of Nasal extension functions (in src/Scripting/NasalSys.cxx), to read:

 // ....
   { "airportinfo", f_airportinfo },
   { "cool", f_cool }, //this is where we add our new function
   { 0, 0 }
 };

Once you have made these modifications and rebuilt FlightGear (running make in the Scripting sub folder and relinking FG should suffice), you can simply invoke your new function, of course it will not yet do anything useful, though:

 #cooltest.nas (to be saved in $FG_ROOT/Nasal or run via the Nasal console)
 cool();

So, in order to make it slightly more interesting, we are going to change the return statement to return something else, instead of nil:

  static naRef f_cool (naContext c, naRef me, int argc, naRef* args)
  {
      const char* cool="cool";
      naRef retval;
      naStr_fromdata(retval, cool, strlen(cool) );
      return retval;
  }

So, after rebuilding the FlightGear binary, whenever you call the new "cool" API function, it will always return a "cool" string, which you could for example print out using a script like the following:

 #cooltest.nas (to be saved in $FG_ROOT/Nasal)
 var result=cool();
 print(result);

Argument Processing

Consider the callback signature:

 static naRef f_cool (naContext c, naRef me, int argc, naRef* args)

The arguments to the function are passed in in the args array. The number of arguments is passed via the argc parameter (this is basically consistent with the standard signature of main in C/C++).

So, if you know that you require a certain number of arguments, you can also directly check argc to to match your requirements and show an error message, return nil, or throw an exception using naRuntimeError():

 // Throw an error from the current call stack.  This function makes a
 // longjmp call to a handler in naCall() and DOES NOT RETURN.  It is
 // intended for use in library code that cannot otherwise report an
 // error via the return value, and MUST be used carefully.  If in
 // doubt, return naNil() as your error condition.  Works like
 // printf().
 void naRuntimeError(naContext c, const char* fmt, ...);

The "me" reference is set if the function was called as a method call on an object (e.g. object.your_function() instead of just your_function(), in which case "me" would be set to the object (Nasal hash)).

The naRef objects can be manipulated using the functions in nasal.h. For the latest copy of the file, see: http://gitorious.org/fg/simgear/blobs/next/simgear/nasal/nasal.h

Basically, you can check the type of the reference with the following naIs*() functions:

 int naIsNil(naRef r);
 int naIsNum(naRef r);
 int naIsString(naRef r);
 int naIsScalar(naRef r);
 int naIsVector(naRef r);
 int naIsHash(naRef r);
 int naIsCode(naRef r);
 int naIsFunc(naRef r);
 int naIsCCode(naRef r);

And then get the appropriate value out with things like naNumValue(), naStr_data(), etc...

So, if we're now to change the interface of our earlier "cool" function in the example, to only return "cool" if we are passed at least one numerical parameter, and otherwise return "uncool", it could look like this:

 static naRef f_cool (naContext c, naRef me, int argc, naRef* args)
 {
     const char* cool="cool";
     const char* uncool="uncool";
     const char* result;
     naRef retval;
     if (!argc || !isNaNum(args[0]))
       result=uncool;
     else
       result=cool;     
     return naStr_fromdata(retval, result, strlen(result) );
 }

You can test this, by running a simple Nasal script like the following:

 # test new cool() function:
 print( cool("foo") ); 
 print( cool(100)   );

A common way to evaluate parameters passed to Nasal extension functions in C, looks like this:

 static naRef f_foo (naContext c, naRef me, int argc, naRef* args) {
    naRef param1 = argc > 0 ? args[0] : naNil();
    naRef param2 = argc > 1 ? args[1] : naNil();
    naRef param3 = argc > 2 ? args[2] : naNil();
    //further parameter processing
    return naNil;
 }

This will basically look for 3 parameters, and assign them accordingly - if they are not available, they will just be assigned nil using the naNil() calls (the next step would be to check if the parameters have the right types using the naIs* helpers mentioned above).

Creating Nasal Data Structures

There are also functions to create Nasal data structures (hash tables, vectors, etc...) which you can return to the caller simply by returning the resulting naRef:

 naRef naNil();
 naRef naNum(double num);
 naRef naNewString(naContext c);
 naRef naNewVector(naContext c);
 naRef naNewHash(naContext c);
 naRef naNewFunc(naContext c, naRef code);
 naRef naNewCCode(naContext c, naCFunction fptr);

String Manipulation API

// String utilities:
int naStr_len(naRef s);
char* naStr_data(naRef s);
naRef naStr_fromdata(naRef dst, const char* data, int len);
naRef naStr_concat(naRef dest, naRef s1, naRef s2);
naRef naStr_substr(naRef dest, naRef str, int start, int len);
naRef naInternSymbol(naRef sym);


Vector Manipulation API

// Vector utilities:
int naVec_size(naRef v);
naRef naVec_get(naRef v, int i);
void naVec_set(naRef vec, int i, naRef o);
int naVec_append(naRef vec, naRef o);
naRef naVec_removelast(naRef vec);
void naVec_setsize(naRef vec, int sz);


Hash Manipulation API

// Hash utilities:
int naHash_size(naRef h);
int naHash_get(naRef hash, naRef key, naRef* out);
naRef naHash_cget(naRef hash, char* key);
void naHash_set(naRef hash, naRef key, naRef val);
void naHash_cset(naRef hash, char* key, naRef val);
void naHash_delete(naRef hash, naRef key);
void naHash_keys(naRef dst, naRef hash);

// Retrieve the specified member from the object, respecting the
// "parents" array as for "object.field".  Returns zero for missing
// fields.
int naMember_get(naRef obj, naRef field, naRef* out);
int naMember_cget(naRef obj, const char* field, naRef* out);

A common way to set up hash field members easily, is using a simple macro like:

#define HASHSET(s,l,n) naHash_set(naHash, naStr_fromdata(naNewString(c),s,l),n)
//... do your hash setup here
#undef HASHSET

As you can see, this allows you to easily set up hash fields.

This will call the naHash_set() helper, with the target hash being its first argument, the key name coming next and the naRef (n) being last.

The macro can be even further simplified by automatically computing the length of the hash key using strlen() instead of the explicit length argument:

#define HASHSET(s,n) naHash_set(naHash, naStr_fromdata(naNewString(c),s,strlen(s)),n)
//... do your hash setup here
#undef HASHSET

Conversion & Comparison routines

Some useful conversion/comparison routines include:

int naEqual(naRef a, naRef b);
int naStrEqual(naRef a, naRef b);
int naTrue(naRef b);
naRef naNumValue(naRef n);
naRef naStringValue(naContext c, naRef n);

Wrapping C++ Classes as Nasal Objects (ghosts)

To be written by Hooray & Zakalawe

For the time being, it's a good idea to take a look at the FGPositioned wrappers for the navdb to see how this is done in Nasal, see $FG_SRC/Scripting/NasalPositioned.cxx. Also, the implementation of the SGPropertyNode bindings in nasal-props.cxx contains additional examples.

Background

Quoting: http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg17546.html

Absolutely agreed that making as much data as possible  
available is valuable - as you say, I'm always amazed by the neat  
things people do in nasal scripts. [...] 
I'm still thinking on different ways to make the core datasets a  
bit more 'lazy'.

Quoting: http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg29032.html

One problem is that functions like airportinfo() already do *way* too much, and making it do more will not 
help. We need to stop exposing *functions* to Nasal, and start exposing *objects*, with properties. 

Notably, amongst the airportinfo() structure is a runways hash, which contains data that depends on the Airport Scenery data - this means it can trigger the 
lazy loading of that data, and hence require a disk access. I've noticed the same problem with the map dialog. Most people accessing airportinfo() only want 
one or two pieces of info, but we compute the whole lot each time. I realise that making proper Nasal ghosts for FGPositioned, FGNavRecord, 
FGRunway, FGAirport is more work, but it would be the right path to putting the Nasal API on parity with the C++ one, and would enable a huge amount of 
FMS/digital charting/ATC functions to be built in Nasal.

If that sounds impossible, then I'd strongly advocate creating separate special purpose-functions in the short term, that do *one* thing, and be more easily 
subsumed if/when we properly expose the C++ types to Nasal.

Quoting: http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg29034.html

Yes I'd like to second the idea of returning objects (with attributes
and methods for doing interesting things), I'm guessing we don't need to
abstract it too far from what is provided underneath.

However I really like the idea of getting back an array of airports
within some radius of a centre lat/lon pair, and/or within a bounding
box (2 or 4 lat/lon pairs), and if the same could be done with other
navigation elements in nav.dat it would be most excellent!

Quoting: http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg29035.html

Well, this is exactly what all the query methods on FGPositioned do - the 
problem is, they return FGPositioned subclasses, which aren't much use to Nasal 
right now! Exposing the FGPositioned query interfaces (and the related, 
specialised versions on FGAirport, FGNavRecord, etc) would be quite quick, but 
it's pointless until the results are useful / interesting to Nasal..

One interim step - it would be possible to wrap the query methods, but write 
code (exactly as airportinfo() is currently implemented!) to map the 
FGPositioned subclasses to a 'static' naHash. That's easier than creating 
proper naGhosts, and could have a forwards-compatible APi when the ghosts are 
written.

Quoting: http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg37134.html

I'm overhauling how we expose FMS related data to Nasal, to hopefully make 
FMS/CDU construction in Nasal easier, but also re-using the C++ code we already 
have, since many systems tie into the data.

Along the way, I'm learning a lot about the easiest way to expose C++ data to 
Nasal, and I'm also trying to reduce code duplication. So where we have an 
fg-command for an operation, I'll try to wrap the command in a nice Nasal 
syntax. (With some exceptions).

As part of this, there's various features that I'd like to clean up, but I need 
to be careful about compatibility. I'll list them below, but in general the 
question is how conservative or aggressive I can be in changing this stuff now, 
given it's 50 days until freeze - and more than 100 until release.

The things I'd propose to remove / change / break:

1)
        on navinfo() results, I want to remove the 'distance' and 'bearing' 
values, since these depend on the query position, and can be computed easily 
*if necessary* by the new courseAndDistance method. (Which works for any 
waypoint / navaid / airport / geo.Coord / etc)

        [as far as I can tell, nothing in fgdata uses this feature anyway]

2) 
        on airportinfo() results, I'd like to make the runways hash be a 
function, instead of a child hash. This will mean it can be generated lazily, 
and since airports have many runways, and we have quite a lot of data in the 
runways hash, it would save a ton of hash-building.

        In practice the only change to code would be calling apt.runways() 
instead of apt.runways, but that will still break aircraft that use the old 
syntax...

        [the only users of this data I can find in fgdata are GUI dialogs which 
are easy to fix, and actually would benefit from a revised API to get a single 
runway by ident, which I plan to add - but I assume this is the feature other 
FMS/CDU scripts are most likely using ....]

3)
        The route-manager has an internal 'command' API to manipulate the route 
- this is used by the route-manager dialog. With the new Nasal functions, this 
could be replaced with nicer Nasal calls. The question is if anyone has code 
which is using this API - it's been around for a long time. Again, in fgdata, 
no one is.

1) is low risk, I will almost certainly do it.
2) I would really like to do, but it really depends on the fall-out.
3) is less important, since the old interface can remain in parallel

A standard interface for mapping C++ classes to Nasal objects

There's a plan to eventually generalize and unify all those helpers in FGPositioned.cxx and provide them as part of the FGNasalSys class (or some custom abstract base class that C++ classes should implement to expose objects to Nasal), so that they automatically become available to all users of the Nasal subsystem, just by implementing a certain interface.

Looking at the code in NasalPositioned.cxx:

The function bodies for these helpers are largely identical:

  • naRef ghostForPositioned(naContext c, const FGPositioned* pos);
  • naRef ghostForAirport(naContext c, const FGAirport* apt)
  • naRef ghostForNavaid(naContext c, const FGNavRecord* n)
  • naRef ghostForRunway(naContext c, const FGRunway* r)
  • naRef ghostForWaypt(naContext c, const flightgear::Waypt* wpt)

So they could be replaced by a single template:

template <class T>
static T* 
getGhost(naRef r)
{      
 if( naGhost_type(r) == &T::GhostType)
  return (T*) naGhost_ptr(r);
  return 0;
}

This would only require adding a new naGhostType member to the FGPositioned class (and automatically to FGAirport, FGNavRecord, FGRunway because they are derived from FGPositioned).

So it would make sense to simply use a new "FGNasalWrapper" class, add it to the top of NasalSys.hxx, i.e. for C++ classes that are intended to be exposed to Nasal ?

class FGNasalWrapper {
public:
protected:
 naGhostType GhostType;
private:
};

This would provide an opportunity to come up with an interface that is required to expose C++ classes as Nasal ghosts.

And then, by changing positioned.hxx such that it derives from the new "FGNasalWrapper" class:

class FGPositioned : public SGReferenced, public FGNasalWrapper { 
// ...
};

Once each instance/subclass of FGPositioned, has such a "GhostType" field, the getMember() wrappers you created at the top of NasalPositioned.hxx, could be moved to the new FGNasalWrapper class, so that the GhostType field could be initialized by the ctor, using its own getMember() method:

class FGNasalWrapper {
public:
 FGNasalWrapper(void *ghost_destructor, const char* ghost_name
 const char* getMember(naContext c, void* g, naRef field, naRef* out);
protected:
 naGhostType GhostType;
private:
};

The ghost_destructor could also be implemented as a method of FGNasalWrapper (rather than being a global void* function), so that each C++ class that is to be exposed to Nasal, would only need to implement the class-specific methods.

Ultimately, this would also make it possible to reimplement the ghostFor* helpers as a template: http://gitorious.org/fg/flightgear/blobs/78afdb3c2227e75fc1542b3992dcea26181d98cf/src/Scripting/NasalPositioned.cxx#line139

  • ghostForPositioned();
  • ghostForAirport();
  • ghostForNavaid();
  • ghostForRunway();
  • ghostForWaypt();

So that the "getGhost" method could be a part of the NasalWrapper class:

template <class T>
naRef getGhost(naContext c, const T* pos)
{
  if (!pos) {
    return naNil();
  }
  T::get(pos); // take a ref
  return naNewGhost2(c, &T::GhostType, (void*) pos);
}

Passing Pointers to Nasal scripts

Things get more complicated if you need to pass a handle to a C/C++ object into a Nasal script. There, you need to use a wrapped handle type called a ghost ("Garbage-collectable Handle to an OutSide Thing"), which has a callback that you need to implement to deal with what happens when the Nasal interpreter garbage collects your object.

Ghost utilities:

 typedef struct naGhostType {
     void(*destroy)(void*);
     const char* name;
 } naGhostType;
 naRef        naNewGhost(naContext c, naGhostType* t, void* ghost);
 naGhostType* naGhost_type(naRef ghost);
 void*        naGhost_ptr(naRef ghost);
 int          naIsGhost(naRef r);

This needs to be specifically implemented for each new "ghost" type.

For an example of how something like this is done in the FlightGear context, you may want to check out $FG_SRC/Scripting/nasal-props.cxx, which wraps the SGPropertyNode class (SimGear) inside a nasal ghost, so that the C++ class is exposed to Nasal scripts.

Locking a Nasal context (threading)

// Acquires a "modification lock" on a context, allowing the C code to
// modify Nasal data without fear that such data may be "lost" by the
// garbage collector (nasal data on the C stack is not examined in
// GC!).  This disallows garbage collection until the current thread
// can be blocked.  The lock should be acquired whenever nasal objects
// are being modified.  It need not be acquired when only read access
// is needed, PRESUMING that the Nasal data being read is findable by
// the collector (via naSave, for example) and that another Nasal
// thread cannot or will not delete the reference to the data.  It
// MUST NOT be acquired by naCFunction's, as those are called with the
// lock already held; acquiring two locks for the same thread will
// cause a deadlock when the GC is invoked.  It should be UNLOCKED by
// naCFunction's when they are about to do any long term non-nasal
// processing and/or blocking I/O.  Note that naModLock() may need to
// block to allow garbage collection to occur, and that garbage
// collection by other threads may be blocked until naModUnlock() is
// called.  It must also be UNLOCKED by threads that hold a lock
// already before making a naCall() or naContinue() call -- these
// functions will attempt to acquire the lock again.
void naModLock();
void naModUnlock();

Things to avoid

And just for completeness: things get *really* complicated if you need to *store* a naRef you got from a Nasal script in a place where the interpreter can't see it.

There is a minimal API to handle this:

// "Save" this object in the context, preventing it (and objects
// referenced by it) from being garbage collected.
void naSave(naContext ctx, naRef obj);


But this API doesn't yet have a corresponding call to release a saved object. Just don't do this, basically. Always keep your Nasal data in Nasal space.

If you really need to do this, you may however want to check out the gcSave/gcRelease methods of the FGNasalSys class:

   // This mechanism is here to allow naRefs to be passed to
   // locations "outside" the interpreter.  Normally, such a
   // reference would be garbage collected unexpectedly.  By passing
   // it to gcSave and getting a key/handle, it can be cached in a
   // globals.__gcsave hash.  Be sure to release it with gcRelease
   // when done.
   int gcSave(naRef r);
   void gcRelease(int key);