Nasal Namespaces in-depth

Namespaces in FlightGear

If you look at the graph up above, you will see a representation of how namespaces look in FlightGear. At the top there is the global namespace, called "globals", and various other namespaces branch down from there. On the left there are the namespaces created from the modules in $FG_ROOT and $FG_HOME, e.g. controls.nas makes the namespace "controls". In the center there are the ‘special’ namespaces, for the joystick(s) and the keyboard (there is only support for one keyboard hence only one namespace). Everything found in the joystick or keyboard files are executed in their respective namespaces – the <nasal><script> section at the top and all of the <binding>s. Next are the GUI dialogs, which have "__dialog:" as a prefix in front of the dialog's name (which I believe comes from the filename).

As you look at the tree structure, notice how one can move around on it. One can go up to a more global namespace or one can go down to a sub-namespace of the current namespace. Each "leaf" or "branch" only has one parent (e.g. the only thing above "controls" is "globals") while they can contain multiple children (the reverse of the previous comment applies, globals contains not only controls but io, gui, etc., as well).

Hands-On Example

Let's consider a common operation to perform. As I'm sure you all know, to put the gear down from a joystick button (which is in the namespace __js0 for example), you call a script like this:

controls.gearDown(1);

The "controls" means that namespace and the "[dot] gearDown" means that we want to access it's member "gearDown" (and then call it with the parenthesis). But how does this work? In the graph there clearly isn't a line between __js0 and controls that could take us there! But the key here is the other lines – we can first go up to globals from __js0 and then back down to controls, then we search for "gearDown" there. With this in mind, let me tell you what happens when Nasal interprets the above script:

Namespace Lookup

Nasal starts looking for the leftmost side of the name – "controls". First it checks inside our current namespace, __js0, and since it doesn't find it there (we hope!) it then has to recurse up into the "parent" namespace, which is globals. It then finds a controls namespace there and then goes into it and looks for "gearDown". Upon finding it, Nasal then executes that script with "1" as an argument, and the gear lowers. Thus it is really two separate lookups – Nasal looks for controls and then gearDown inside that. For the first lookup, it is looking for a variable and can recurse into higher namespaces; for the second it is looking for a member can only go down from the namespace it found. This recursion into other namespaces is kind of like the lookup into hashes' "parents" vector, any "get" operations go look in the hash proper, then into the first "parent" index (though namespaces only have one parent), and any parents of the parents, etc., until something is found, checking the second parent if the first fails. Unlike hashes, however, "set" operations do not always stay in the actual hash: Nasal first tries to find the variable the same way as it does with "get" operations and will set it if it is found, and only if it isn't found it creates one in the current namespace. With the 'var' keyword, however, it doesn't look anywhere but just creates a new one in the current namespace.

Namespaces also resemble hashes in another way – they in fact are hashes! Nasal thinks of namespaces as just another hash, and this can challenge a common notion of programmers, that of "lvalues". Typically, programmers can only declare named variables that are legal lvalues – that is, they have to fit a certain pattern. The pattern is usually something like this: a name can start with an underscore or alpha character and all remaining characters must be an underscore or an alphanumeric character. The lvalue ends at the first character that doesn't fit that pattern, e.g. a punctuation mark or bit of white space. But in Nasal hashes, one can use arbitrary scalars (strings or numbers) as hash indexes:

a_hash["(illegal lvalue)"] = 78;

This means that namespaces can contain ‘variables’ that aren't valid lvalues and thus can't be used in typical code. Also notice that the dialog's namespaces are not lvalues, as they have punctuation in them, and thus can't be used like the controls namespace can. Instead, one must use various methods to access non-lvalues, like indexing the globals namespace (though this is not recommended as they are supposed to be "private"):

globals["__dlg:foo"].setName("foo");

Namespaces in Functions

Every single namespace in Nasal is just a hash. The most common namespaces are the ones inside the Nasal modules ($FG_ROOT/Nasal) and the globals namespace, these are probably the easiest to understand. In fact everything needs to run in namespace, and that includes joystick bindings. But less understood is the fact that every function creates it's own anonymous namespace to run in for each call, e.g. each call to controls.gearDown gives it a new namespace. These namespaces are not assigned a name like the controls namespace is and in fact are only stored in Nasal's call stack. This has the obvious implications that any variables created inside the function stay in that function (and possibly get destroyed after the function returns). A function can also modify variables in its outer scope, either by leaving out the 'var' qualifier or by using closure().

This notion of functions creating new namespaces has some interesting implications, see "Advanced namespace hacking: security wrappers". It also has implications in creating closures for func{} expressions; more on this later as well.

Dealing with Different Objects that have the Same Name

Back to the graph: each portion of the graph has its own variables, some of which might be functions with their own namespaces, and some are hashes that are used as objects or namespaces. We learned that the Nasal code executor looks first at the current namespace for a symbol, then the next higher namespace, until it reaches the top. But what if we define a variable in this scope and we want to access another variable that has the same name but lives in a different namespace? This is what happens in debug.nas; it defines a string function which obscures the string namespace (in computer science this is called [en.wikipedia.org/Variable_Shadowing shadowing]). To use the string namespace, there are two options: use caller() to manually "hop" up the namespace tree or prefix "globals[dot]" to the symbol like this:

globals.string.isascii(n);

The caller method is not recommended because it requires a different level for just being in debug.nas and being inside a function in debug.nas, and it also obfuscates the purpose of the code unless a comment is put in. Arguably neither solution looks good, but the only other option is to avoid using those symbols – which could sacrifice the usability and clarity of the API to access those functions in that namespace. That said, if a namespace declares a globals variable (e.g. like props.nas), then it will have to use caller() to get into the global namespace, as any reference to "globals" will stop in the current namespace not the actual global namespace.

Here is the reason why prefixing "globals" even works (be warned that it doesn't hold for other Nasal implementations!):

-+- [flightgear]/src/Scripting/NasalSys.cxx, line 634 -+-
    // Start with globals.  Add it to itself as a recursive
    // sub-reference under the name "globals".  This gives client-code
    // write access to the namespace if someone wants to do something
    // fancy.
    _globals = naInit_std(_context);
    naSave(_context, _globals);
    hashset(_globals, "globals", _globals);

Since it is a symbol in the global namespace (which is actually kept in C++-space), code that needs it ends up recursing to there and then getting it again (but without any more recursions possible). Also since it is recursive, one can prefix globals as many times as they want to without any change; it also leads to a stack overflow when debug.dump() tries to dump the global namespace (there are a couple hacks to fix this, though).

Namespaces and Security

Advanced namespace hacking: security functions.

Consider what we learned: functions are anonymous and they create new namespaces. Now consider this: anonymous namespaces. Nasal can create these by making anonymous functions and immediately evaluating it. Let's look at an example from io.nas to create a layer of security over removelistener():

    # wrap removelistener
    globals.removelistener = (func {
        var _removelistener = globals.removelistener;
        func(n) {
            if (n != rval and n != wval)
                return _removelistener(n);

            die("removelistener(): removal of protected listener #'" ~ n ~ "' denied (unauthorized access)\n ");
        }
    })();

It redefines the removelistener() function inside of an anonymous namespace. It first copies the function that was previously defined, whether it was implemented in C or if it was another wrapper. Notice how this cannot be accessed outside of the namespace. It then defines a function (which is implicitly returned) that checks some conditions and either passes the argument and returns the result of the call to the original removelistener function, or it dies and denies access to the original function. This cleverly removes any access to the original function but it still allows access to it – after adding a layer of security. Note however that calling closure(removelistener, 0) will return our "anonymous" namespace – which can give access to the old removelistener via the _removelistener handle. This can be a security flaw, but redefining closure() can make the environment fully secure – see io.nas for how this is done.

Doing More with Your Namespaces, Nasal Style.

The Nasal library has a number of interesting functions to deal with namespaces and calling functions. These are:

• call()
• caller()
• closure()
• bind()

These all deal with namespaces is some way, shape, or form: closure() gives the namespace (closure) of the function at the specified level – more on this later; bind() can modify those closure (somewhat indirectly); caller() gives info about the caller: locals namespace, function, name of source file, currently executing line; and call() gives enormous control over how a function is executed, from the 'me' variable and the arguments to the namespace it will be executed in. The ‘official’ prototype given for the latter is this:

call(fn, args=[], me=nil, namespace=nil, error=nil)

However this is not quite accurate, as in fact the error vector has a movable position. For each of the optional arguments, if they are not the right type (vector for "args" and "error", hash for "me" and "namespace") then it is treated as nil/non-existent. The error vector is in fact the last argument if there are more than two arguments, and thus all of these are valid:

• call(fn, arg);
• call(fn, arg, var err=[]);
• call(fn, arg, me, var err=[]);
• call(fn, arg, me, namespace, var err=[]);

The second and third of which are respectively equivalent to:

• call(fn, arg, nil, nil, var err=[]);
• call(fn, arg, me, nil, var err=[]);

Let's see some of these in action:

Example: Class Constructor

Let's say you are making your own class in Nasal and you have a gazillion members you new() method needs to populate. Each member is passed in as an argument which gets tiring for you (also limits you to 16 members, as functions can't have more arguments than that). This is what a first attempt might look like:

var class = {
    new: func(a, b, c, d, e) {
        var m = { parents: [me] };
        m.a = a;
        m.b = b;
        m.c = c;
        m.d = d;
        m.e = e;
        return m;
    }
};

And it goes on and on. What if you could just do it automatically? Your first attempt might go like this:

var class = {
    new: func(a, b, c, d, e) {
        var m = { [parents: [me]] };
        foreach (var sym; ["a", "b", "c", "d", "e"]) {
            m[sym] = caller(0)[0][sym];
        }
        return m;
    }
};

But this is still a little inefficient as you have to manually add each symbol. There are two ways around this. Firstly, you could do a foreach on the symbols in the namespace and ignore the symbols if it is "m" – that is, our hash we are populating. Or you could wrap that in a function which generates the hash itself and doesn't need to skip it. Here they are in order:

var class = {
    new: func(a, b, c, d, e) {
        var m = { parents: [me] };
        foreach (var sym; caller(0)[0]) {
            if (sym == "m" or sym == "me" or sym == "arg") continue;
            m[sym] = caller(0)[0][sym];
        }
        return m;
    }
};

var _gen_new = func() {
    var symbols = caller(1)[0];
    var m = { parents: [symbols["me"]] };
    foreach (var sym; symbols) {
        if (sym == "me" or sym == "arg") continue;
        m[sym] = symbols[sym];
    }
    return m;
};
var class = {
    new: func(a, b, c, d, e) {
        return _gen_new();
    }
};

The advantage of the second is that you can use it for any new() method without duplicating code; it also has less symbols it has to avoid.

Wrap-Up: Namespaces as Closures

Finally we will consider how namespaces actually recurse. It turns out that recursion is specified in the function, not the namespace. This means that controls doesn't necessarily go to globals always, but it is up to the creator of the function object to decide.

In the C code, there is a naFunc object that represents a function. In addition to specifying the executable part (which is a naCode or naCCode) it specifies the namespace and another naFunc to get the next namespace from. These naFunc’s end up being chained together to form the namespace hierarchy – each link is referred to as a "closure". The builtin closure() function follows the chain through the links (where 0 specifies the first link, etc.) and returns thay closure – a small slice of the variables the function has access to. The builtin bind() does the opposite: it specifies the first link in the chain for that function (its embedded link to the namespace) and possibly another function that spcifies the next namespace and the next function, etc. (or nil if it only has one closure). See Nasal Meta-programing for more on how they can be used together, but the key thing to remember is that namespaces are built on the backs of functions by using hashes as their key datatype.

Nasal Namespace Browser

Philosopher's latest invention: a Nasal namespace browser

Philosopher created a GUI browser for viewing the Nasal state (Nasal being our built-in,JavaScript-like, FlightGear scripting language). It's sorta like the property browser, except in Canvas, and still a basic prototype, but already pretty useful for debugging purposes. Tested on 2.10, 2.11 and 2.12.

• Close button (or the escape key) closes the dialog, upon which point the code refreshes so that changes made will appear when it is next opened (rapid prototyping).

• Clicking on the title bar (next to the title) will go back in history
• clicking on an entry will go inside of that entry -- either a vector or a hash as both can be viewed (other types being displayed inline).
• Clicking and dragging on the displayed text will scroll the window (hey, it's still a prototype)

To use, place the file in Nasal/nasal_browser.nas and (preferably) add an entry to the menubar (the name tag should be "Toggle Nasal Browser" or similar and the Nasal script should be «nasal_browser.toggle_window()»).

To learn more, see http://forum.flightgear.org/viewtopic.php?f=71&t=20385