20,741
edits
Nasal Namespaces in-depth: Difference between revisions
Jump to navigation
Jump to search
no edit summary
No edit summary |
No edit summary |
||
| Line 4: | Line 4: | ||
= Nasal Namespaces: in-depth = | = Nasal Namespaces: in-depth = | ||
I am not an expert in scripting languages, but I do claim that Nasal has a unique notion of namespaces. Rather than dwell on that, I'll tell you how they work in Nasal ( | I am not an expert in scripting languages, but I do claim that Nasal has a unique notion of namespaces. Rather than dwell on that, I'll tell you how they work in Nasal (and FlightGear) and leave you to decide for yourself. | ||
== Namespaces in FlightGear == | == Namespaces in FlightGear == | ||
If you look at the graph on the right, you will see a representation of how namespaces look in FlightGear. At the top there is the global namespace, called "globals", and various other namespaces branch down from there. On the left there are the namespaces created from the modules in $FG_ROOT and $FG_HOME, e.g. controls.nas makes the namespace "controls". In the center there are the ‘special’ namespaces, for the joystick(s) and the keyboard ( | If you look at the graph on the right, you will see a representation of how namespaces look in FlightGear. At the top there is the global namespace, called "globals", and various other namespaces branch down from there. On the left there are the namespaces created from the modules in $FG_ROOT and $FG_HOME, e.g. controls.nas makes the namespace "controls". In the center there are the ‘special’ namespaces, for the joystick(s) and the keyboard (there is only support for one keyboard hence only one namespace). Everything found in the joystick or keyboard files are executed in their respective namespaces – the <nasal><script> section at the top and all of the <binding>s. Next are the GUI dialogs, which have "__dialog:" as a prefix in front of the dialog's name (which I believe comes from the filename). | ||
As you look at the tree structure, notice how one can move around on it. One can go up to a more global namespace or one can go down to a sub-namespace of the current namespace. Each "leaf" or "branch" only has one parent (e.g. the only thing above "controls" is "globals") while they can contain multiple children (the reverse of the previous comment applies, globals contains not only controls but io, gui, etc., as well). | As you look at the tree structure, notice how one can move around on it. One can go up to a more global namespace or one can go down to a sub-namespace of the current namespace. Each "leaf" or "branch" only has one parent (e.g. the only thing above "controls" is "globals") while they can contain multiple children (the reverse of the previous comment applies, globals contains not only controls but io, gui, etc., as well). | ||
| Line 17: | Line 17: | ||
== Namespace Lookup == | == Namespace Lookup == | ||
Nasal starts looking for the leftmost side of the name – "controls." First it checks our current namespace, __js0, and since it doesn't find it there (we hope!) it then has to recurse up into the "parent" namespace, which is globals. It finds a controls namespace there and goes into it and looks for "gearDown." Upon finding it, Nasal then executes that script with "1" as an argument, and the gear lowers. Thus it is really two separate lookups – Nasal looks for controls and then gearDown inside that. For the first lookup, it is looking for a variable and can recurse into higher namespaces, for the second it can only go down from the namespace it found. This recursion into other namespaces is kind of like the lookup into hashes' "parents" vector, any "get" operations go look in the hash proper, then into the first "parent" index (though namespaces only have one parent), and any parents of the parents, etc., until something is found. Unlike hashes, however, "set" operations do not always stay in the actual hash, | Nasal starts looking for the leftmost side of the name – "controls." First it checks our current namespace, __js0, and since it doesn't find it there (we hope!) it then has to recurse up into the "parent" namespace, which is globals. It finds a controls namespace there and goes into it and looks for "gearDown." Upon finding it, Nasal then executes that script with "1" as an argument, and the gear lowers. Thus it is really two separate lookups – Nasal looks for controls and then gearDown inside that. For the first lookup, it is looking for a variable and can recurse into higher namespaces, for the second it can only go down from the namespace it found. This recursion into other namespaces is kind of like the lookup into hashes' "parents" vector, any "get" operations go look in the hash proper, then into the first "parent" index (though namespaces only have one parent), and any parents of the parents, etc., until something is found. Unlike hashes, however, "set" operations do not always stay in the actual hash, Nasal first tries to find the variable the same way as it does with "get" operations, but then if it isn't found it creates one in the current namespace. With the 'var' keyword, however, it doesn't look anywhere but just creates a new one in the current namespace. Namespaces also resemble hashes in another way – they in fact are hashes! Nasal thinks of namespaces as just another hash, and this challenges a common notion of programmers, that of "lvalues." Typically (I think?), programmers can only declare named variables that are legal lvalues – that is, they have to fit a certain pattern. The pattern is usually something like this: a name can start with an underscore or alpha character and all remaining characters must be an underscore or an alphanumeric character. The lvalue ends at the first character that doesn't fit that pattern, e.g. a punctuation mark. But in Nasal hashes, one can use arbitrary scalars (strings and/or numbers) as indexes: | ||
a_hash["(foo)"] = 78; | a_hash["(foo)"] = 78; | ||
This means that namespaces can contain ‘variables’ that aren't valid lvalues and thus can't be used in typical code. Also notice that the dialog's namespaces are not lvalues and thus can't be used like the controls namespace can. Instead, one must use various methods to access non-lvalues. For more, see the "Doing more with your namespace – Nasal style" section. | This means that namespaces can contain ‘variables’ that aren't valid lvalues and thus can't be used in typical code. Also notice that the dialog's namespaces are not lvalues as they have a colon and thus can't be used like the controls namespace can. Instead, one must use various methods to access non-lvalues. For more, see the "Doing more with your namespace – Nasal style" section. | ||
== | == Namespaces in Functions == | ||
Every single namespace in Nasal is just a hash. The most common namespaces are the ones inside the Nasal modules ($FG_ROOT/Nasal) and the globals namespace, these are probably the easiest to understand. In fact everything needs to run in namespace, and that includes joystick bindings. But less understood is the fact that every function creates it's own anonymous namespace to run in, e.g. gearDown in controls.nas has its own namespace that it runs in. These namespaces are not assigned a name like the controls namespace is. Instead, they simply exist as a part of the function and can only be fetched using closure() level 0. Anyways, this has the obvious implications that any variables created inside the function stay in that function (and get destroyed after the function returns). A function can also modify variables in the outer scope, either by leaving out the 'var' qualifier or by using caller(). Local variables cannot be accessed from the outer scope, as far as I know, though I am not sure what closure returns for such things. This notion of functions creating new namespaces has some interesting implications, see "Advanced namespace hacking: security wrappers". | Every single namespace in Nasal is just a hash. The most common namespaces are the ones inside the Nasal modules ($FG_ROOT/Nasal) and the globals namespace, these are probably the easiest to understand. In fact everything needs to run in namespace, and that includes joystick bindings. But less understood is the fact that every function creates it's own anonymous namespace to run in, e.g. gearDown in controls.nas has its own namespace that it runs in. These namespaces are not assigned a name like the controls namespace is. Instead, they simply exist as a part of the function and can only be fetched using closure() level 0. Anyways, this has the obvious implications that any variables created inside the function stay in that function (and get destroyed after the function returns). A function can also modify variables in the outer scope, either by leaving out the 'var' qualifier or by using caller(). Local variables cannot be accessed from the outer scope, as far as I know, though I am not sure what closure returns for such things. This notion of functions creating new namespaces has some interesting implications, see "Advanced namespace hacking: security wrappers". | ||
== Dealing with | == Dealing with Different Namespaces at Once == | ||
Anyways, back to the graph. Each portion of the graph has its own variables, some of which might be functions with their own namespaces, and some are hashes that are used as objects – not namespaces. We learned that the Nasal parser looks first at the current namespace for a symbol, then the next higher namespace, until it reaches the top. But what if we define a variable in this scope and we want to access another variable that has the same name but lives in a different namespace? This is what happens in debug.nas; it defines a string function which obscures the string namespace (in computer science this is called [en.wikipedia.org/ | Anyways, back to the graph. Each portion of the graph has its own variables, some of which might be functions with their own namespaces, and some are hashes that are used as objects – not namespaces. We learned that the Nasal parser looks first at the current namespace for a symbol, then the next higher namespace, until it reaches the top. But what if we define a variable in this scope and we want to access another variable that has the same name but lives in a different namespace? This is what happens in debug.nas; it defines a string function which obscures the string namespace (in computer science this is called [[en.wikipedia.org/Variable_Shadowing_|shadowing]]). To use the string namespace, there are two options: use caller() to manually "hop" up the namespace tree or prefix "globals[dot]" to the symbol like this: | ||
globals.string.isascii(n); | globals.string.isascii(n); | ||
The caller method is not recommended because it requires a different level for just being in debug.nas and being inside a function | The caller method is not recommended because it requires a different level for just being in debug.nas and being inside a function in debug.nas, and it also obfuscates the purpose of the code unless a comment is put in. Arguably neither solution looks good, but the only other option is to avoid using those symbols – which could sacrifice the usability and clarity of the API to access those functions in that namespace. That said, if a namespace declares a globals variable (e.g. like props.nas), then it will have to use caller() to get into the global namespace, as any reference to "globals" will stop in the current namespace not the actual global namespace. | ||
Here is the reason why prefixing "globals" even works (be warned that it doesn't hold for other Nasal implementations!): | |||
<syntaxhighlight lang="php"> | |||
-+- [flightgear]/src/Scripting/NasalSys.cxx, line 634 -+- | |||
// Start with globals. Add it to itself as a recursive | |||
// sub-reference under the name "globals". This gives client-code | |||
// write access to the namespace if someone wants to do something | |||
// fancy. | |||
_globals = naInit_std(_context); | |||
naSave(_context, _globals); | |||
hashset(_globals, "globals", _globals); | |||
</syntaxhighlight> | |||
Since it is recursive, one can prefix globals as many times as they want to without any change, it also leads to a stack overflow when debug.dump() tries to dump the global namespace (there are a couple hacks to fix this, though). | |||
== Namespaces and Security == | == Namespaces and Security == | ||
Advanced namespace hacking: security functions | Advanced namespace hacking: security functions. | ||
Consider what we learned: functions are anonymous and they create new namespaces. Now consider this: anonymous namespaces. Nasal can create these by making anonymous functions and immediately evaluating it. Let's look at an example from io.nas to create a layer of security over removelistener(): | Consider what we learned: functions are anonymous and they create new namespaces. Now consider this: anonymous namespaces. Nasal can create these by making anonymous functions and immediately evaluating it. Let's look at an example from io.nas to create a layer of security over removelistener(): | ||
<syntaxhighlight lang="php"> | <syntaxhighlight lang="php"> | ||
# wrap removelistener | # wrap removelistener | ||
globals.removelistener | globals.removelistener = (func { | ||
var _removelistener = globals.removelistener; | var _removelistener = globals.removelistener; | ||
func(n) { | func(n) { | ||
| Line 47: | Line 58: | ||
})(); | })(); | ||
</syntaxhighlight> | </syntaxhighlight> | ||
It redefines the removelistener() function inside of an anonymous namespace. It first copies the function that was previously defined, whether it was implemented in C or if it was another wrapper. Notice how this cannot be accessed outside of the namespace. It then defines a function (which is implicitly returned) that checks some conditions and either passes the argument and returns the result of the call to the original removelistener function, or it dies and denies access to the original function. This cleverly removes any access to the original function but it still allows access to it – after adding a layer of security. | |||
== Doing more with your namespaces, Nasal style. == | |||
== Doing more with your namespaces, Nasal style.== | |||
The Nasal library has a number of interesting functions to deal with namespaces and calling functions. These are: | The Nasal library has a number of interesting functions to deal with namespaces and calling functions. These are: | ||
| Line 61: | Line 71: | ||
The function call() gives enormous control over how a function is executed, from the 'me' variable and the arguments to the namespace it will be executed in. The ‘official’ prototype given is this: | The function call() gives enormous control over how a function is executed, from the 'me' variable and the arguments to the namespace it will be executed in. The ‘official’ prototype given is this: | ||
call(fn, args=[], me=nil, namespace=nil, error=nil) | call(fn, args=[], me=nil, namespace=nil, error=nil) | ||
However this is not accurate, as in fact the error vector has a movable position. For each of the optional arguments, if they are not the right type (vector for "args" and "error", hash for "me" and "namespace") then it is treated as nil/non-existent. The error vector is in fact the last argument if there are more than two arguments, and thus all of these are valid: | However this is not quite accurate, as in fact the error vector has a movable position. For each of the optional arguments, if they are not the right type (vector for "args" and "error", hash for "me" and "namespace") then it is treated as nil/non-existent. The error vector is in fact the last argument if there are more than two arguments, and thus all of these are valid: | ||
* call(fn, arg); | * call(fn, arg); | ||
* call(fn, arg, var err=[]); | * call(fn, arg, var err=[]); | ||
* call(fn, arg, me, var err=[]); | * call(fn, arg, me, var err=[]); | ||
* call(fn, arg, me, namespace, var err=[]); | * call(fn, arg, me, namespace, var err=[]); | ||
The second and third of which are respectively equivalent to: | The second and third of which are respectively equivalent to: | ||
* call(fn, arg, nil, nil, var err=[]); | * call(fn, arg, nil, nil, var err=[]); | ||
* call(fn, arg, me, nil, var err=[]); | * call(fn, arg, me, nil, var err=[]); | ||
For the record, I am not sure how namespaces recurse when set by call()/naCall (meaning what namespace they search in next when they don't find the symbol in this namespace). The last function – bind() – allows manual setting of the namespace and parent function that the function runs in (the parent function, called ‘next’, is where the function is originally defined (I think), i.e. it would be the controls namespace for gearDown). Let's see some of these in action: | |||
Let's say you are making your own class in Nasal and you have a gazillion members you new() method needs to populate. Each member is passed in as an argument which gets tiring for you (also limits you to 16 members, as functions can't have more arguments than that). This is what a first attempt might look like: | |||
Let's say you are making your own class in Nasal and you have a gazillion members you new() method needs to populate. Each member is passed in as an argument which gets tiring for you (also limits you to 16 members, as functions can't have more arguments than that). This is what | |||
<syntaxhighlight lang="php"> | <syntaxhighlight lang="php"> | ||
var class = { | var class = { | ||
| Line 89: | Line 95: | ||
}; | }; | ||
</syntaxhighlight> | </syntaxhighlight> | ||
And it goes on and on. What if you could just do it automatically? Your first attempt might go like this:<syntaxhighlight lang="php"> | |||
And it goes on and on. What if you could just do it automatically? Your first attempt might go like this: | |||
<syntaxhighlight lang="php"> | |||
var class = { | var class = { | ||
new: func(a, b, c, d, e) { | new: func(a, b, c, d, e) { | ||
| Line 102: | Line 106: | ||
}; | }; | ||
</syntaxhighlight> | </syntaxhighlight> | ||
But this is still a little inefficient as you have to manually add each symbol. There are two ways around this. Firstly, you could do a foreach on the symbols in the namespace and ignore the symbols if it is "m" – that is, our hash we are populating. Or you could wrap that in a function which generates the hash itself and doesn't need to skip it. Here they are in order: | But this is still a little inefficient as you have to manually add each symbol. There are two ways around this. Firstly, you could do a foreach on the symbols in the namespace and ignore the symbols if it is "m" – that is, our hash we are populating. Or you could wrap that in a function which generates the hash itself and doesn't need to skip it. Here they are in order: | ||
<syntaxhighlight lang="php"> | <syntaxhighlight lang="php"> | ||
var class = { | var class = { | ||
new: func(a, b, c, d, e) { | new: func(a, b, c, d, e) { | ||
var m = { | var m = { parents: [me] }; | ||
foreach (var sym; caller(0)[0]) { | foreach (var sym; caller(0)[0]) { | ||
if (sym == "m") continue; | if (sym == "m" or sym == "me" or sym == "arg") continue; | ||
m[sym] = caller(0)[0][sym]; | m[sym] = caller(0)[0][sym]; | ||
} | } | ||
| Line 116: | Line 118: | ||
} | } | ||
}; | }; | ||
</syntaxhighlight> | |||
<syntaxhighlight lang="php"> | |||
var _gen_new = func() { | var _gen_new = func() { | ||
var symbols = caller(1)[0]; | var symbols = caller(1)[0]; | ||
var m = { parents: [symbols["me"]] }; | var m = { parents: [symbols["me"]] }; | ||
foreach (var sym; symbols) { | foreach (var sym; symbols) { | ||
if (sym == "me" or sym == "arg") continue; | |||
m[sym] = symbols[sym]; | m[sym] = symbols[sym]; | ||
} | } | ||
| Line 131: | Line 135: | ||
}; | }; | ||
</syntaxhighlight> | </syntaxhighlight> | ||
The advantage of the second is that you can use it for any new() method without duplicating code. | The advantage of the second is that you can use it for any new() method without duplicating code, it also has less symbols it has to avoid. | ||