Howto:Serializing Nasal data structures

From FlightGear wiki
Revision as of 11:49, 13 August 2017 by Hooray (talk | contribs) (→‎Objective)
Jump to navigation Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.
This article is a stub. You can help the wiki by expanding it.

Objective

  1. serialization: convert a Nasal data structure to a string using valid Nasal syntax
  2. deserialization: compile that string to a temporary Nasal function, call it to retrieve the original data structure

Background

Nasal's hash syntax is already close enough to JSON that Nasal could even serialie a property tree without requiring much work, analogous to props.getValues() which is already traversing a property tree node and which turns it into a Nasal data structure (hash).

[1]


Another possibility would be using a subset of JSON, i.e. to serialie/unserialize Nasal data structures as scalars, vectors and hashes - which should be pretty straightforward. The format would then be plain text, and it would just be evaluated via Nasal's compile()/call() APIs to turn data structures back into Nasal types.[2]

you could then also directly use io.load_nasal() to load a Nasal file from disk (think mydata.nas) and for serialiation purposes, write to the same file - just make sure that it is valid Nasal[3]


depending on how much can be saved using a space-time or time-memory tradeoff, you could compute lookup tables using Nasal vectors/hashes (or both) and serialie those to disk - e.g. using JSON, which is basically equivalent to the syntax used by Nasal. You would basically compute your lookup tables once and then use a nested for-loop to read the lookup tables and serialize/write those to a custom file in $FG_HOME/Export, at that point you can then copy the whole thing into a folder where it can be loaded at runtime using the helpers available in io.nas (using JSON would be much less verbose than using PropertyList/XML, but obviously not as efficient as using a binary serialization format). Basically, this would be equivalent to programmatically creating a complex texture and storing that on disk to save reduce the runtime overhead (treating the texture as a multi-dimensional lookup table). https://en.wikipedia.org/wiki/Space%E2%80%93time_tradeoff

https://en.wikipedia.org/wiki/JSON[4]


Proof of concept

Nasal Console

var serializeVector = func(vector) {
retVal = "[";
foreach(var item; vector) {
retVal ~= serialize(item) ~ ',';
} 

return retVal~"]";
}

var serializeHash = func(hash) {
var retVal = "{";
foreach(var member; keys(hash)) {
retVal ~= serialize(member) ~ ',';

}

return retVal ~"}";
}


var serialize = func (data) {
var type = typeof(data);
if (type == "scalar") return data;
if (type == "vector") return serializeVector(data);
if (type == "hash") return serializeHash(data);
if (type == "ghost" or type == "func") return "nil";
die("cannot serialize unsupported data type:", type);
} 


# create an empty vector
var myVector = [];

# populate the vector with elements with [index, index^2]
for (var i=0;i<=40;i+=1) {
append(myVector,[i, i*i]);
}



# debug.dump(myVector);

# serialize the data structure to a string that is valid Nasal syntax
var serialized = serialize(myVector);

# debug.dump(serialized);

# next, compile the serialized string to a Nasal function 
var compiled = compile(serialized);
# actually call the function, and store the return value 
var result = compiled();

# dump the deserialized return value to the console
debug.dump(result);

Finally

the original Nasal example, already contains a helper function to serialie a Nasal type:[5]

http://plausible.org/nasal/sample.nas

##
## A rockin' metaprogramming hack.  Generates and returns a deep copy
## of the object in valid Nasal syntax.  A warning to those who might
## want to use this: it ignores function objects (which cannot be
## inspected from Nasal) and replaces them with nil references.  It
## also makes no attempt to escape special characters in strings, which
## can break re-parsing in strange (and possibly insecure!) ways.
##
dump = func(o) {
    result = "";
    if(typeof(o) == "scalar") {
        n = num(o);
        if(n == nil) { result = result ~ '"' ~ o ~ '"'; }
        else { result = result ~ o; }
    } elsif(typeof(o) == "vector") {
        result = result ~ "[ ";
        if(size(o) > 0) { result = result ~ dump(o[0]); }
        for(i=1; i<size(o); i=i+1) {
            result = result ~ ", " ~ dump(o[i]);
        }
        result = result ~ " ]";
    } elsif(typeof(o) == "hash") {
        ks = keys(o);
        result = result ~ "{ ";
        if(size(o) > 0) {
            k = ks[0];
            result = result ~ k ~ ":" ~ dump(o[k]);
        }
        for(i=1; i<size(o); i=i+1) {
            k = ks[i];
            result = result ~ ", " ~ k ~ " : " ~ dump(o[k]);
        }
        result = result ~ " }";
    } else {
        result = result ~ "nil";
    }
    return result;
}


References

References
  1. Hooray  (Oct 9th, 2016).  Re: Why not use Erlang instead of .
  2. Hooray  (Sep 12th, 2015).  Re: String manipulation and file I/O (pm2thread) .
  3. Hooray  (Sep 12th, 2015).  Re: String manipulation and file I/O (pm2thread) .
  4. Hooray  (Aug 12th, 2017).  Re: Space Shuttle .
  5. Hooray  (Sep 12th, 2015).  Re: String manipulation and file I/O (pm2thread) .