|
|
Line 1: |
Line 1: |
| {{cleanup|reason=broken quoting, see [[Fixing up articles with wrong quotes]] for a browser extension to fix things}}
| |
| {{Template:Nasal Internals}}
| |
|
| |
| Last update: 10/2011
| |
|
| |
| As more and more code in FlightGear is moved to the base package and thus implemented in Nasal space, some Nasal related issues have become increasingly obvious.
| |
|
| |
| On the other hand, Nasal has a proven track record of success in FlightGear, and has shown remarkably few significant issues so far. Most of the more prominent issues are related to a wider adoption in FlightGear, and thus more complex features being implemented in Nasal overall.
| |
|
| |
| So, rather than having Nasal flame wars and talking about "alternatives" like Perl, Python, Javascript or Lua, the idea is to document known Nasal issues so that they can hopefully be addressed eventually.
| |
|
| |
| If you are aware of any major Nasal issues that are not yet covered here, please feel free to add them here, however it is also a good idea to use the FlightGear bug tracker in such cases: http://flightgear-bugs.googlecode.com/
| |
|
| |
| = Explicit loading order and reset support =
| |
| See [http://flightgear.org/forums/viewtopic.php?f=30&t=19487].
| |
|
| |
| = Improve the garbage collector =
| |
|
| |
| As of 06/2012, a new incremental Nasal GC is being worked on [http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg37579.html]:
| |
|
| |
| I have been working on a 4-color incremental mark/sweep collector with
| |
| the intention of merging it into the Nasal interpreter.
| |
|
| |
| The work so far can be found at http://github.com/chrisforbes/incgc;
| |
| There's still quite a lot to do, but the path is clear.
| |
|
| |
|
| |
| Also see: [[How the Nasal GC works]]
| |
|
| |
| Year: 2011-2012
| |
|
| |
|
| |
| URL: http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg33190.html
| |
|
| |
| '''Problem:''' Nasal has a garbage collection problem. One solution to it is - we avoid
| |
| Nasal code wherever possible and try to hard-code everything. But Nasal
| |
| crops up on a lot of places - complex aircraft such as the Concorde come
| |
| to my mind, interactive AI models, lots of really nifty and useful
| |
| applications... - so instead of fixing things in a lot of places, one
| |
| could also think about it the other way and fix just one thing, i.e. the
| |
| garbage collection such that it doesn't hit a single frame. I fully well
| |
| realize that dragging out complicated operations across many frames while
| |
| everything else keeps changing is at least an order of magnitude more
| |
| complicated (about 1/3 of Local Weather deal with precisely that
| |
| problem...) - but I don't believe it can't be done at all. It sort of bugs
| |
| me a bit that somehow the fault is always supposed to be in using Nasal...
| |
|
| |
| I think it's great if we have a discussion where the issues are placed on
| |
| the table to give everyone the change to learn and understand more, and
| |
| then reasonably decide what to do. Nasal has advantages and disadvantages,
| |
| so has C++, sometimes accessibility and safety are worth a factor 3
| |
| performance (to me at least), sometimes not. But I don't really want to
| |
| discuss dogmatics where 'truth' is a priori clear. There is a case for
| |
| having high-level routines in Nasal, there's a case to be made to switch
| |
| low level workhorses to C++ - and there's always the question of what is
| |
| the most efficient way of doing something. But I'm clearly not considering
| |
| Nasal-based systems immature or experimental per se.
| |
|
| |
| URL: http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg31918.html
| |
|
| |
| As discussed in "Stuttering at 1 Hz rate" we now know that regular and
| |
| unpleasant stuttering is caused by Nasals garbage collector.
| |
| So I thought about possibilities to improve it.
| |
| What if we could decouple the following function as a separate thread, so
| |
| that it runs *asynchronously* from the main thread?
| |
| This way it would not interfere (or much less) with the main thread and our
| |
| fps would be more consistent.
| |
|
| |
| This is the function causing the jitter:
| |
| In "simgear/nasal/gc.c"
| |
| static void garbageCollect()
| |
|
| |
| The thread will need to share some of the global variables from the main
| |
| thread.
| |
|
| |
|
| |
| URL: http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg31919.html
| |
|
| |
| I'm not an expert in nasal garbage collection, but I think the problem is
| |
| that garbage collection is not something we can divide up into chunks (which
| |
| is essentially what threading would do.) In addition, threading adds a lot
| |
| of potential order dependent bugs.
| |
|
| |
| In the case of nasal, I believe the garbage collection pass must be done in
| |
| a single atomic step, otherwise it would leave the heap in
| |
| an inconsistent state and adversely affect the scripts.
| |
|
| |
| URL: http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg31637.html
| |
| I don't know much about our Nasal implementation, but I suspect that
| |
| the garbage collector could be changed to trace only a portion of
| |
| Nasal's heap at each invocation, at the risk of increased memory use.
| |
|
| |
|
| |
| URL: http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg31921.html
| |
|
| |
| There are algorithms for incremental and/or concurrent and/or parallel
| |
| garbage collection out there. They most likely not easy to implement and
| |
| as far as I have seen so far would require (at least for concurrent and
| |
| /or parallel GC) all writes of pointers to the Nasal heap (and possibly
| |
| reads) to be redirected via wrapper functions (also known as
| |
| (GC) read/write barriers).
| |
|
| |
| This will not be an easy task but in my opinion it would be a promising
| |
| option. It might be possible to use a GC module from a GPL:d Java vm or
| |
| similar.
| |
|
| |
| Btw, just running the normal (mutually exclusive) Nasal GC in another
| |
| thread than the main loop is not hard - but since it is mutually exclusive
| |
| to executing Nasal functions it doesn't help much when it comes to
| |
| reducing the worst case latency.
| |
|
| |
| The small changes needed to add a separate GC thread are available here:
| |
| http://www.gidenstam.org/FlightGear/misc/test/sg-gc-2.diff
| |
| http://www.gidenstam.org/FlightGear/misc/test/fg-gc-1.diff
| |
|
| |
|
| |
| Also, I had a brief look at exactly which Nasal timers caused a jitter.
| |
| And the winner is...
| |
| ... well, any. Any Nasal timer, even if it's almost empty, will every
| |
| now and then consume a much larger amount of time than normal.
| |
| Seems to be a general issue with the Nasal execution engine: could be
| |
| triggered by Nasal's garbage collector, which every now and then needs
| |
| to do extra work - and runs within the context of a normal Nasal call.
| |
| It could also be a result of Nasal's critical sections: other threads
| |
| may acquire a temporary lock to alter Nasal data structures - which may
| |
| block the execution of Nasal timers at certain points. Hmm... Best
| |
| practices for debugging a multi-threaded program anyone? :)
| |
|
| |
| Concerning the frequency of the jitter: I guess it isn't related to the
| |
| FDM at all. It's probably just a result of Nasal complexity. The more
| |
| Nasal code is running, the more often/likely garbage collection /
| |
| blocking may occur. Frame rate may also influce it: many Nasal timers
| |
| run at delay 0 (in every update loop).
| |
|
| |
| URL: http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg37308.htmla
| |
|
| |
| A significant part of Nasal-related frame rate impact is
| |
| caused by garbage collection. Its delay/jitter only depends on the
| |
| number of Nasal objects and their references which need to be searched.
| |
| Increasing the number of Nasal objects (code+data) existing in memory
| |
| also increases the delay.
| |
|
| |
| The amount of Nasal code which is actually being executed only
| |
| influences the g/c frequency, i.e. whether the effect is visible every
| |
| few seconds vs several times per second.
| |
|
| |
|
| |
| URL: http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg37310.html
| |
|
| |
| I did look at incremental GC for Nasal last year, but couldn't find a 'simple
| |
| enough' generational algorithm. Still happy for someone else to try - the Nasal
| |
| GC interface is very clean and self-contained, so quite easy to experiment with
| |
| different GC schemes.
| |
|
| |
|
| |
| URL: http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg37338.html
| |
|
| |
| But as I said, I think really the GC needs to be addressed. There's only so
| |
| much hacking around the actual problem one can do.
| |
|
| |
| URL: http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg37338.html
| |
|
| |
| Right, a problem is that I've possibly studied all Nasal documentation I could
| |
| get without finding any reference of the GC problem - that was only transmitted
| |
| to me much later. I think you'll find that most Nasal users are not aware of
| |
| any such problems, because it's not documented anywhere. It doesn't help so
| |
| much if you are aware of it.
| |
|
| |
| URL: http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg37338.html
| |
|
| |
| the current GC is bad, and big Nasal shows this while small Nasal doesn't.
| |
|
| |
|
| |
| URL: http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg37338.html
| |
|
| |
|
| |
| We have an implementation of Nasal which dumps all the GC into a single frame
| |
| and is apparently sensitive to the total amount of code, regardless if the code
| |
| is actually run or not. This fact has historically not been widely advertized
| |
| or explained. That turns out to be a problem.
| |
|
| |
| The way this usually comes across is 'Advanced Weather causes stutter'. But it
| |
| actually doesn't really (or at least that remains to be shown) - what causes
| |
| stutter is mainly the GC, and Advanced Weather just happens to trigger this.
| |
| The range of suggested solutions in the past included almost everything, from
| |
| avoiding Nasal to porting code to Nasal to hacking around the problem to
| |
| loading things on-demand - except fixing the actual cause of the problems.
| |
|
| |
| I don't honestly know how complex code to collect garbage across many frames
| |
| is, but somehow I doubt that in terms of man-hours the effort beats porting the
| |
| existing large-scale Nasal codes to C++. Just my 2 cents in any case.
| |
|
| |
|
| |
| == Separate GC implementations ==
| |
| * [http://engineering.twitter.com/search?q=Garbage+Collector+ Ruby Enterprise Edition] - performance blog
| |
| * [http://www.hpl.hp.com/personal/Hans_Boehm/gc/ Boehm GC]
| |
| * [http://www.dekorte.com/projects/opensource/libgarbagecollector/ libgarbagecollector]
| |
| * [http://www.friday.com/bbum/2008/11/11/autozone-the-objective-c-garbage-collector/ AutoZone]
| |
| * http://mono-project.com/Generational_GC
| |
| * http://mono-project.com/Working_With_SGen
| |
| * http://www.mono-project.com/Compacting_GC
| |
| * http://www.utdallas.edu/~ramakrishnan/Projects/GC_for_C/index.htm
| |
|
| |
|
| |
| = Better debugging/development support = | | = Better debugging/development support = |
|
| |
|
| Besides making a full IDE (which would be ''really'' cool), there are several things that can be done by editing the source code of Nasal to enhance debugging support and increase development : | | Besides making a full IDE (which would be ''really'' cool), there are several things that can be done by editing the source code of Nasal to enhance debugging support and increase development : |
|
| |
|
| * Being able to dump the global namespace (see [http://flightgear.org/forums/viewtopic.php?f=30&t=19049&p=182930&#p182930 this topic] for a possible solution) or at least dump things prettily (an unreleased version of the file discussed in [[Nasal Meta-programming]] has good support for nice formatting). | | * Being able to dump the global namespace (see [http://flightgear.org/forums/viewtopic.php?f=30&t=19049&p=182930&#p182930 this topic] for a possible solution) or at least dump things prettily (an unreleased version of the file discussed in [[Nasal Meta-Programming]] has good support for nice formatting). |
| * Register a callback for handling errors using call() | | * Register a callback for handling errors using call() |
| * Register a callback for OP_FCALL et al. to be able to time function calls. Example (relies on the gen.nas module and debug.decompile): | | * Register a callback for OP_FCALL et al. to be able to time function calls. Example (relies on the gen.nas module and debug.decompile): |
| <syntaxhighlight lanh="php"> | | <syntaxhighlight lang="php"> |
|
| |
|
| # Debugging: | | # Debugging: |
Line 272: |
Line 68: |
|
| |
|
| </syntaxhighlight> | | </syntaxhighlight> |
| * Set breakpoints: not needed (just call a function to do what you want). | | * Set breakpoints: register callbacks for values of <tt>(struct Frame*)->ip</tt>. |
| * Time other parts of Nasal (not just VM) with a compile-time flag? | | * Time other parts of Nasal (not just VM) with a compile-time flag? |
| * Better error messages – in progress. | | * Better error messages – in progress. |
Line 291: |
Line 87: |
| *** Or one could look behind to see a naCode constant being pushed, and give some indication to its naFunc that it now has a name. This I still somewhat illegal, but not dangerous and thus could be done. | | *** Or one could look behind to see a naCode constant being pushed, and give some indication to its naFunc that it now has a name. This I still somewhat illegal, but not dangerous and thus could be done. |
| ** Option 3: abandon <tt>var foo = func(){}</tt> for ECMAscript-like function declaration syntax <tt>function foo() {}</tt>. This would not affect the use of anonymous func expressions but would instead be applicable in cases where we want to say "this function is static (i.e. permanent) and should have a name" (as opposed the the case of temporary storage variables for functions). Regardless of the method used, a name member will have to be added to naFunc's and the VM and error handling procedures will have to be changed according. | | ** Option 3: abandon <tt>var foo = func(){}</tt> for ECMAscript-like function declaration syntax <tt>function foo() {}</tt>. This would not affect the use of anonymous func expressions but would instead be applicable in cases where we want to say "this function is static (i.e. permanent) and should have a name" (as opposed the the case of temporary storage variables for functions). Regardless of the method used, a name member will have to be added to naFunc's and the VM and error handling procedures will have to be changed according. |
| | * Timing parts of VM: use the callbacks and systime/unit.time to time things. Need hooks into the GC as well. Statistics worth tracking: |
| | ** Per function: |
| | *** ncalls per frame/cumulative |
| | *** time per call on avg/cumulative |
| | *** min/max time |
| | *** number of GC invocations & avg/min/max time |
| | *** number of naNew() calls |
| | *** number & list of names of once-use variables; n that are numbers (i.e. non-GC-managed). |
| | ** Per context/global: |
| | *** time spent |
| | *** number of GC invocations: |
| | **** per frame |
| | **** min/max/avg |
| | ** Data per function call (not displayed, dumped to a file): |
| | *** caller line/name |
|
| |
|
|
| |
|