Talk:Resource Tracking for FlightGear

Nasal Callback duration

A while ago, I played with tracking the duration of Nasal callback execution, to get a list of scripts that tend to trigger the GC more frequently (i.e. higher "GC pressure"). And here, it's quite obvious that some scripts and function have a higher probability of triggering the Nasal GC than others - including some of your LW/AW loops, some of them trigger the GC very rarely, while others trigger it regularly. It shouldn't be too hard adapt that code and expose additional GC-internals to the property tree, so that we know how much memory is consumed per memory pool, and possibly even per loaded script or sub module.

VRAM tracking

looking at the wiki, the patch you've updated now should be fairly easy to extend for getting VRAM utilization, too - are you on NVIDIA or ATI/AMD hardware ?

Here are code snippets for AMD/ATI and NVIDIA, which can probably be copied directly into the ::update() method of the LinuxMemoryInterface class:

• http://www.geeks3d.com/20100531/programming-tips-how-to-know-the-graphics-memory-size-and-usage-in-opengl/
• http://www.gamedev.net/topic/640599-check-video-ram-usage/
• http://stackoverflow.com/questions/4552372/determining-available-video-memory
• https://www.opengl.org/discussion_boards/showthread.php/182034-Reasons-for-the-lack-of-an-API-to-query-memory-related-info

This would give you the amount of VRAM that is available/used, which should also be useful for troubleshooting/feature-scaling purposes.

the right way for doing this the "OOP way" would be to come up with a "GPUInfo" class and inherit from that 2-3 classes for ATI/AMD, NVIDIA and Intel - i.e. to be filled in with the stubs for each vendor:

    // this is the base class, with the only method being a virtual method 
    // which needs to be implemented by any child classes
    // a pointer of this class will be added to the LinuxMemoryInterface class
    class GPUInfo {
    private:
    protected:
    public:
    virtual int getVRAMUsageInKB() = 0;
    };

    // Actually implement the GPUInfo class for all 3 GPU vendors:

    class NVIDIA_GPU: public GPUInfo {
    public:
    virtual int getVRAMUsageInKB() {
    SG_LOG(SG_GENERAL, SG_ALERT,"NVIDIA VRAM tracking function not yet implemented !");
    return 100;
    }

    };

    class ATI_GPU: public GPUInfo {
    public:
    virtual int getVRAMUsageInKB() {
    SG_LOG(SG_GENERAL, SG_ALERT,"ATI VRAM tracking function not yet implemented !");
     return 200;
    }

    };

    class INTEL_GPU : public GPUInfo {
    public:
    virtual int getVRAMUsageInKB() {
    SG_LOG(SG_GENERAL, SG_ALERT,"Intel VRAM tracking function not yet implemented !");
     return 500;
    }

    };

(100/200 and 500 would be replaced with the corresponding code)

In the LinuxMemoryInterface::LinuxMemoryInterface() constructor, it would need to read the gpu-vendor property (as per the help/about dialog) and instantiate the correct class, by adding a pointer to it that holds the _gpu object:

    class LinuxMemoryInterface {
    +public:
    + LinuxMemoryInterface() {}
    + typedef map<const char*, double> RamMap;
    +//protected:
    + virtual void update() = 0;
    +
    + double getTotalSize() const {return _total_size;}
    + //virtual void setTotalSize(double t) {_total_size=t;}
    +
    + double getSwapSize() const {return _swap_size;}
    + //virtual void setSwapSize(double s) {_swap_size=s;}
    +protected:
    + RamMap _size;
    + std::string _path;
    + std::stringstream _pid;
    + GPUInfo* _gpu;
    +
    + double _total_size;
    + double _swap_size;
    +};

Next, we would need to change the constructor there to read the GL-vendor property and dynamically select the correct GPU class (ATI/AMD, NVIDIA or Intel):

    + LinuxMemoryInterface() {
    + SG_LOG(SG_GENERAL, SG_ALERT, "TODO: Check for GPU vendor here, and dynamically allocate correct class");
    + }

For that, we need to know the correct property to check, for instance, here's the help/about dialog showing the gl-vendor property:

The file being $FG_ROOT/gui/dialogs/about.xml, the exact line (=property) being: http://sourceforge.net/p/flightgear/fgdata/ci/next/tree/gui/dialogs/about.xml#l196

So, a corresponding check would have to do a case-insensitive check to look for the "NVIDIA" token as a substring in the property /sim/rendering/gl-vendor using the fgGetString() API: http://wiki.flightgear.org/Howto:Work_with_the_Property_Tree_API#Strings

    #include <algorithm>
    #include <string>

    MemoryInterface() {
    // get the string
    std::string fallback = "NVIDIA"; //default value

    // get the actual GL vendor string from the property tree, using the fallback
    std::string glvendor = fgGetString("/sim/rendering/gl-vendor",fallback.c_str() );

    // make it upper case: http://stackoverflow.com/questions/735204/convert-a-string-in-c-to-upper-case
    std::transform(glvendor.begin(), glvendor.end(), glvendor.begin(), ::toupper);

    // look for the "NVIDIA" substring: http://www.cplusplus.com/reference/string/string/find/
    std::size_t found = str.find("NVIDIA");
      if (found!=std::string::npos) {
      SG_LOG(SG_GENERAL, SG_ALERT, "Supported GPU found: NVIDIA");
      _gpu = new NVIDIA_GPU;
    }
    // else if ATI/AMD ...
    // else if INTEL ...
    else {
    SG_LOG(SG_GENERAL, SG_ALERT, "Unsupported GPU vendor:" << glvendor);
    }


    // the destructor would need to be changed to delete the _gpu pointer:

    LinuxMemoryInterface::~LinuxMemoryInterface() {
    delete _gpu;
    }

next, we need to change the update method to also call _gpu->getVRAMUsageInKB():

    // or log the whole thing to the property tree
    if (_gpu)
    SG_LOG(SG_GENERAL, SG_ALERT, "VRAM usage:" << _gpu->getVRAMUsageInKB() << std::endl );
    else SG_LOG(SG_GENERAL, SG_ALERT, "VRAM tracking unsupported for your CPU");

Nasal GC Tracking

ThorstenB once posted a patch that tracks GC invocations over time (printf) which should be easy to also expose to the property tree and log/plot accordingly. Also, it shows "GC pressure" (=symbols/references) per namespace.

you don't need a new/dedicated subsystem for tracking GC stuff: The Nasal GC is invoked as part of Nasal, which already is a subsystem, and which will be invoked by other subsystems, too (via listeners & timers).

So, you could patch nasal (in SimGear) to accept an option callback pointer to log this type of stuff, and then register a C++ static method in FGNasalSys that receives the corresponding notifications and logs those via SG_LOG() and/or fgSetDouble()

For instance, The Nasal C engine needs to be initialized via something like naInitNasal() - for details, refer to FGNasalSys::FGNasalSys() (the constructor) and/or the ::init() method.

The corresponding code could be extended to also accept a callback that tracks GC invocations, i.e. total references/symbols per namespace.

ThorstenB's patch should be a good start for this, i.e. the location would remain "as is", but the callback would be registered from FGNasalSys, so that this could be something like static void FGNasalSys::trackGC()

with &FGNasalSys::trackGC being registed then.

looking at the code, there's something called initGlobals at: http://sourceforge.net/p/flightgear/simgear/ci/next/tree/simgear/nasal/code.c#l161

with the globals struct being defined at: http://sourceforge.net/p/flightgear/simgear/ci/next/tree/simgear/nasal/code.h#l40

So this could be easily extended to also hold a void (*gc_callback)() pointer

The C++ constructor for wrapping the Nasal C engine already sets up some C++ stuff: http://sourceforge.net/p/flightgear/flightgear/ci/next/tree/src/Scripting/NasalSys.cxx#l210

the whole initGlobals() thing happens when a new context is created: http://sourceforge.net/p/flightgear/simgear/ci/next/tree/simgear/nasal/code.c#l196

So will be automatically invoked by the C++ code at: http://sourceforge.net/p/flightgear/flightgear/ci/next/tree/src/Scripting/NasalSys.cxx#l819

The ::init() method contains additional setup logic: http://sourceforge.net/p/flightgear/flightgear/ci/next/tree/src/Scripting/NasalSys.cxx#l812

I would do it like this:

• change the Globals struct to add a new pointer for an optional callback
• edit code.c/code.h to add an optional registerGCTracker() that sets the globals->gc_track_cb
• edit FGNasalSys to add a static method there like "trackGC" use SG_LOG(SG_GENERAL, SG_ALERT, "GC tracker invoked !");
• edit FGNasalSys::FGNasalSys() (the ctor) to set up the globals->gc_track_cb to point at this "trackGC" method

That way, you are registering a C++ callback with the Nasal engine, which can be invoked by checking via:

    if (globals->gc_track_cb!=NULL) {
    gc_track_cb();
    }

to make it process useful information, just change the signature to accept either a naContext or a naGlobals pointer, so that the C++ routine you registered, can access everything.

And then the C++ code can do stuff like fgSetInt(), SG_LOG() etc easily

SimGear: http://codepad.org/ZSGZbMiV/raw.txt

diff --git a/simgear/nasal/code.c b/simgear/nasal/code.c
index 2bf4645..61a7ae8 100644
--- a/simgear/nasal/code.c
+++ b/simgear/nasal/code.c
@@ -22,6 +22,19 @@ void printStackDEBUG(naContext ctx);
 
 struct Globals* globals = 0;
 
+extern unsigned int gRefCount,gObjCount;
+
+void registerGCTracker(TrackGCcb cb) {
+// check if globals is a valid pointer 
+if (!globals) {
+fprintf(stderr, "Globals must be set up first!)");
+return;
+}
+
+printf("Registering GC tracking callback!");
+globals->gc_cb=cb;
+}
+
 static naRef bindFunction(naContext ctx, struct Frame* f, naRef code);
 
 #define ERR(c, msg) naRuntimeError((c),(msg))
@@ -168,6 +181,8 @@ static void initGlobals()
     globals->sem = naNewSem();
     globals->lock = naNewLock();
 
+    gObjCount = gRefCount = 0;
+
     globals->allocCount = 256; // reasonable starting value
     for(i=0; i<NUM_NASAL_TYPES; i++)
         naGC_init(&(globals->pools[i]), i);
diff --git a/simgear/nasal/code.h b/simgear/nasal/code.h
index e7cb3f3..5b0befc 100644
--- a/simgear/nasal/code.h
+++ b/simgear/nasal/code.h
@@ -41,6 +41,7 @@ struct Globals {
     // Garbage collecting allocators:
     struct naPool pools[NUM_NASAL_TYPES];
     int allocCount;
+    TrackGCcb gc_cb;
 
     // Dead blocks waiting to be freed when it is safe
     void** deadBlocks;
diff --git a/simgear/nasal/gc.c b/simgear/nasal/gc.c
index 5ac9c43..6819d19 100644
--- a/simgear/nasal/gc.c
+++ b/simgear/nasal/gc.c
@@ -4,6 +4,9 @@
 
 #define MIN_BLOCK_SIZE 32
 
+int gRefCount=0;
+int gObjCount=0;
+
 static void reap(struct naPool* p);
 static void mark(naRef r);
 
@@ -38,6 +41,8 @@ static void garbageCollect()
     int i;
     struct Context* c;
     globals->allocCount = 0;
+    // https://www.mail-archive.com/flightgear-devel%40lists.sourceforge.net/msg31762.html
+    gObjCount = gRefCount=0;    
     c = globals->allContexts;
     while(c) {
         for(i=0; i<NUM_NASAL_TYPES; i++)
@@ -53,6 +58,14 @@ static void garbageCollect()
         c = c->nextAll;
     }
 
+    // check if there is a GC callback registered, if so call it
+
+    TrackGCcb cb = globals->gc_cb;
+    if(cb) {
+	// callback is registered, so call it
+	(*cb) (gRefCount, gObjCount);
+    }
+
     mark(globals->save);
     mark(globals->save_hash);
     mark(globals->symbols);
@@ -242,6 +255,7 @@ static void markvec(naRef r)
 static void mark(naRef r)
 {
     int i;
+    gRefCount ++;
 
     if(IS_NUM(r) || IS_NIL(r))
         return;
@@ -249,6 +263,8 @@ static void mark(naRef r)
     if(PTR(r).obj->mark == 1)
         return;
 
+    gObjCount++;
+
     PTR(r).obj->mark = 1;
     switch(PTR(r).obj->type) {
     case T_VEC: markvec(r); break;
diff --git a/simgear/nasal/nasal.h b/simgear/nasal/nasal.h
index 8b6b2c9..48ed460 100644
--- a/simgear/nasal/nasal.h
+++ b/simgear/nasal/nasal.h
@@ -44,6 +44,10 @@ void* naGetUserData(naContext c) GCC_PURE;
 
 // run GC now (may block)
 void naGC();
+struct Globals;
+typedef void (*TrackGCcb) (unsigned int refs, unsigned int objects);
+void registerGCTracker(TrackGCcb cb); 
+
 
 // "Save" this object in the context, preventing it (and objects
 // referenced by it) from being garbage collected.

FlightGear: http://codepad.org/xqOoW7rB

diff --git a/src/Scripting/NasalSys.cxx b/src/Scripting/NasalSys.cxx
index 8e7622e..ea6890d 100644
--- a/src/Scripting/NasalSys.cxx
+++ b/src/Scripting/NasalSys.cxx
@@ -60,6 +60,11 @@ void postinitNasalGUI(naRef globals, naContext c);
 
 static FGNasalSys* nasalSys = 0;
 
+static
+void TrackGarbageCollectionCallback(unsigned int refs, unsigned int objs) {
+	SG_LOG(SG_NASAL, SG_ALERT, "Nasal GC: References:"<<refs<< " Objects:"<<objs);
+}
+
 // Listener class for loading Nasal modules on demand
 class FGNasalModuleListener : public SGPropertyChangeListener
 {
@@ -819,6 +824,9 @@ void FGNasalSys::init()
 
     _context = naNewContext();
 
+    // register a custom GC tracking callback
+    registerGCTracker(&TrackGarbageCollectionCallback);
+
     // Start with globals.  Add it to itself as a recursive
     // sub-reference under the name "globals".  This gives client-code
     // write access to the namespace if someone wants to do something
@@ -875,6 +883,7 @@ void FGNasalSys::init()
     simgear::PathList directories = nasalDir.children(simgear::Dir::TYPE_DIR+
             simgear::Dir::NO_DOT_OR_DOTDOT, "");
     for (unsigned int i=0; i<directories.size(); ++i) {
+	SG_LOG(SG_NASAL, SG_ALERT, "Adding Nasal module:" << directories[i].file() );
         simgear::Dir dir(directories[i]);
         simgear::PathList scripts = dir.children(simgear::Dir::TYPE_FILE, ".nas");
         addModule(directories[i].file(), scripts);