Talk:FlightGear benchmark

Repeatable Settings

The following mechanisms can be used for creating a repeatable setup:

• Fgfsrc
• a custom Preferences.xml configuration file
• pre-recorded flights
• Fgtape (via Instant Replay)
• Route manager (flight plans)
• Nasal

Metrics

• frame rate
• frame spacing (latency)
• delta time for each subsystem (via the system monitor)
• osg stats (need to be exposed as properties)
• RAM utilization (via a patch)

Baseline settings

The impact of disabling/enabling different features needs to be measured using a safe subset of FG settings:

• minimal startup profile (using the FDM-driven UFO mentioned by bugman)
• draw masks with scenery/terrain etc disabled
• different OSG threading modes
• Rembrandt
• osgEarth
• GUI (PUI menubar) entirely disabled

Approach

The benchmark should preferably execute once all systems finished initialization, i.e. typically 60-120 seconds after booting FlightGear, possibly using accelerated simulator time to speed up the test, or gather more samples over a longer period of sim time.

All metrics need to be exposed as properties, and written to a file on disk, probably in CSV frmat, so that it can be easily post-processed using Excel and/or gnuplot.

Logging

We are using CSV (comma-separated values) files for now:

• Property Logging
• Nasal I/O (io.nas) (can only write to $FG_HOME/Export)

gui dialog

canvas dialog showing effect of draw masks

Referring to: http://forum.flightgear.org/viewtopic.php?f=4&t=25961&p=264702#p264702

##
# http://wiki.flightgear.org/Draw_masks

var setDrawMask = func(name, value) {
  setprop("/sim/rendering/draw-mask/",name, value);
};

var enableFeature = func(name) {
 setDrawMask(name,1);
};

var disableFeature = func(name) {
 setDrawMask(name,0);
}


##
# enable all draw masks
#

var drawMasks = ['terrain', 'aircraft', 'models', 'clouds'];
foreach(var mask; drawMasks) {
print("Disabling feature: ", mask);
disableFeature(mask);
}


##
# get current frame rate

var baseline_fps = getFPS();
print("Baseline frame rate: ", baseline_fps   );

var (width,height) = (320,160);
var title = 'FlighGear Benchmark v. 0.1';

var window = canvas.Window.new([width,height],"dialog").set('title',title);

# adding a canvas to the new window and setting up background colors/transparency
var myCanvas = window.createCanvas().set("background", canvas.style.getColor("bg_color"));

# Using specific css colors would also be possible:
# myCanvas.set("background", "#ffaac0");

# creating the top-level/root group which will contain all other elements/group
var root = myCanvas.createGroup();

var withCanvas_fps = getFPS();
print("frame rate with Canvas: ", withCanvas_fps   );

var vbox = canvas.VBoxLayout.new();
myCanvas.setLayout(vbox);

addMetric(root, vbox, "Baseline framerate:", "341");