Talk:FlightGear benchmark

From FlightGear wiki
Jump to navigation Jump to search

Repeatable Settings

The following mechanisms can be used for creating a repeatable setup:

Metrics

  • frame rate
  • frame spacing (latency)
  • delta time for each subsystem (via the system monitor)
  • osg stats (need to be exposed as properties)
  • RAM utilization (via a patch)

Baseline settings

The impact of disabling/enabling different features needs to be measured using a safe subset of FG settings:

  • minimal startup profile (using the FDM-driven UFO mentioned by bugman)
  • draw masks with scenery/terrain etc disabled
  • different OSG threading modes
  • Rembrandt
  • osgEarth
  • GUI (PUI menubar) entirely disabled

Approach

The benchmark should preferably execute once all systems finished initialization, i.e. typically 60-120 seconds after booting FlightGear, possibly using accelerated simulator time to speed up the test, or gather more samples over a longer period of sim time.

All metrics need to be exposed as properties, and written to a file on disk, probably in CSV frmat, so that it can be easily post-processed using Excel and/or gnuplot.

Logging

We are using CSV (comma-separated values) files for now:

gui dialog

canvas dialog showing effect of draw masks

Referring to: http://forum.flightgear.org/viewtopic.php?f=4&t=25961&p=264702#p264702

##
# http://wiki.flightgear.org/Draw_masks

var setDrawMask = func(name, value) {
  setprop("/sim/rendering/draw-mask/",name, value);
};

var enableFeature = func(name) {
 setDrawMask(name,1);
};

var disableFeature = func(name) {
 setDrawMask(name,0);
}


##
# enable all draw masks
#

var drawMasks = ['terrain', 'aircraft', 'models', 'clouds'];
foreach(var mask; drawMasks) {
print("Disabling feature: ", mask);
disableFeature(mask);
}


##
# get current frame rate

var baseline_fps = getFPS();
print("Baseline frame rate: ", baseline_fps   );

var (width,height) = (320,160);
var title = 'FlighGear Benchmark v. 0.1';

var window = canvas.Window.new([width,height],"dialog").set('title',title);

# adding a canvas to the new window and setting up background colors/transparency
var myCanvas = window.createCanvas().set("background", canvas.style.getColor("bg_color"));

# Using specific css colors would also be possible:
# myCanvas.set("background", "#ffaac0");

# creating the top-level/root group which will contain all other elements/group
var root = myCanvas.createGroup();

var withCanvas_fps = getFPS();
print("frame rate with Canvas: ", withCanvas_fps   );

var vbox = canvas.VBoxLayout.new();
myCanvas.setLayout(vbox);

addMetric(root, vbox, "Baseline framerate:", "341");