Testing

Note  Also see FlightGear Benchmark and FlightGear Headless

Screen shot showing a the performance monitor in a patched version of FlightGear 3.2 where subsystem initialization is made better configurable and increasingly optional by allowing subsystems to be explicitly disabled/enabled during startup. Decoupling internal subsystem dependencies means that we can more easily provide support for benchmarking, but also headless regression testing - and eventually, also a standalone FGCanvas startup mode.

All suggestions and edits welcome.

I want to develop a set of tests for flightgear next using various scenarios and options. This would include testing of build options and various runtime settings in flightgear.

The purpose of testing is to find issues that need fixing, not proving that a certain combination of options is without problems.

Success will be finding and properly documenting an issue for the devs to fix.

A recent good example of what I'd like to find is described the recent post osg multithreading crashes.

To get started I think we need a list of features to test and the options that enable them in builds and in runtime. Remember that these should be areas likely to exhibit bugs:

I'll also need advice on how to gather the right relevant information so the issues can be understood and addressed

So what's on your list?

-Pat

What info do you need to start testing productively?

• Do we need more basic information about setting up a test? (Yes. Some of us come from other platforms, c++ skills vary etc.)
• What basic skills should we assume? Do we want to encourage testing by someone who is good at testing, can read code, but can't write to the level needed? What support would they need to be effective testers?
• What would you need to learn how to do?
  • build a debug version of flightgear
  • learn how to start and operate the debugger for your platform.
  • learn to read c++ (not write, read!)
  • know the basic structure of the flightgear application, major components, interfaces, protocols, event loops.

Would it help to include suggested command lines for starting these scenarios?

Flightgear Test Scenarios

Test Name	References	Required Hardware
osg multithreading	Property /sim/rendering/multithreading-mode=AutomaticSelection	more than one graphics card
FDM selection	Flightgear FDM properties Property Tree/Native Protocol Slaving	any, Possibly multiple machines/monitors
Reset & re-init	Ensure the sim can reset/re-init	any
built in telnet daemon	Property Tree Servers Telnet usage	any
built in httpd daemon	Property Tree Servers	any
in air startups	need list of command line parameters for this, otherwise, reposition to in-air.	any
logging options	do the log levels give us the right information to pinpoint a problem? Property Logging SG LOG	any