Aircraft testing checklist

This is a set of basic checks to run on an aircraft, to validate compatibility with a FlightGear version

For aircraft in FGAddon which have been claimed for testing, see the tester list

Test procedure

Claim an aircraft, by adding your name and the aircraft to the tester list. Then do a test flight going through the steps below. Then open a ticket (one per aircraft!) in the tracker with any failures / issues. (You don’t need to list the things that worked, just the things that didn’t)

Some important details

The ticket should have the following set

• Title of "<aircraft name> test report" or similar
• milestone=2020.2
• set the label ‘aircraft-qa’
• if the aircraft does not have a maintainer, add a label ‘nomaintainer’

This makes it easier to find which aircraft have open issues, and which aircraft are abandoned. If the aircraft has an obvious active maintainer, it would be excellent to point them at the ticket

General points

• avoid issues ’smaller’ than the ones on this list, broadly. If the fuel cap is in the wrong place for the 1972 model of the Piper Archer, that’s something to take up with the aircraft maintainer or learn how to fix yourself. There's no point spending hours testing such details if there is no one working on the aircraft who might fix them.
• The list is not exhaustive: if the Harrier can’t take off vertically, or a water-bomber can’t pick up water, those are major issues for that aircraft, despite not being listed below
• check the aircraft help, since it might list known issues / limitations. (“Fuel consumption is not accurate’, etc)
• for aircraft which are actively under development / maintained, check with the developer / development team before testing them, since otherwise you might be duplicating effort.
• If you encounter an issue which you suspect is generic to all of FlightGear, report it as normal in the tracker, or ask about it on the developer list, don't hide it inside an aircraft testing report
• make sure the aircraft is updated before testing it

Metadata

• does the aircraft have a clear name (not so long it overflows)
• is the description appropriate? Long enough to be relevant but not an essay..
• does it have new-style previews / splash screen in the launcher?
  • if it has new style-previews, do they follow the style guidelines: especially, no logos or text in them, which break readability of the splash and launcher screens.

Basic experience

• start on a runway, and start the engine manually
• start via autostart (assuming the acft has one)
• start in-air: eg use the 'on approach' feature of the launcher (note some aircraft may not work for an in-air start at all)

• if the aircraft has terrain variants (eg, skis, floats) and you have time, check they work as expected (eg seaport takeoff and landing for floats).
• if it's Naval aircraft, do the circuit mentioned below from a carrier

• check throttle (and pitch and mixture) work as expected (Eg, does the engine cut out at full lean? Does it redline at full throttle on the ground?)
• check brakes response and parking brake

• check avionics power: do displays turn on/off, does audio turn on/off based on the power/volume switches.
• tune COM1 and ensure ATIS is audible
  • check volume adjustment if there's a control for it

• check altimeter setting (can you set it, can you select inHG vs HPA if the acft supports it)

• tune NAV1 and check behavior, including from/to flag (might need to be done in air for reception)

• check ground steering (especially if ground tiller mode is supported)
• if there's pushback support, does it work okay?

• fly a circuit
  • check takeoff power handling (P-factor on takeoff roll, the sensitivity of rudder inputs, etc).
  • Check the rotation speed is plausible, if you can find some data on it.
  • check flaps/gear work as expected
  • check basic instruments update (use HUD to cross-check)
  • check radar altimeter if fitted (the HUD is good to cross-check this)
  • check stall performance in clean config (is it vaguely plausible, especially)
    • clean configuration stall can be checked on the circuit downwind
    • approach configuration stall can be checked on approach, but you might need to go around
  • check level flight over-speed performance (time to hit the barber pole at full power).
  • check engine performance instruments are somewhat working, keeping in mind most aircraft have very simple fuel/oil/temperature modeling.
  • check pitch trim works: can you trim for level flight and leave the controls untouched for a few seconds without the aircraft departing straight and level flight?

• check speedbrakes / spoilers do something (if fitted)

• fly a visual approach
  • check stall performance in approach config (is it vaguely plausible?..)
  • check ILS reception / deviation
  • check DME readings
  • check OM/MM/IM reception and lamps
  • on landing rollout, check the brakes work, and reverse thrust if supported, and that the ground handling on rollout is acceptable

• check fuel consumption / fuel dialog if possible
• execute a reposition, check nothing breaks

• execute a reset, check nothing breaks :) (the sim might crash...)

Visuals

• frame-rate : do you see a large frame-rate drop compared to other aircraft / the UFO. Is it only when in an inside or outside view?
• lighting states in the dark? (beacon, strobe, etc)?
• is the default view position okay?
• animations work as expected (when implemented) : especially basic ones (control surfaces move is most important).
• check all the views work, especially custom views, do they track okay using the joystick had and/or mouse look?
• default livery, is it something or 'unpainted grey'?
• does the cockpit use tooltips / knob animations for readability

Non-standard stuff

This is nice-to-haves, not mandatory

• If the aircraft has custom dialogs for Radios, Autopilot, Payload or Fuel, check the default dialogs either work, or are disabled / replaced. (See note about the AP dialog below)
• Check appearance under a compositor build
• Check appearance under very high ALS settings?

• failures: if the aircraft supports them, are they plausible. (eg overspeed with flaps extended). Especially, can they be reset back to working from a UI dialog, and does re-positioning cause them to trigger incorrectly.
• GPWS: if it's fitted, check its callouts on landing are correct

Autopilot checks

• If the aircraft has an autopilot implemented in the cockpit, check controls work for basic modes (ROLL, PITCH). If the aircraft does not have an autopilot implemented, there is /no/ requirement for the generic autopilot dialog to work successfully, but would be nice to note if it does or doesn't.
  • If the aircraft has a custom AP, but the generic dialog is enabled, but non-functional, note this. (The generic dialog should be disabled in this case)
• Intercept a VOR / leg : check the turning to intercept, and overshoot / roll out when reading the correct position
• check the AP is stable at 2x time acceleration, and see if it is at 4x or 8x (lots aren't....)
• Setup a basic route in the route-manager, and check interaction with the AP if supported: especially following waypoints and sequencing above them. This is more important for glass cockpits.