Redesigning the Replay System

This article or section contains out-of-date information Please help improve this article by updating it. There may be additional information on the talk page.

Request for Comments:

FlightGear Run Levels SGModule - standalone SGSubsystems An Integrated AI Traffic System Autopilot Enhancements Backwards Compatibility Initiative CDI instrument Decoupling the AI Traffic System Multiplayer Enhancements Modularizing, parallelizing and distributing FlightGear FDM engine feature standardization FlightGear Headless FlightGear Sessions Keyboard function priority list Next Generation Scenery Property Tree Reorganization Recommended Property Tree Enhancements Recommended Project Policies Redesigning the Replay System Remote Properties Simplifying Aircraft Deployment

The Instant Replay system in its current form is not able to deal with many possible FlightGear use/case scenarios, among others it has currently the following known limitations:

• replaying environmental state (time, weather, visibility) is not yet supported
• replaying AI state (traffic, ATC) is not yet supported
• replaying flights with specific custom aircraft animations is not yet fully supported
• replaying flights done with aircraft that make use of Nasal scripts for some advanced features is not yet sufficiently supported (or standardized)

Also, the replay system is directly related to a number of other important use/case scenarios, such as:

• saving/loading whole flights (see prerecorded flights using the flightrecorder system)
• saving/loading in-flight "situations" (see Aircraft Checklists)
• resuming flights at a particular position in time (FDM issues)
• expose the replay buffers to Nasal space (i.e. for use in Map dialog, flight path evaluation/analysis) [1]
• master/slave setups sync'ed via generic protocols

While there are several other systems/techniques (such as the generic protocol or logging systems, or the ac-state.nas script to save aircraft state) being used to compensate for some of the shortcomings of the current replay system, it is still getting obvious that there needs to be an orchestrated effort to redesign the instant replay subsystem to make it become more flexible and configurable, so that it provides key functionality in a standardized fashion.

It would probably make sense to log another feature request regarding the flight recorder system - because it's all about coming up with all aircraft-specific properties (including animations etc) - so the flight recorder system is the most recent, and most modern, component that addresses the same underlying core problem, namely that of sampling aircraft-specific properties. In other words, generalizing the system and using its data to also determine what properties to sample for multiplayer or multi-instance setups is simply logical. And the good thing is that aircraft developers would be responsible to provide and maintain the correspoinding configuration file, while ensuring that a handful of related FG features would be supported using a single file.

Status

It's basically a "property recorder", so anything that's in the property tree can be recorded - and replayed. Aircraft-specific XML descriptions can be used to specify properties, data type and interpolation method (discrete, linear, or rotational in degrees/radiant) for each "signal".

There'll be a set of default property lists which can be included, so only custom properties need to be specified manually. Naturally, all (existing) aircraft not providing any recorder configuration will use a default, matching the hard-coded system of <= FG 2.4.0.

1. Sessions can be saved to/loaded from disk. Simply fly along, then select "Save flight recorder tape" from the file menu and press "Save". You can load the tape again at any later time. You can keep the tape recording for your own amusement - or send it to someone else.

2. You can add text messages to a flight recording. This turns the recording into a flight tutorial. It's somewhere in between the existing tutorial system and a Youtube video: unlike the existing tutorials, the recorded flight can be used to demonstrate complete flight manoeuvres, such as how to fly a complex approach into an interesting airport. And unlike a fixed video tutorial, users can change views, inspect instruments, or take over control at any point.

Right, the "my controls" button is currently intentionally disabled for JSBSim and for YASim helicopters. Only works with YASim aircraft so far.

YASim reads in all relevant properties before each iteration, so any external change to a property (aircraft angle, velocity) takes immediate effect. This is why it is easy to just take over control of a "replay session": when the replay system has restored the state of all relevant properties, we can just reenable the FDM calculation - and YASim just takes it from there and continues.

Unfortunately that's different with YASim helicopters: the rotor speed (pretty much the most important property of a helicopter) isn't read from the property tree. So, if you took over control in mid-air, YASim ignores the rotor speed set by the replay system - and the helicopter dropped like a stone. I tried to fix this a while ago, but it turned out to be more complex than expected.

It's the same issue with JSBSim, if I remember correctly. It doesn't read (at least not all) properties prior to an iteration, so external changes to aircraft properties don't have any effect.

It's possible to start JSBSim with a given velocity/angle/.. though, like we do when initially starting FlightGear or trigger a simulator reset. However, we currently can't just reinitialize or tear down and recreate a single subsystem - at least that is currently not working with the FDM subsystem. The whole (re)initialization phase has a lot of dependencies and fixed sequences. So that currently also isn't an option.

The hope is that will change. We have already cleaned up a lot of subsystem / initialization things over the past year - and more things are planned. Eventually, I hope we can just recreate an FDM at any time ("out of thin air"... ;-) ) with arbitrary initial settings.

main improvement is the option of making it work properly with any type of aircraft and any custom properties. The old system only worked perfectly with certain propeller aircraft and piston engines.

Important to mention though: by default, nothing changes for existing aircraft. The old system already covered a huge number of properties - but it's impossible to just record everything. I did some tests trying to auto-detect aircraft types and properties to be recorded - but that cannot really work with all aircraft for a number of reasons. And any fixed selection eventually doesn't work perfectly with some aircraft. Hence, it seems a better idea to avoid any kind of guessing and hard-coded logic - and rely on configuration alone. The default, obviously, is the same set as hard-coded for FG2.4 and earlier.

The new system is still easy to adapt since several ready-to-use configuration files are available - which simply need to be included, depending on aircraft/engine/.. type. And it's not much more work to customize.

For those interested, there's a README.flightrecorder in the Docs folder. There's also a few examples in fgdata showing different levels of customization: the ASK13 (glider), the c172p (prop/piston), the b1900d (turboprop), the UH-1 (helicopter), and the 777-200ER (jet).

As announced, I haven't changed the actual replay buffering. But, as someone requested, all buffer properties are now configurable (see /sim/replay/buffer properties). So, if you have enough memory, you could increase the buffer sizes/rates. There's still no configuration GUI for these properties though.

The most obvious change though is probably the new GUI dialog: looks like a video player, provides play/pause/skip controls, also controls replay speed. You can also use the 4 arrow keys to control replay (they were useless during replay so far). Finally, since replay can be paused now, it was necessary to move the "stop replay" key binding to the "ESC" key (instead of pressing "pause" twice) – which feels more intuitive to me anyway.

The replay on MP was really annoying (see new "sim/multiplay/freeze-on-replay" switch. ), since it also happened to well behaved MP players, which you normally don't ignore. I've seen people following ATC instructions, and nicely taxiing to their gate - and then they suddenly engaged replay - being unaware of the effect and meaning no harm. So changing the default here avoids most issues.

Limitations

Saving/loading and replaying/continuing an arbitrary flight is directly related to a number of hard limitations in the FG core code base, it touches MANY areas and problems that have been around for over a decade and which were never really solved. So, adding support for saving/loading flights would no longer be "just" about the replay system or the "flight recorder" system, but it would touch LOTS of places in FG, and require plenty of re-architecting to make things more explicit.

Basically, even if the fgfs core code were prepared to support this at some point, most aircraft and other base package resources, would still require fixing and explicit coding, in the form of registering listeners to handle sim-wide signals to free resources in a sane fashion. Just imagine all the Nasal code running most of the time etc.

Also, the option of taking over flight controls at any point during a replay is difficult to get right. The latter isn't as easy as it sounds, since most FDMs don't really like being repositioned or even having the speed changed externally.

I have updated the replay system, to allow taking over control at any point during the replay. So, if you messed up your approach, you can go back in time with the replay system, and try again - and again...

There is a new red button "My controls!", currently only shown with YASim aircraft: hitting it causes the simulation to continue at the current replay time/position. How well that works also depends on the properties being recorded/replayed - so if an aircraft isn't configured to record it's non-standard gear lever...

Unfortunately it's not working with YASim helicopters. YASim doesn't allow to set all necessary initial values externally - specifically I can't set "rotor engines" to an initial rpm. Starting up a helicopter in the air with rpm = 0 results in a significantly reduced "flight" time... :)

Flight Recorder

The ability to save and reload (and share) replays is extremely useful for so many things such as flight training to check student performance of flight exercises, research, testing, virtual airlines checking/verifying flights, contests, etc, etc.

The replay system uses three buffer levels: short term memory records 60 seconds at full frame rate, mid term buffer records another 10 minutes at 2fps, and the long term buffer holds 1 hour at 1/5fps. The buffer durations and rates are exposed by properties now. So, if you had enough memory, you could increase the buffer sizes or change their rates.

Potential Optimizations

Streaming data to disc would be nice - but should be optional and not be enabled by default. It would also be nice to have an option to save a recorded instant replay buffer (when you haven't enabled the stream-to-disc feature earlier). I can't comment on the different interpolation methods. But what would also help is to change the recording scheme: Currently the valuesof a fixed set of properties is recorded at each frame. However, most properties rarely change. Only the a/c position and speed properties tend to change in every frame. Properties like gear position and control surfaces are almost constant (from frame to frame). So, it might be a good idea to record all properties in an interval of a few seconds only, while only recording properties that have actually changed with every frame. That should allow to drastically reduce the amount of recorded data - and allow much longer high resolution memory recordings. And you could still fast-forward within the buffer, since you'll have a complete set of properties every now and then. It's a simple but effective compression method which is used in many areas. It would also be nice to support recording of custom properties. We already have a similar solution for multiplayer mode. If the replay system recorded the same properties configured in the "aircraft-set.xml" (the "generic" int/float/string properties), it would mean a nice improvement. It would also have other advantages, such as allowing simple tests of an aircraft's multiplayer configuration. In fact, frames of tagged values is the same method we're already using in encoding data for multiplayer transmission. Though that module could also do with some improvements (*yikes* it's currently packing each 8bit string character into a 32bit value for transmission). Also, it doesn't reduce traffic by detecting unchanged values yet (UDP transmissions loose some data, so that's more tricky here, though also not impossible). Yes, the tags are additional overhead. But I'm quite convinced that this schemes outweighs the disadvantage - if we used this to encode changing values only. I checked our current recording data structure: it has a huge size of 1160 bytes for every frame. Also, the fixed structure needs to reserve space for the maximum number of wheels/engines/tanks etc - so even if an a/c has fewer tanks/.., it still uses up the space. And it doesn't work for aircraft which somehow exceed the default (btw, I saw a maximum of 4 tanks and 3 wheels is configured, so that is why there are gear replay issues with 4 or 5-gear aircraft, like the Concorde...). And looking at many of the recorded properties, I think many of them indeed will hardly ever change from frame to frame, so this 1K structure could probably be reduced significantly - despite extra tagging bytes. Thinking about it, it should also be possible to join some of the multiplayer and replay code. The replay system could use the existing encoder of the multiplay manager to generate the data packets - but then record them locally instead of transmitting them via UDP. During replay, these packets could then also be decoded by existing multiplayer code. Might be worth a thought, if someone wanted to dig in this area. — ThorstenB

Community Feedback about Instant Replay Issues

• "Be warned that there are multiple bugs in the recording/playback system, including FG code, simgear code, and FG data. Some of the bugs have been found and fixed. Others have not."[2]
• "First off, what parameters are logged by it, or what defines if a parameter is logged by it? ie, thrust reverser use do not replay, but flaps do. This doesn't seem to be by model builder choice (?), unless Instant Replay is tied into multi-player parameters?"[3]

• 05/2010: "We recognized that to record hours of high resolution flight data would blow up the memory footprint of FlightGear on any PC. So we went for a balance"[4]

• 05/2010: "We record the most recent 60 seconds of data at full resolution, then the previous "n" minutes (I forget the exact number) is recorded at 1 second intervals. And then I think we go even further back recording at an even sparser interval. This way you get something as far back as you like, and you get high detail if you replay the last 60 seconds." [5]

• 05/2010: "Originally, the recorder got turned off during the replay. Now the recorder continues to record while you replay ... so you are recording the replay as you replay it and you sort of end up with a recursive infinite loop. This actually has a nice feature that when you replay a flight, it loops for ever until you quit the replay, but know that each time through you are replaying a recording of the previous replay, and losing resolution each time through the cycle. So that's a bug, it needs to be fixed, I haven't had a chance to dig into it." [6]

• 05/2010: "The problem with this particular bug is that because you are recording as you replay, you never actually get to see that 60 seconds of highly detailed data (assuming you replay more than 60 seconds which most people probably do.) So we are missing out on all the subtle nuances of your landing when you replay your landing for instance." [7]

• 05/2010: "One other thing. The data structure that is recorded is the FGNativeFDM structure defined in src/Network/native_fdm.hxx. You do bring up a good question: what values should be recorded every frame? Given that aircraft designers can make up their own properties and drive them with nasal or other means, we can't know in advance a fixed set of property names that cover every situation." [8]

• 05/2010: "So for the replay system we decided to go with a fixed, pre-defined binary structure ... and then you do live with a few replay warts if you are viewing an aircraft that uses "non-standard" properties ... and that's not a knock on the aircraft designer ... some airplanes have more than 4 engines, the AN-225 has like 185 landing gear assemblies, many aircraft have unique control systems and control surfaces ... and we can't take 20 minutes worth of snapshots of the entire property tree @ 60hz ... not on my PC anyway." [9]

• 05/2010: "So the 60 second marker you mention is interesting ... that could correspond with the system that records the most recent 60 seconds at a high data rate. There could be some boundary logic there that didn't take every possible situation into consideration." [10]