Clock story in TimeManager

This article is a stub. You can help the wiki by expanding it.

Clock story, an exploration of timeManager.cxx

We'll make a travel in the clock's country, to understand what's happening in TimeManager.cxx. Initially only dedicated to calculate the frame time increment (dt) passed to lot's of subsystems in the update loops, but now having a responsibility in time synchronization between different FG session.

Prior 2018

The simulation time: the simulated time clock was something basic, we started at 0, do some clamping on the delta, (sim/max-simtime-per-frame) then only allow dt to be multiple of 1/modelHz (120 by default) this give a "dtRemainder" difference between the time we mesured, and the time we keep as timestamp; Then is applied time acceleration, giving a sim time unrelated to real time flow, for time accel != 1

we have a clock :

• having 1/modelHz as basic step (before time accel)
• able to pause
• suffering for acceleration if time accel is used

This simulation time was used for the flight simulation as intended, but was the timestamp reference for mp protocol. That's what lead to the 2018 change, the need to a clock for the mp protocol, not based on simulated time, but a "real time" timestamp.

2018 - 2020

The multiplayer achitecture (multi server), and the slow mp packet sending rate made impossible a good synchronization with only a server ping, so we moved to a "real time" way of addressing lag issues, helped by the introduction of 2 clocks in TimeManager.

We still have the same way of providing the simulation time dt, with the same characteristics as before, but we added:

• a "steady" clock: this clock start synchronized to the wall clock, then follow the "dt" increment, without pausing and accelerating, using a monotonic timer. The goal is to have the wall clock tuned by ntp or ptp, and the different mp pilot are in time with each other by relying on dedicated network time service.

We provide a way to know if the steady clock is still on time with the wall clock, setting "sim/time/compute-clock-drift" to "true" give the current difference in "sim/time/steady-clock-drift-ms"

• the mpprotocol clock: if the ntp works well, it's the same as the steady clock, but to cope with player not sing ntp, or eg flying with a recorded flight, there's an offset to adjust this mpprotocol clock relative to the steady clock (sim/time/mp-clock-offset_sec). this clock give us the timestamp used in the mp protocol.

This offset is exposed in the lag menu, if you need it :) (more to see here).

This was a bit of a headhache to add in the TimeManager, but the result is quite good (for who know how to set it up, still need improvement for the end user)

Maybe you've noticed that those changes were made only during winter, so the question is, what needed to be changed this winter ?

2021 (hopefully)

To summarize, we have a time step multiple of 1/modelHz, a steady clock based on the wall clock, and a good concept for the multiplayer lag compensation based on time concordance. So what could be wrong?

We won't deal this time with the multiplayers, but with something a bit more delicate, ie the net external protocol, and that lead to investigate precisely what was possible from a flightgear perspective.

The question was: can we make each flightgear loop stick to a time grid with a fixed step ?

This would allow multiple FG sessions to be delayed inside the frame, and in synch with some 3rd party software (simulink, JSBSim standalone etc...) Assuming the wall clocks are in synch (ntp over internet or ptp in local networks or on the same pc) there were 2 problem in Flightgear:

• the prop "/sim/frame-rate-throttle-hz" allow FG to run at a specified frame rate, but the tests showed inegal spacing between frame because:
  • we have minimal step equal to 1/modelHz and if modelHz is not a multiple of Throttle-frame-rate, we'll have different length of frame time reported, even if the throttle is equally spaced.
  • the waiting implementation was based on the last timestamp from the system, if we are a bit late, this accumulate and make us loose frame in the long time.

• the steady clock was initialised with the wall clock time at startup, that make it random.

there are 2 parts for the "hopefully" 2021 winter change:

• we don't start the steady clock with the value from the wall clock directly, but we are finding the timestamp just before on a time grid of 1/modelHz, and the difference goes in dtRemainder (the one we introduced in the first topic :) ) this make the steady clock pass by the full second. To allow a volontary shift of the steady clock there's an offset (sim/time/frame-time-offset-ms) used to delay in time two FG sessions.
• Throttle-frame rate has 2 change:
  • we wait not a fixed time, but until the next step on the time grid (1/modelHz). If the load allow this work very well (you can log "sim/time/dt-remainder" to have an idea of far you are from the time grid.
  • We don't try to follow the value of throttle-frame-rate precisely, but find the closest value giving equally spaced frame on a 1/modelHz step, ie for modelHz=120, we can have 120,60,40,30,24,20,15,10,8,6,5,3,1

Other values are possible but need a change of modelHz.

For now this work well only if we set the different prop at start, changing modelHz in sim will lead to randon steady clock offset, and the frame-time-offset-ms is only applied on the initial run and stay the same. To ensure a steady frame rate stick to precise grid, you should give the same value at modelHz and throttle-frame-rate.

Possible improvements:

• allow only a limited time windows for the timestamp we start the loop, if we're too late, just skip a frame and start on time on the next grid step.
• make the frame time offset tunable, eg to find the best value when dealing with a source with unknown offset in frame (case with simulink packet replay with tcpreplay, wich allow strict time grid, but unknown start time)

I'll let you see this in action in this simulink exploration, when it'll be done :)