FlightGear wiki - User contributions [en]

A simulink exploration

2022-01-09T16:32:23Z

Jano: /* trying to run FlightGear and Simulink in synchronization */

{{stub}}

=== A Simulink exploration (one month free trial :) ===

If you can't wait, see [https://forum.flightgear.org/viewtopic.php?f=27&t=38421 the topic on the forum] :P
here are some tips to improve visuals for simulink driven Flightgear videos.
I only used the month free trial for matlab, so i can't run it anymore, but i still have netfdm packets recorded during some simulink run.

== simulink - fg synchronization ==

I used the asbhl20 project from simulink, which is a spacial ship short final landing. Flighgear use the netfdm input, with something like:

<code>''' --native-fdm=socket,in,30,,5502,udp --fdm=null '''</code>

You need to set the destination ip and port for the netfdm output from Simulink to match Flightgear ip and port, and if you're lucky, the plane fly to land safely in Flightgear when you start a simulink run !

Notice the netfdm protocol had some change after 2020.3 FG version (some tank props were added at jsbsim request), so using a more recent Flightgear with the same simulink i used ( 2020) will end as failure.

== a not so smooth visual ==

I didn't put a video example here, but the visual flow of an externally driven Flightgear is not a fluid experience, it's more like a compilation of frozen state played one after an other.
The current netfdm implementation consist simply in having the plane's position slaved to the last netfdm position received. Depending simulink and FG fps, we can have some frames having the same position, waiting from a netfdm update, or netfdm position being skipped because FG was to slow to render or load 3D stuff.

It's bearable when the plane is solo in the air, but this lead to big jitter if we try to have two planes together (or more) and is obvious close to the ground for solo plane.

== trying to run FlightGear and Simulink in synchronization ==

Later 2020.3 releases (>= 2020.3.9) include a way to run Flightgear more précisely on a time grid, we'll use this to have a better synchronization, where each simulink frame is displayed, and Flightgear use each simulink positions for only one frame.

* FlightGear:
Using the throttle frame rate option in the rendering menu make Flightgear starting each frame on a time grid, if it run fast enough, and if not, it will start on the next step given from the model-hz option, so to use a time grid at 30fps, we need both the model-hz and the throttle frame rate at 30:

<code>--model-hz=30 --prop:/sim/frame-rate-model-hz=30</code>

You can see the error between the time grid and the real time the frame started with /sim/time/dt-remainder-sec in the properties, if you want to log it and see if FG is stable enough.

* Simulink:
Sadly i didn't manage to have simulink run and send the netfdm packet at a nice 30fps (maybe because of the computer old age ? or the need of the real time mode not available to me), if you need to use feedback information from FlightGear, this is something to look at, notice Simulink provide a log of the offset between the sim running time and the real running time (add a scope to the output of the "set pace" module)
i didn't need feedback from Flightgear, so i changed the block waiting for it to use a record (iirc, otherwise Simulink was waiting nearly 5s for FG packet (not sent) before starting)

Here's what my log looked like: 1s late early, closing on real time, then in a 50ms window, not precise enough for my need (the result was the same from wireshark log analysis)

[[File:Simulink pace error log.jpg|none|pace error log]]

The cheat: as i couldn't manage a stable 30fps from simulink, i simply recorded the netfdm packets (with wireshark) and replayed them at 30fps with a network tool (tcpreplay on linux)

*Both together:
With a steady netfdm flow of 30 fps and FlightGear running on a time grid, the result was way better, there are still some frame a bit late to create some stuttering, but this is something to expect on my old dual core.
FlightGear run on a time grid passing by the full second, but that's not the case for tcpreplay witch start at random (no clue for simulink) so you may need a bit of tries before having a good result, if both run at nearly the same time, you'll notice more jitter!
There's a simple way to see how both are interacting: the record of the network packets involved (wireshark or similar is your friend) if you make FlightGear send mp packet each frame, you'll see how in synchronization they are.

== Video tips ==

Using a network tool to record and replay a simulink session allow to separate the Simulink run, from the video run, you can eg run Simulink and Flightgear at 1 fps and then replay the netfdm at 30 fps for the video.

== using 2 planes ==

Next was the need for a video with two planes interacting, Both FlightGear were driven from simulink netfdm, and they are seeing the other by the mp protocol. the idea here is for each frame to run Simulink , a bit later FG1 and a bit later FG2, this way FG1 will see FG2's plane one frame late, but the result should be visually good on FG2, if it manage to receive FG1's position from the same frame.
We will use a time offset in Flightgear, to shift the time grid a bit, using:
<code>--prop:/sim/time/steady-offset-ms=16.6</code>

here it shift the time grid by 16.6 ms, half the 33.3333ms length of a 30fps frame, so FG1 and FG2 will run one after the other every 16.6ms, and depending when Simulink play in the loop, you'll have a nice video on FG1 or FG2, after probably some bad tries (Simulink/tcpreplay nearly at the same time of one of FG making jitter planes)
Notice that the clock used for the time grid is initialized by the wall clock, so having multiple FlightGear on multiple computers running on a precise timing is possible if the wall clocks are in synchronization, ntp, ptp or alike are your friends ...

== more than 2 planes ==

the best solution for displaying more than 2 planes from Simulink in FlightGear would be to directly send the mp protocol, one FlightGear displaying all the multiplayers at once.
an other way is to have each plane running a FlightGear, each sending the mp protocol to a FlightGear session used as final visual.

If Simulink doesn't output the mp protocol, a little soft to convert netfdm to mp protocol would do the job, but there's the need for a timestamp in the netfdm protocol.

If you use multiple Flightgear on the same computer, be sure they use different CPU cores, here it run always in the first core and i need to change the core affinity once started. Notice that disk access can make the frames a bit longer, so avoiding log files or the like is better, and a ssd HD is probably a good choice.

== possible improvements ==

The current netfdm protocol doesn't include a timestamp, so we lack the time information for the position we have, that's why running both in synchronization is the only way to have a smooth result as of now (end 2021)

With a timestamp and the planes's velocity added (was not sent from the project i used), the jitter between two planes externally driven could be removed as this allow to predict the mp plane's position, but that would need a bit of work in FlightGear to use this timestamp as time in the mp protocol to display the other planes, and to send in the mp packets.

[[Category:Interfacing protocols| ]]

A simulink exploration

2022-01-08T23:15:34Z

Jano: /* trying to run FlightGear and Simulink in synchronization */

{{stub}}

=== A Simulink exploration (one month free trial :) ===

If you can't wait, see [https://forum.flightgear.org/viewtopic.php?f=27&t=38421 the topic on the forum] :P
here are some tips to improve visuals for simulink driven Flightgear videos.
I only used the month free trial for matlab, so i can't run it anymore, but i still have netfdm packets recorded during some simulink run.

== simulink - fg synchronization ==

I used the asbhl20 project from simulink, which is a spacial ship short final landing. Flighgear use the netfdm input, with something like:

<code>''' --native-fdm=socket,in,30,,5502,udp --fdm=null '''</code>

You need to set the destination ip and port for the netfdm output from Simulink to match Flightgear ip and port, and if you're lucky, the plane fly to land safely in Flightgear when you start a simulink run !

Notice the netfdm protocol had some change after 2020.3 FG version (some tank props were added at jsbsim request), so using a more recent Flightgear with the same simulink i used ( 2020) will end as failure.

== a not so smooth visual ==

I didn't put a video example here, but the visual flow of an externally driven Flightgear is not a fluid experience, it's more like a compilation of frozen state played one after an other.
The current netfdm implementation consist simply in having the plane's position slaved to the last netfdm position received. Depending simulink and FG fps, we can have some frames having the same position, waiting from a netfdm update, or netfdm position being skipped because FG was to slow to render or load 3D stuff.

It's bearable when the plane is solo in the air, but this lead to big jitter if we try to have two planes together (or more) and is obvious close to the ground for solo plane.

== trying to run FlightGear and Simulink in synchronization ==

Later 2020.3 releases (>= 2020.3.9) include a way to run Flightgear more précisely on a time grid, we'll use this to have a better synchronization, where each simulink frame is displayed, and Flightgear use each simulink positions for only one frame.

* FlightGear:
Using the throttle frame rate option in the rendering menu make Flightgear starting each frame on a time grid (after 2020.3.??), if it run fast enough, and if not, it will start on the next step given from the model-hz option, so to use a time grid at 30fps, we need both the model-hz and the throttle frame rate at 30:

<code>--model-hz=30 --prop:/sim/frame-rate-model-hz=30</code>

You can see the error between the time grid and the real time the frame started with /sim/time/dt-remainder-sec in the properties, if you want to log it and see if FG is stable enough.

* Simulink:
Sadly i didn't manage to have simulink run and send the netfdm packet at a nice 30fps (maybe because of the computer old age ? or the need of the real time mode not available to me), if you need to use feedback information from FlightGear, this is something to look at, notice Simulink provide a log of the offset between the sim running time and the real running time (add a scope to the output of the "set pace" module)
i didn't need feedback from Flightgear, so i changed the block waiting for it to use a record (iirc, otherwise Simulink was waiting nearly 5s for FG packet (not sent) before starting)

Here's what my log looked like: 1s late early, closing on real time, then in a 50ms window, not precise enough for my need (the result was the same from wireshark log analysis)

[[File:Simulink pace error log.jpg|none|pace error log]]

The cheat: as i couldn't manage a stable 30fps from simulink, i simply recorded the netfdm packets (with wireshark) and replayed them at 30fps with a network tool (tcpreplay on linux)

*Both together:
With a steady netfdm flow of 30 fps and FlightGear running on a time grid, the result was way better, there are still some frame a bit late to create some stuttering, but this is something to expect on my old dual core.
FlightGear run on a time grid passing by the full second, but that's not the case for tcpreplay witch start at random (no clue for simulink) so you may need a bit of tries before having a good result, if both run at nearly the same time, you'll notice more jitter!
There's a simple way to see how both are interacting: the record of the network packets involved (wireshark or similar is your friend) if you make FlightGear send mp packet each frame, you'll see how in synchronization they are.

== Video tips ==

Using a network tool to record and replay a simulink session allow to separate the Simulink run, from the video run, you can eg run Simulink and Flightgear at 1 fps and then replay the netfdm at 30 fps for the video.

== using 2 planes ==

Next was the need for a video with two planes interacting, Both FlightGear were driven from simulink netfdm, and they are seeing the other by the mp protocol. the idea here is for each frame to run Simulink , a bit later FG1 and a bit later FG2, this way FG1 will see FG2's plane one frame late, but the result should be visually good on FG2, if it manage to receive FG1's position from the same frame.
We will use a time offset in Flightgear, to shift the time grid a bit, using:
<code>--prop:/sim/time/steady-offset-ms=16.6</code>

here it shift the time grid by 16.6 ms, half the 33.3333ms length of a 30fps frame, so FG1 and FG2 will run one after the other every 16.6ms, and depending when Simulink play in the loop, you'll have a nice video on FG1 or FG2, after probably some bad tries (Simulink/tcpreplay nearly at the same time of one of FG making jitter planes)
Notice that the clock used for the time grid is initialized by the wall clock, so having multiple FlightGear on multiple computers running on a precise timing is possible if the wall clocks are in synchronization, ntp, ptp or alike are your friends ...

== more than 2 planes ==

the best solution for displaying more than 2 planes from Simulink in FlightGear would be to directly send the mp protocol, one FlightGear displaying all the multiplayers at once.
an other way is to have each plane running a FlightGear, each sending the mp protocol to a FlightGear session used as final visual.

If Simulink doesn't output the mp protocol, a little soft to convert netfdm to mp protocol would do the job, but there's the need for a timestamp in the netfdm protocol.

If you use multiple Flightgear on the same computer, be sure they use different CPU cores, here it run always in the first core and i need to change the core affinity once started. Notice that disk access can make the frames a bit longer, so avoiding log files or the like is better, and a ssd HD is probably a good choice.

== possible improvements ==

The current netfdm protocol doesn't include a timestamp, so we lack the time information for the position we have, that's why running both in synchronization is the only way to have a smooth result as of now (end 2021)

With a timestamp and the planes's velocity added (was not sent from the project i used), the jitter between two planes externally driven could be removed as this allow to predict the mp plane's position, but that would need a bit of work in FlightGear to use this timestamp as time in the mp protocol to display the other planes, and to send in the mp packets.

[[Category:Interfacing protocols| ]]

User:Jano

2021-11-06T03:26:27Z

Jano:

Moderator in the french forum, I'm in the FG world since 2007, with some hiatus. 
Mostly enjoying close flight formation, refueling action and the like mp stuff, this lead me to bug discoveries, then nasal and c++ basic learning, to implement what nobody want to do for me :) 
Don't expect a fast pace, as coding is not like anything i do at work, and to be honest paragliding took over most of the time i used to spend on FlightGear, leaving only a winter c++ windows ;)

==fg stuff==

* some obscur nasal ai speed function
* a hand in the nasal ai tanker
* some fdm related stuff mostly related to bug between yasim and jsbsim.
* time synchonization issues mostly in mp lag compensation.

==WIP==
* [[Improving real close formation flight|Improving close formation flight]]
* [[Mp-patch|MP Patch]]
* [[:fr:convertir un avion en dds|DDS conversion script]]
* [[DDS texture conversion]]
* [[Network tools usage]]
* [[Real Time manual]]
* [[A simulink exploration]]

==external link==

[http://www.youtube.com/user/hiphiphip3 FG related youtube channel]

User:Jano

2021-11-06T03:25:17Z

Jano: /* fg stuff */

Moderator in the french forum, I'm in the FG world since 2007, with some hiatus. 
Mostly enjoying close flight formation, refueling action and the like mp stuff, this lead me to bug discoveries, then nasal and c++ basic learning, to implement what nobody want to do for me :) 
Don't expect a fast pace, as coding is not like anything i do at work, and to be honest paragliding took over most of the time i used to spend on FlightGear leaving only a winter c++ windows ;)

==fg stuff==

* some obscur nasal ai speed function
* a hand in the nasal ai tanker
* some fdm related stuff mostly related to bug between yasim and jsbsim.
* time synchonization issues mostly in mp lag compensation.

==WIP==
* [[Improving real close formation flight|Improving close formation flight]]
* [[Mp-patch|MP Patch]]
* [[:fr:convertir un avion en dds|DDS conversion script]]
* [[DDS texture conversion]]
* [[Network tools usage]]
* [[Real Time manual]]
* [[A simulink exploration]]

==external link==

[http://www.youtube.com/user/hiphiphip3 FG related youtube channel]

User:Jano

2021-11-06T03:23:15Z

Jano: /* WIP */

Moderator in the french forum, I'm in the FG world since 2007, with some hiatus. 
Mostly enjoying close flight formation, refueling action and the like mp stuff, this lead me to bug discoveries, then nasal and c++ basic learning, to implement what nobody want to do for me :) 
Don't expect a fast pace, as coding is not like anything i do at work, and to be honest paragliding took over most of the time i used to spend on FlightGear leaving only a winter c++ windows ;)

==fg stuff==

* some obscur nasal ai speed function
* a hand in the nasal ai tanker
* some fdm related stuff mostly related to bug between yasim and jsbsim.

==WIP==
* [[Improving real close formation flight|Improving close formation flight]]
* [[Mp-patch|MP Patch]]
* [[:fr:convertir un avion en dds|DDS conversion script]]
* [[DDS texture conversion]]
* [[Network tools usage]]
* [[Real Time manual]]
* [[A simulink exploration]]

==external link==

[http://www.youtube.com/user/hiphiphip3 FG related youtube channel]

User:Jano

2021-11-06T03:22:02Z

Jano:

Moderator in the french forum, I'm in the FG world since 2007, with some hiatus. 
Mostly enjoying close flight formation, refueling action and the like mp stuff, this lead me to bug discoveries, then nasal and c++ basic learning, to implement what nobody want to do for me :) 
Don't expect a fast pace, as coding is not like anything i do at work, and to be honest paragliding took over most of the time i used to spend on FlightGear leaving only a winter c++ windows ;)

==fg stuff==

* some obscur nasal ai speed function
* a hand in the nasal ai tanker
* some fdm related stuff mostly related to bug between yasim and jsbsim.

==WIP==
* [[Improving real close formation flight|Improving close formation flight]]
* [[Mp-patch|MP Patch]]
* [[:fr:convertir un avion en dds|DDS conversion script]]
* [[DDS texture conversion]]
* [[Network tools usage]]
* [[Real Time manual]]

==external link==

[http://www.youtube.com/user/hiphiphip3 FG related youtube channel]

Mp-patch

2021-11-06T03:08:32Z

Jano: /* 2021-2022 winter offensive */

== Visual ==
Mp formation flight, two sessions on the same pc:

[[File:Mp-planes-lag-free.jpg|800px|exemple of mp formation flight lag-free]]

Mp refueling session video (with recorded planes):

[http://www.youtube.com/watch?v=kGHRDrc_n98 victor refueling two lightnings]

The first video with two mp pilots, one from canada, the other from france:

[http://www.youtube.com/watch?v=V0hb187rnWQ F-4N in the sunset]

== A bit of history ==
Long time ago, we were recording some refueling action, and it became evident that the lag was affecting our respective position, the refueling planes were very close to the tanker , but the tanker saw them quite far, as can be seen in this [http://www.youtube.com/watch?v=VWn6_RFp97Y vidéo taken from the tanker].

Then started an attempt to compensate for the lag, and smooth jitter problems (rubber band effect).

Where it can be helpful is when we need to flight close, with no rubber band effect, e.g:
* Refueling with a mp tanker
* Towing a glider, without broking the rope because of rudder band effect
* Having an aerobatic patrol

Currently this work well locally (with 127.0.0.1 as server) very smooth with YASim planes and a little less with JSBSim, for predictive time in the 0.2 s.

== The code ==
The patch found it's way in next, a bit simplified but working (no acceleration sent), you can find it by using FG development version :D

== How to use it ==
Start the patched fgfs, with the patched datas, and use the multiplayer > lag-adjust menu to set it working.
You need to have the master switch checked, and "apply to close mp" to have the close mp plane put in the future.

Both the time and distance sliders allow you to play with a general delay offset, and the distance within the mp planes start to be displayed in the future. A good start is the mpserver's ping value (need some test flight to be more precise).

For a starter, start fg with 127.0.0.1 as server, start the patch (with the "apply to close mp" checkbox, and play with the time slider to see the result on yourself as mp plane.

== Pilots using this patch ==
If you want to try the lag correction, or just see the result on a formation flight, tag along the pilots using a recent enough FG version, and doing some close flight stuff (formation, refueling etc ...)

== Parts involved ==

In order to compensate for the lag, one need to predict the plane position ahead of time, for that you need to address somehow the jitter issue, find a way to know the lag between the players, and to send plane FDM parameters which allow the prediction to be done.

=== Predictive system ===
The existing predictive system was used in very few case, most probably only when a player quit, and fg continued to interpolate 5s, then keep the last position 5s more before removing the plane.

I first tried to use the existing code, and the "few Euler steps" were quite good if the speed vector tag along the plane longitudinal axis, but degrade then we got sideslip, or big alpha.

For now i separated the position, and the orientation, having only position, speed and acceleration used for determinate the position in the future, with a simple trajectory equation, and using the rotation terms to guess the current orientation (for now i keep the old equation for this, but I'm not satisfied with the result.)

In case of jitter the old code was coping fast with the new lag value to keep the plane in the time between packets, causing the "rubber band effect", so in the current code, the response to jitter is very slow, with "jump" if we exceed 1s, this is part of what need some tune from real mp flight.
I did some test using the timestamp from mp players and assuming the clocks were sync, but the result were not so smooth as the current code.

=== FDM properties ===
The properties used by the mp code are:

* A timestamp, telling the time from the sender's side
* The position, in ECEF coordinates
* The speed, what i use is the speed in ECEF, expressed in body coordinates, for YASim i needed to remove the wind component ({{issue|201|bug}})
* The acceleration vector: initially not sent, the result was a good behaviour in straight flight, but a jumpy plane when taking "g". the YASim one seems to be quite fine, but for JSBSim, i need a better acceleration, seems i don't have the ECEF acceleration for now.
* The plane's orientation
* The angular velocity
* The angular acceleration (didn't check if it was sent)

=== FPS ===
After some tests, the time needed to have mp plane in sync with the main aircraft depend of fps (it's done using 127.0.0.1 as server, then using the slider to adjust the lag correction).

I think that the mp position is taken from the precedent frame, while the current main aircraft position is from the current frame.

This make the "in the future" plane vibrate if the fps are not steady, and this will need something in the code.

=== Mp transmission rate ===
The same as above, a change in speed rate from the sender, change the lag correction needed, i tried to adjust this by using a "0.48lag" coefficient, taken from a rough interpolation.
this is biased when the plane is reported to have 10 pps, and networks and mpservers problems make it only 0.5 pps :) , so this term will probably disappear in future version (if there are any, anyone want to help ???)

=== Time acceleration ===
Something i saw just a few days ago is that in case the mp player is using time acceleration, we need to change the speeds and accelerations accordingly, otherwise the mp plane will be seen jumping between factor 1 speed portions.
This was taken care when sending the paquets, only version 3.6 and after will be ok.

=== Server ping ===
Here's the most crucial part, we have to somehow know the time travel to and back from the server. For now, i just give some ping value in the nasal file (lag_adjust.nas), so before the flight you need to ping the server to set the value.

A better idea should be to have time information exchange between the mp pilots and the server, so as to have a real time lag value, and a good guess of the jitter.

A way to deal with this is to change the time reference used, instead of the sim time from the start, the utc time could be used, to give a common reference to all user (and this could be "quite" good using something like a ntp slaving on each fg PC). Even if in reality there is a little offset between each user leading for some to compensate more than other, this would provide good synch, without the need of server/interserver pings.
ntp slave can be done on linux, mac or windows easily with dedicaced software, so it's not something to be implemented in FG.

=== Jitter ===
Jitter is when the lag value change.

It can be seen as the difference between the sent timestamp and our clock change, for now this is addressed by a slow response to the drift, and a jump if the value is to big, it allow the sender's clock to drift a little, and make the rubber band phenomenon far less noticeable, but still a little when in very close flight.

=== Inter server lag (and jitter) ===
We have a simple way to know the lag to the server we are connected to, but no simple way to know the inter server lag.
In case the mp pilots are on 2 different mp server, we should compensate for the inter server lag, as it's far for be done , it's best to flight on formation using the same server.

Currently we don't even know on which servers are the mp pilots connected!

Notice that this could be addressed using ntp (see above).

=== VRP plane position ===

We are using the CG position, velocity and acceleration to guess the plane's futur, so we need to know the cg position relative to the vrp, and then add some calcul to obtain the cg real position, and the to have the in futur plane vrp position. for this we need to transmit the cg position.

== Utility ==

=== Spectator mode ===
Here we try to completely avoid doing prediction, by displaying the others further in the past.

This can be used when no interaction is needed, like in case of an UFO filming.

== Known bugs ==
* The behaviour is strange if some mp pilots are using time acceleration (doing jump each new packet) - should be ok now for mp planes using the futur 3.6 FG - .
* YASim planes are dancing on the ground :) with predictive system on - should be ok, i disabled prediction for small velocity value -.
* Without transmitted acceleration, the plane is more jumpy in high g figures, it's a bit better with.
* It make you addict to the patch :D (but it's a feature)

== TODO list, with no time limit :) ==

=== FG menu integration ===

=== Spectator mode ===
The idea is to display the mp planes "very late", with a strong jitter annulation, to stay in the "interpolation between packets" zone.

This can be used when you don't need to interact with others mp pilots, and need very smooth mp behaviour:

* Camera plane in a mp session
* Linux tag demo ;)

=== Lag and jitter ===
* Lag: find a way to have the server ping from fg, instead of letting the user tune the correction value.

=== JSBSim ===
* Acceleration vector suitable for the task: partially done. Currently the acceleration provided make the plane in the future "shaky", except if i head east or west , in which case mp display is very smooth. Got to use east-west oriented runways for aerobatic demos. :) I strongly suspect a problem of geocentric/geodetic rotation.

=== YASim ===
* Find acceleration suitable: done, but not included into FG yet

=== Timestamp ===

This is the biggest problem, currently the timestamp is of arbitrary origin (=0 when you start FG), and we have no way to use it to sync the mp planes with the same time reference.

Add to this the fact that when the frame time is too long (excessing the "max-sim-time-per-frame") the time stamp make a pause, so it's not a real time clock, eg when loading 3d stuff for age.

It could be better to use a realtime timestamp, with users synchronised via ntp or a gps :) .
edit 2018: that's what the new WIP does, i hope it should allow a "realtime" mode.

furthermore, the timestammp is not attached to the fdm state, ie we send the current timestamp with the fdm state computed a frame before. in case of unregular fps, this lead to unsteady mp plane. We could address this by sending our position juste after running the fdm instead of before, this would reduce the latence by a frame time, eg at 20fps, 50ms better.
edit 2018, this is not the case anymore, position sending is now done after the fdm is computed.

== FG integration ==
It went into next, curently without acceleration being sent, and without a way to add an offset per mp plane (can be added later, as this is nasal stuff). I hope this way flight test will be easier!

[[Category:Core development projects]]
[[Category:Multiplayer]]

== 2018 WIP ==

here are some informations on the "next" mp patch started in winter 2017 :) (for now it's only on my private branch mp on sourceforge). It's for my personnal reference first as i only code every few years :) but can be used to help understand what's going on...

=== timestamp ===

there's a new clock, kept on sync with the wall clock, this address the previous devious behaviour (non continuous and non linear clock with pause or time acceleration). If users are using ntpd to sync their times (based on time servers, or by e.g. a gps with pulse mode), i expect a "real time" can be introduced, where no lag or jitter is considered, but instead we just assume the other player to be in sync (or quite close) and we compute its position for the time we are in.
In recent FG, the mp packet is sent after the fdm is run, so there's no more latency here, and the timestamp used is in sync with the fdm state.

=== lag and jitter ===

Assuming both sender and receiver's clocks are more or less running at the same flow, the challenge now is to find the lag value hidden in the jitter noise.
there are two jitter sources:
* the network itself introduce some jitter.
* new packets are checked each new frame, so they can be waiting from nearly the frame dt, compared to a basic ping, it add like 0.5dt when we average the lag value. notice that for very low fps < send rate, the value is clamped to 0.51/send rate. as there are more packets arrived in one frame time.

The WIP code use a more statistical approach, we consider a mean value for the lag, which is considered the time the mp player packet arrive. Then depending the lag tuning, we compute for a more in the past or in the future time with a fixed time offset.

=== real time mode ===

using ntpd or similar to be sync to utc time should allow to not care about lag and jitter, and simply compute the mp plane's position for the time we are in. This is not tested yet, linux works fine with that, and on windows i managed to get a ntp daemon running, not sure how well it work, and the wallclock precision is no more than one millisecond (in the std::chrono implementation of system time on windows). That's not a big deal if some players have to compensate a bit more, specially if that simplify a lot the lag correction.

Notice the master-slave protocol could be improved this way, with pc in sync locally, and if we predict the plane position in the slave (not done)

=== not real time mode ===

in this case, we can't assume horloges in sync, so we need a bit of talk between players, with nasal i guess, if there is no noticeable clock drift, we can set an offset between two players, and made them talk to part the lag value in two, and agree on their respective offset.
this is simple for 2 players, but is bit more difficult to set up with more, so for now it's only made by hand :)
I'm not too sure how this will be done, one player is designated the master lag setter and other comply, or using a 3rd player to give other 2players difference as seen on the mp server ??

=== plane's position prediction ===

the current code is a bit heavy for high future value, and each time a new packet arrive, there's a little jump:
* because the plane can have diverted from the predicted position (more visible with high g turns), i don't use acceleration for now.
* because the position precision can be up to the meter as based on latitude and longitude value with a limited resolution (double?) it's a bit annoying in very close flight.

We'll see what come in the code for this, after some flights tests for sure ...

=== packet send rate ===

The player tell in mp protocol at wich speed it send it's packets, and if we want to avoid predicting the mp planes position, we have to take this in account. In this case the planes's position and float props are simply interpolated between two known positions (integer are dealt without interpolation, recently :) ).

But, the assumed send rate is not allways what we got in the reveiver side :)
* there can be lot's of udp loss, depending network charge and QoS
* depending the respective mp servers, we can only have the lazy relai once per second, or even less (i've seen one packet every3 or so second)

that's why the WIP code keep a watch on the received rate.

=== 2021-2022 winter offensive ===

The bad weather season is coming again, and so come the time to start again a bit of c++ :)

From mp sessions with the last FlightGear, using clock synchronization for the lag compensation work well, but only for some particular cases:
pilots need to use ntp, or tune the clock offset to be in a 300ms range (usually lag is in the 80ms range for same continent pilots)
fps and packet send rate have to be at a correct value, as the code go off the real time mode once we go over 300ms prediction time, making planes jumpy.
most of the pilots don't know they can improve the formation flight experience, and to be honest the lag menu is not in a friendly end user state and need a rewrite.

* on the c++ side:

the real time mode test need to be done only when receiving a new mp packet, not each frame as of now, and we need an hysteresis here to switch between real time mode and fixed lag compensation mode.
big frames may give the impression the lag made a big gap (it can be on sending side, or receiving side) so we nedd a way to follow the lag value by omitting this case.
the lag value take age to stabilize after a clock offset change, need to have a reset when change happen, either on sending or receiving side.
having a filtered acceleration send could help the predicting code, instant acceleration is pretty bad as the tests showed earlier.

* on the fgdata side

we need a simple way to select a player as time reference, if we don't use ntp, so that the clock offset is found automatically, either by comparing of lag player's value and calculating the middle point, or by a fixed lag compensation if that's not possible.
not sure we should keep the system displaying planes later when a bit far to improve visual keeping only a "spectator" mode can be enough.
keeping the planes in an interpolated zone should be done from the pps value, it's nonsensical to try to predict 2s for planes being received at 0.5 pps

== Related content ==
* [[Howto:Record, analyze and replay multiplayer flights with network tools]]
* [[Improving real close formation flight]]
* [[Real Time manual]]

Mp-patch

2021-11-06T03:07:29Z

Jano: /* 2021-2022 winter offensive */

== Visual ==
Mp formation flight, two sessions on the same pc:

[[File:Mp-planes-lag-free.jpg|800px|exemple of mp formation flight lag-free]]

Mp refueling session video (with recorded planes):

[http://www.youtube.com/watch?v=kGHRDrc_n98 victor refueling two lightnings]

The first video with two mp pilots, one from canada, the other from france:

[http://www.youtube.com/watch?v=V0hb187rnWQ F-4N in the sunset]

== A bit of history ==
Long time ago, we were recording some refueling action, and it became evident that the lag was affecting our respective position, the refueling planes were very close to the tanker , but the tanker saw them quite far, as can be seen in this [http://www.youtube.com/watch?v=VWn6_RFp97Y vidéo taken from the tanker].

Then started an attempt to compensate for the lag, and smooth jitter problems (rubber band effect).

Where it can be helpful is when we need to flight close, with no rubber band effect, e.g:
* Refueling with a mp tanker
* Towing a glider, without broking the rope because of rudder band effect
* Having an aerobatic patrol

Currently this work well locally (with 127.0.0.1 as server) very smooth with YASim planes and a little less with JSBSim, for predictive time in the 0.2 s.

== The code ==
The patch found it's way in next, a bit simplified but working (no acceleration sent), you can find it by using FG development version :D

== How to use it ==
Start the patched fgfs, with the patched datas, and use the multiplayer > lag-adjust menu to set it working.
You need to have the master switch checked, and "apply to close mp" to have the close mp plane put in the future.

Both the time and distance sliders allow you to play with a general delay offset, and the distance within the mp planes start to be displayed in the future. A good start is the mpserver's ping value (need some test flight to be more precise).

For a starter, start fg with 127.0.0.1 as server, start the patch (with the "apply to close mp" checkbox, and play with the time slider to see the result on yourself as mp plane.

== Pilots using this patch ==
If you want to try the lag correction, or just see the result on a formation flight, tag along the pilots using a recent enough FG version, and doing some close flight stuff (formation, refueling etc ...)

== Parts involved ==

In order to compensate for the lag, one need to predict the plane position ahead of time, for that you need to address somehow the jitter issue, find a way to know the lag between the players, and to send plane FDM parameters which allow the prediction to be done.

=== Predictive system ===
The existing predictive system was used in very few case, most probably only when a player quit, and fg continued to interpolate 5s, then keep the last position 5s more before removing the plane.

I first tried to use the existing code, and the "few Euler steps" were quite good if the speed vector tag along the plane longitudinal axis, but degrade then we got sideslip, or big alpha.

For now i separated the position, and the orientation, having only position, speed and acceleration used for determinate the position in the future, with a simple trajectory equation, and using the rotation terms to guess the current orientation (for now i keep the old equation for this, but I'm not satisfied with the result.)

In case of jitter the old code was coping fast with the new lag value to keep the plane in the time between packets, causing the "rubber band effect", so in the current code, the response to jitter is very slow, with "jump" if we exceed 1s, this is part of what need some tune from real mp flight.
I did some test using the timestamp from mp players and assuming the clocks were sync, but the result were not so smooth as the current code.

=== FDM properties ===
The properties used by the mp code are:

* A timestamp, telling the time from the sender's side
* The position, in ECEF coordinates
* The speed, what i use is the speed in ECEF, expressed in body coordinates, for YASim i needed to remove the wind component ({{issue|201|bug}})
* The acceleration vector: initially not sent, the result was a good behaviour in straight flight, but a jumpy plane when taking "g". the YASim one seems to be quite fine, but for JSBSim, i need a better acceleration, seems i don't have the ECEF acceleration for now.
* The plane's orientation
* The angular velocity
* The angular acceleration (didn't check if it was sent)

=== FPS ===
After some tests, the time needed to have mp plane in sync with the main aircraft depend of fps (it's done using 127.0.0.1 as server, then using the slider to adjust the lag correction).

I think that the mp position is taken from the precedent frame, while the current main aircraft position is from the current frame.

This make the "in the future" plane vibrate if the fps are not steady, and this will need something in the code.

=== Mp transmission rate ===
The same as above, a change in speed rate from the sender, change the lag correction needed, i tried to adjust this by using a "0.48lag" coefficient, taken from a rough interpolation.
this is biased when the plane is reported to have 10 pps, and networks and mpservers problems make it only 0.5 pps :) , so this term will probably disappear in future version (if there are any, anyone want to help ???)

=== Time acceleration ===
Something i saw just a few days ago is that in case the mp player is using time acceleration, we need to change the speeds and accelerations accordingly, otherwise the mp plane will be seen jumping between factor 1 speed portions.
This was taken care when sending the paquets, only version 3.6 and after will be ok.

=== Server ping ===
Here's the most crucial part, we have to somehow know the time travel to and back from the server. For now, i just give some ping value in the nasal file (lag_adjust.nas), so before the flight you need to ping the server to set the value.

A better idea should be to have time information exchange between the mp pilots and the server, so as to have a real time lag value, and a good guess of the jitter.

A way to deal with this is to change the time reference used, instead of the sim time from the start, the utc time could be used, to give a common reference to all user (and this could be "quite" good using something like a ntp slaving on each fg PC). Even if in reality there is a little offset between each user leading for some to compensate more than other, this would provide good synch, without the need of server/interserver pings.
ntp slave can be done on linux, mac or windows easily with dedicaced software, so it's not something to be implemented in FG.

=== Jitter ===
Jitter is when the lag value change.

It can be seen as the difference between the sent timestamp and our clock change, for now this is addressed by a slow response to the drift, and a jump if the value is to big, it allow the sender's clock to drift a little, and make the rubber band phenomenon far less noticeable, but still a little when in very close flight.

=== Inter server lag (and jitter) ===
We have a simple way to know the lag to the server we are connected to, but no simple way to know the inter server lag.
In case the mp pilots are on 2 different mp server, we should compensate for the inter server lag, as it's far for be done , it's best to flight on formation using the same server.

Currently we don't even know on which servers are the mp pilots connected!

Notice that this could be addressed using ntp (see above).

=== VRP plane position ===

We are using the CG position, velocity and acceleration to guess the plane's futur, so we need to know the cg position relative to the vrp, and then add some calcul to obtain the cg real position, and the to have the in futur plane vrp position. for this we need to transmit the cg position.

== Utility ==

=== Spectator mode ===
Here we try to completely avoid doing prediction, by displaying the others further in the past.

This can be used when no interaction is needed, like in case of an UFO filming.

== Known bugs ==
* The behaviour is strange if some mp pilots are using time acceleration (doing jump each new packet) - should be ok now for mp planes using the futur 3.6 FG - .
* YASim planes are dancing on the ground :) with predictive system on - should be ok, i disabled prediction for small velocity value -.
* Without transmitted acceleration, the plane is more jumpy in high g figures, it's a bit better with.
* It make you addict to the patch :D (but it's a feature)

== TODO list, with no time limit :) ==

=== FG menu integration ===

=== Spectator mode ===
The idea is to display the mp planes "very late", with a strong jitter annulation, to stay in the "interpolation between packets" zone.

This can be used when you don't need to interact with others mp pilots, and need very smooth mp behaviour:

* Camera plane in a mp session
* Linux tag demo ;)

=== Lag and jitter ===
* Lag: find a way to have the server ping from fg, instead of letting the user tune the correction value.

=== JSBSim ===
* Acceleration vector suitable for the task: partially done. Currently the acceleration provided make the plane in the future "shaky", except if i head east or west , in which case mp display is very smooth. Got to use east-west oriented runways for aerobatic demos. :) I strongly suspect a problem of geocentric/geodetic rotation.

=== YASim ===
* Find acceleration suitable: done, but not included into FG yet

=== Timestamp ===

This is the biggest problem, currently the timestamp is of arbitrary origin (=0 when you start FG), and we have no way to use it to sync the mp planes with the same time reference.

Add to this the fact that when the frame time is too long (excessing the "max-sim-time-per-frame") the time stamp make a pause, so it's not a real time clock, eg when loading 3d stuff for age.

It could be better to use a realtime timestamp, with users synchronised via ntp or a gps :) .
edit 2018: that's what the new WIP does, i hope it should allow a "realtime" mode.

furthermore, the timestammp is not attached to the fdm state, ie we send the current timestamp with the fdm state computed a frame before. in case of unregular fps, this lead to unsteady mp plane. We could address this by sending our position juste after running the fdm instead of before, this would reduce the latence by a frame time, eg at 20fps, 50ms better.
edit 2018, this is not the case anymore, position sending is now done after the fdm is computed.

== FG integration ==
It went into next, curently without acceleration being sent, and without a way to add an offset per mp plane (can be added later, as this is nasal stuff). I hope this way flight test will be easier!

[[Category:Core development projects]]
[[Category:Multiplayer]]

== 2018 WIP ==

here are some informations on the "next" mp patch started in winter 2017 :) (for now it's only on my private branch mp on sourceforge). It's for my personnal reference first as i only code every few years :) but can be used to help understand what's going on...

=== timestamp ===

there's a new clock, kept on sync with the wall clock, this address the previous devious behaviour (non continuous and non linear clock with pause or time acceleration). If users are using ntpd to sync their times (based on time servers, or by e.g. a gps with pulse mode), i expect a "real time" can be introduced, where no lag or jitter is considered, but instead we just assume the other player to be in sync (or quite close) and we compute its position for the time we are in.
In recent FG, the mp packet is sent after the fdm is run, so there's no more latency here, and the timestamp used is in sync with the fdm state.

=== lag and jitter ===

Assuming both sender and receiver's clocks are more or less running at the same flow, the challenge now is to find the lag value hidden in the jitter noise.
there are two jitter sources:
* the network itself introduce some jitter.
* new packets are checked each new frame, so they can be waiting from nearly the frame dt, compared to a basic ping, it add like 0.5dt when we average the lag value. notice that for very low fps < send rate, the value is clamped to 0.51/send rate. as there are more packets arrived in one frame time.

The WIP code use a more statistical approach, we consider a mean value for the lag, which is considered the time the mp player packet arrive. Then depending the lag tuning, we compute for a more in the past or in the future time with a fixed time offset.

=== real time mode ===

using ntpd or similar to be sync to utc time should allow to not care about lag and jitter, and simply compute the mp plane's position for the time we are in. This is not tested yet, linux works fine with that, and on windows i managed to get a ntp daemon running, not sure how well it work, and the wallclock precision is no more than one millisecond (in the std::chrono implementation of system time on windows). That's not a big deal if some players have to compensate a bit more, specially if that simplify a lot the lag correction.

Notice the master-slave protocol could be improved this way, with pc in sync locally, and if we predict the plane position in the slave (not done)

=== not real time mode ===

in this case, we can't assume horloges in sync, so we need a bit of talk between players, with nasal i guess, if there is no noticeable clock drift, we can set an offset between two players, and made them talk to part the lag value in two, and agree on their respective offset.
this is simple for 2 players, but is bit more difficult to set up with more, so for now it's only made by hand :)
I'm not too sure how this will be done, one player is designated the master lag setter and other comply, or using a 3rd player to give other 2players difference as seen on the mp server ??

=== plane's position prediction ===

the current code is a bit heavy for high future value, and each time a new packet arrive, there's a little jump:
* because the plane can have diverted from the predicted position (more visible with high g turns), i don't use acceleration for now.
* because the position precision can be up to the meter as based on latitude and longitude value with a limited resolution (double?) it's a bit annoying in very close flight.

We'll see what come in the code for this, after some flights tests for sure ...

=== packet send rate ===

The player tell in mp protocol at wich speed it send it's packets, and if we want to avoid predicting the mp planes position, we have to take this in account. In this case the planes's position and float props are simply interpolated between two known positions (integer are dealt without interpolation, recently :) ).

But, the assumed send rate is not allways what we got in the reveiver side :)
* there can be lot's of udp loss, depending network charge and QoS
* depending the respective mp servers, we can only have the lazy relai once per second, or even less (i've seen one packet every3 or so second)

that's why the WIP code keep a watch on the received rate.

=== 2021-2022 winter offensive ===

The bad weather season is coming again, and so come the time to start again a bit of c++ :)

From mp sessions with the last FlightGear, using clock synchronization for the lag compensation work well, but only for some particular cases:
pilots need to use ntp, or tune the clock offset to be in a 300ms range (usually lag is in the 80ms range for same continent pilots)
fps and packet send rate have to be at a correct value, as the code go off the real time mode once we go over 300ms prediction time, making planes jumpy.
most of the pilots don't know they can improve the formation flight experience, and to be honest the lag menu is not in a friendly end user state and need a rewrite.

* on the c++ side:

the real time mode test need to be done only when receiving a new mp packet, not each frame as of now, and we need an hysteresis here to switch between real time mode and fixed lag compensation mode.
big frames may give the impression the lag made a big gap (it can be on sending side, or receiving side) so we nedd a way to follow the lag value by omitting this case.
the lag value take age to stabilize after a clock offset change, need to have a reset when change happen, either on sending or receiving side.
having a filtered acceleration send could help the predicting code, instant acceleration is pretty bad as the tests showed earlier.

* on the fgdata side

we need a simple way to select a player as time reference, if we don't use ntp, so that the clock offset is found automatically, either by comparing of lag player's value and calculating of the middle point, or by a fixed lag compensation if that's not possible.
not sure we should keep the system displaying planes later when a bit far to improve visual keeping only a "spectator" mode can be enough.
keeping the planes in an interpolated zone should be done from the pps value, it's nonsensical to try to predict 2s for planes being received at 0.5 pps

== Related content ==
* [[Howto:Record, analyze and replay multiplayer flights with network tools]]
* [[Improving real close formation flight]]
* [[Real Time manual]]

Mp-patch

2021-11-06T03:06:37Z

Jano: /* packet send rate */ winter's job preparation

== Visual ==
Mp formation flight, two sessions on the same pc:

[[File:Mp-planes-lag-free.jpg|800px|exemple of mp formation flight lag-free]]

Mp refueling session video (with recorded planes):

[http://www.youtube.com/watch?v=kGHRDrc_n98 victor refueling two lightnings]

The first video with two mp pilots, one from canada, the other from france:

[http://www.youtube.com/watch?v=V0hb187rnWQ F-4N in the sunset]

== A bit of history ==
Long time ago, we were recording some refueling action, and it became evident that the lag was affecting our respective position, the refueling planes were very close to the tanker , but the tanker saw them quite far, as can be seen in this [http://www.youtube.com/watch?v=VWn6_RFp97Y vidéo taken from the tanker].

Then started an attempt to compensate for the lag, and smooth jitter problems (rubber band effect).

Where it can be helpful is when we need to flight close, with no rubber band effect, e.g:
* Refueling with a mp tanker
* Towing a glider, without broking the rope because of rudder band effect
* Having an aerobatic patrol

Currently this work well locally (with 127.0.0.1 as server) very smooth with YASim planes and a little less with JSBSim, for predictive time in the 0.2 s.

== The code ==
The patch found it's way in next, a bit simplified but working (no acceleration sent), you can find it by using FG development version :D

== How to use it ==
Start the patched fgfs, with the patched datas, and use the multiplayer > lag-adjust menu to set it working.
You need to have the master switch checked, and "apply to close mp" to have the close mp plane put in the future.

Both the time and distance sliders allow you to play with a general delay offset, and the distance within the mp planes start to be displayed in the future. A good start is the mpserver's ping value (need some test flight to be more precise).

For a starter, start fg with 127.0.0.1 as server, start the patch (with the "apply to close mp" checkbox, and play with the time slider to see the result on yourself as mp plane.

== Pilots using this patch ==
If you want to try the lag correction, or just see the result on a formation flight, tag along the pilots using a recent enough FG version, and doing some close flight stuff (formation, refueling etc ...)

== Parts involved ==

In order to compensate for the lag, one need to predict the plane position ahead of time, for that you need to address somehow the jitter issue, find a way to know the lag between the players, and to send plane FDM parameters which allow the prediction to be done.

=== Predictive system ===
The existing predictive system was used in very few case, most probably only when a player quit, and fg continued to interpolate 5s, then keep the last position 5s more before removing the plane.

I first tried to use the existing code, and the "few Euler steps" were quite good if the speed vector tag along the plane longitudinal axis, but degrade then we got sideslip, or big alpha.

For now i separated the position, and the orientation, having only position, speed and acceleration used for determinate the position in the future, with a simple trajectory equation, and using the rotation terms to guess the current orientation (for now i keep the old equation for this, but I'm not satisfied with the result.)

In case of jitter the old code was coping fast with the new lag value to keep the plane in the time between packets, causing the "rubber band effect", so in the current code, the response to jitter is very slow, with "jump" if we exceed 1s, this is part of what need some tune from real mp flight.
I did some test using the timestamp from mp players and assuming the clocks were sync, but the result were not so smooth as the current code.

=== FDM properties ===
The properties used by the mp code are:

* A timestamp, telling the time from the sender's side
* The position, in ECEF coordinates
* The speed, what i use is the speed in ECEF, expressed in body coordinates, for YASim i needed to remove the wind component ({{issue|201|bug}})
* The acceleration vector: initially not sent, the result was a good behaviour in straight flight, but a jumpy plane when taking "g". the YASim one seems to be quite fine, but for JSBSim, i need a better acceleration, seems i don't have the ECEF acceleration for now.
* The plane's orientation
* The angular velocity
* The angular acceleration (didn't check if it was sent)

=== FPS ===
After some tests, the time needed to have mp plane in sync with the main aircraft depend of fps (it's done using 127.0.0.1 as server, then using the slider to adjust the lag correction).

I think that the mp position is taken from the precedent frame, while the current main aircraft position is from the current frame.

This make the "in the future" plane vibrate if the fps are not steady, and this will need something in the code.

=== Mp transmission rate ===
The same as above, a change in speed rate from the sender, change the lag correction needed, i tried to adjust this by using a "0.48lag" coefficient, taken from a rough interpolation.
this is biased when the plane is reported to have 10 pps, and networks and mpservers problems make it only 0.5 pps :) , so this term will probably disappear in future version (if there are any, anyone want to help ???)

=== Time acceleration ===
Something i saw just a few days ago is that in case the mp player is using time acceleration, we need to change the speeds and accelerations accordingly, otherwise the mp plane will be seen jumping between factor 1 speed portions.
This was taken care when sending the paquets, only version 3.6 and after will be ok.

=== Server ping ===
Here's the most crucial part, we have to somehow know the time travel to and back from the server. For now, i just give some ping value in the nasal file (lag_adjust.nas), so before the flight you need to ping the server to set the value.

A better idea should be to have time information exchange between the mp pilots and the server, so as to have a real time lag value, and a good guess of the jitter.

A way to deal with this is to change the time reference used, instead of the sim time from the start, the utc time could be used, to give a common reference to all user (and this could be "quite" good using something like a ntp slaving on each fg PC). Even if in reality there is a little offset between each user leading for some to compensate more than other, this would provide good synch, without the need of server/interserver pings.
ntp slave can be done on linux, mac or windows easily with dedicaced software, so it's not something to be implemented in FG.

=== Jitter ===
Jitter is when the lag value change.

It can be seen as the difference between the sent timestamp and our clock change, for now this is addressed by a slow response to the drift, and a jump if the value is to big, it allow the sender's clock to drift a little, and make the rubber band phenomenon far less noticeable, but still a little when in very close flight.

=== Inter server lag (and jitter) ===
We have a simple way to know the lag to the server we are connected to, but no simple way to know the inter server lag.
In case the mp pilots are on 2 different mp server, we should compensate for the inter server lag, as it's far for be done , it's best to flight on formation using the same server.

Currently we don't even know on which servers are the mp pilots connected!

Notice that this could be addressed using ntp (see above).

=== VRP plane position ===

We are using the CG position, velocity and acceleration to guess the plane's futur, so we need to know the cg position relative to the vrp, and then add some calcul to obtain the cg real position, and the to have the in futur plane vrp position. for this we need to transmit the cg position.

== Utility ==

=== Spectator mode ===
Here we try to completely avoid doing prediction, by displaying the others further in the past.

This can be used when no interaction is needed, like in case of an UFO filming.

== Known bugs ==
* The behaviour is strange if some mp pilots are using time acceleration (doing jump each new packet) - should be ok now for mp planes using the futur 3.6 FG - .
* YASim planes are dancing on the ground :) with predictive system on - should be ok, i disabled prediction for small velocity value -.
* Without transmitted acceleration, the plane is more jumpy in high g figures, it's a bit better with.
* It make you addict to the patch :D (but it's a feature)

== TODO list, with no time limit :) ==

=== FG menu integration ===

=== Spectator mode ===
The idea is to display the mp planes "very late", with a strong jitter annulation, to stay in the "interpolation between packets" zone.

This can be used when you don't need to interact with others mp pilots, and need very smooth mp behaviour:

* Camera plane in a mp session
* Linux tag demo ;)

=== Lag and jitter ===
* Lag: find a way to have the server ping from fg, instead of letting the user tune the correction value.

=== JSBSim ===
* Acceleration vector suitable for the task: partially done. Currently the acceleration provided make the plane in the future "shaky", except if i head east or west , in which case mp display is very smooth. Got to use east-west oriented runways for aerobatic demos. :) I strongly suspect a problem of geocentric/geodetic rotation.

=== YASim ===
* Find acceleration suitable: done, but not included into FG yet

=== Timestamp ===

This is the biggest problem, currently the timestamp is of arbitrary origin (=0 when you start FG), and we have no way to use it to sync the mp planes with the same time reference.

Add to this the fact that when the frame time is too long (excessing the "max-sim-time-per-frame") the time stamp make a pause, so it's not a real time clock, eg when loading 3d stuff for age.

It could be better to use a realtime timestamp, with users synchronised via ntp or a gps :) .
edit 2018: that's what the new WIP does, i hope it should allow a "realtime" mode.

furthermore, the timestammp is not attached to the fdm state, ie we send the current timestamp with the fdm state computed a frame before. in case of unregular fps, this lead to unsteady mp plane. We could address this by sending our position juste after running the fdm instead of before, this would reduce the latence by a frame time, eg at 20fps, 50ms better.
edit 2018, this is not the case anymore, position sending is now done after the fdm is computed.

== FG integration ==
It went into next, curently without acceleration being sent, and without a way to add an offset per mp plane (can be added later, as this is nasal stuff). I hope this way flight test will be easier!

[[Category:Core development projects]]
[[Category:Multiplayer]]

== 2018 WIP ==

here are some informations on the "next" mp patch started in winter 2017 :) (for now it's only on my private branch mp on sourceforge). It's for my personnal reference first as i only code every few years :) but can be used to help understand what's going on...

=== timestamp ===

there's a new clock, kept on sync with the wall clock, this address the previous devious behaviour (non continuous and non linear clock with pause or time acceleration). If users are using ntpd to sync their times (based on time servers, or by e.g. a gps with pulse mode), i expect a "real time" can be introduced, where no lag or jitter is considered, but instead we just assume the other player to be in sync (or quite close) and we compute its position for the time we are in.
In recent FG, the mp packet is sent after the fdm is run, so there's no more latency here, and the timestamp used is in sync with the fdm state.

=== lag and jitter ===

Assuming both sender and receiver's clocks are more or less running at the same flow, the challenge now is to find the lag value hidden in the jitter noise.
there are two jitter sources:
* the network itself introduce some jitter.
* new packets are checked each new frame, so they can be waiting from nearly the frame dt, compared to a basic ping, it add like 0.5dt when we average the lag value. notice that for very low fps < send rate, the value is clamped to 0.51/send rate. as there are more packets arrived in one frame time.

The WIP code use a more statistical approach, we consider a mean value for the lag, which is considered the time the mp player packet arrive. Then depending the lag tuning, we compute for a more in the past or in the future time with a fixed time offset.

=== real time mode ===

using ntpd or similar to be sync to utc time should allow to not care about lag and jitter, and simply compute the mp plane's position for the time we are in. This is not tested yet, linux works fine with that, and on windows i managed to get a ntp daemon running, not sure how well it work, and the wallclock precision is no more than one millisecond (in the std::chrono implementation of system time on windows). That's not a big deal if some players have to compensate a bit more, specially if that simplify a lot the lag correction.

Notice the master-slave protocol could be improved this way, with pc in sync locally, and if we predict the plane position in the slave (not done)

=== not real time mode ===

in this case, we can't assume horloges in sync, so we need a bit of talk between players, with nasal i guess, if there is no noticeable clock drift, we can set an offset between two players, and made them talk to part the lag value in two, and agree on their respective offset.
this is simple for 2 players, but is bit more difficult to set up with more, so for now it's only made by hand :)
I'm not too sure how this will be done, one player is designated the master lag setter and other comply, or using a 3rd player to give other 2players difference as seen on the mp server ??

=== plane's position prediction ===

the current code is a bit heavy for high future value, and each time a new packet arrive, there's a little jump:
* because the plane can have diverted from the predicted position (more visible with high g turns), i don't use acceleration for now.
* because the position precision can be up to the meter as based on latitude and longitude value with a limited resolution (double?) it's a bit annoying in very close flight.

We'll see what come in the code for this, after some flights tests for sure ...

=== packet send rate ===

The player tell in mp protocol at wich speed it send it's packets, and if we want to avoid predicting the mp planes position, we have to take this in account. In this case the planes's position and float props are simply interpolated between two known positions (integer are dealt without interpolation, recently :) ).

But, the assumed send rate is not allways what we got in the reveiver side :)
* there can be lot's of udp loss, depending network charge and QoS
* depending the respective mp servers, we can only have the lazy relai once per second, or even less (i've seen one packet every3 or so second)

that's why the WIP code keep a watch on the received rate.

=== 2021-2022 winter offensive ===

The bad weather season is coming again, and so come the time to start again a bit of c++ :)

From mp sessions with the last FlightGear, using clock synchronization for the lag compensation work well, but only for some particular cases:
pilots need to use ntp, or tune the clock offset to be in a 300ms range (usually lag is in the 80ms range for same continent pilots)
fps and packet send rate have to be at a correct value, as the code go off the real time mode once we go over 300ms prediction, time making planes jumpy.
most of the pilots don't know they can improve the formation flight experience, and to be honest the lag menu is not in a friendly end user state and need a rewrite.

* on the c++ side:

the real time mode test need to be done only when receiving a new mp packet, not each frame as of now, and we need an hysteresis here to switch between real time mode and fixed lag compensation mode.
big frames may give the impression the lag made a big gap (it can be on sending side, or receiving side) so we nedd a way to follow the lag value by omitting this case.
the lag value take age to stabilize after a clock offset change, need to have a reset when change happen, either on sending or receiving side.
having a filtered acceleration send could help the predicting code, instant acceleration is pretty bad as the tests showed earlier.

* on the fgdata side

we need a simple way to select a player as time reference, if we don't use ntp, so that the clock offset is found automatically, either by comparing of lag player's value and calculating of the middle point, or by a fixed lag compensation if that's not possible.
not sure we should keep the system displaying planes later when a bit far to improve visual keeping only a "spectator" mode can be enough.
keeping the planes in an interpolated zone should be done from the pps value, it's nonsensical to try to predict 2s for planes being received at 0.5 pps

== Related content ==
* [[Howto:Record, analyze and replay multiplayer flights with network tools]]
* [[Improving real close formation flight]]
* [[Real Time manual]]

A simulink exploration

2021-10-27T21:16:19Z

Jano: /* more than 2 planes */

{{stub}}

=== A Simulink exploration (one month free trial :) ===

If you can't wait, see [https://forum.flightgear.org/viewtopic.php?f=27&t=38421 the topic on the forum] :P
here are some tips to improve visuals for simulink driven Flightgear videos.
I only used the month free trial for matlab, so i can't run it anymore, but i still have netfdm packets recorded during some simulink run.

== simulink - fg synchronization ==

I used the asbhl20 project from simulink, which is a spacial ship short final landing. Flighgear use the netfdm input, with something like:

<code>''' --native-fdm=socket,in,30,,5502,udp --fdm=null '''</code>

You need to set the destination ip and port for the netfdm output from Simulink to match Flightgear ip and port, and if you're lucky, the plane fly to land safely in Flightgear when you start a simulink run !

Notice the netfdm protocol had some change after 2020.3 FG version (some tank props were added at jsbsim request), so using a more recent Flightgear with the same simulink i used ( 2020) will end as failure.

== a not so smooth visual ==

I didn't put a video example here, but the visual flow of an externally driven Flightgear is not a fluid experience, it's more like a compilation of frozen state played one after an other.
The current netfdm implementation consist simply in having the plane's position slaved to the last netfdm position received. Depending simulink and FG fps, we can have some frames having the same position, waiting from a netfdm update, or netfdm position being skipped because FG was to slow to render or load 3D stuff.

It's bearable when the plane is solo in the air, but this lead to big jitter if we try to have two planes together (or more) and is obvious close to the ground for solo plane.

== trying to run FlightGear and Simulink in synchronization ==

Later 2020.3 releases include a way to run Flightgear more précisely on a time grid, we'll use this to have a better synchronization, where each simulink frame is displayed, and Flightgear use each simulink positions for only one frame.

* FlightGear:
Using the throttle frame rate option in the rendering menu make Flightgear starting each frame on a time grid (after 2020.3.??), if it run fast enough, and if not, it will start on the next step given from the model-hz option, so to use a time grid at 30fps, we need both the model-hz and the throttle frame rate at 30:

<code>--model-hz=30 --prop:/sim/frame-rate-model-hz=30</code>

You can see the error between the time grid and the real time the frame started with /sim/time/dt-remainder-sec in the properties, if you want to log it and see if FG is stable enough.

* Simulink:
Sadly i didn't manage to have simulink run and send the netfdm packet at a nice 30fps (maybe because of the computer old age ? or the need of the real time mode not available to me), if you need to use feedback information from FlightGear, this is something to look at, notice Simulink provide a log of the offset between the sim running time and the real running time (add a scope to the output of the "set pace" module)
i didn't need feedback from Flightgear, so i changed the block waiting for it to use a record (iirc, otherwise Simulink was waiting nearly 5s for FG packet (not sent) before starting)

Here's what my log looked like: 1s late early, closing on real time, then in a 50ms window, not precise enough for my need (the result was the same from wireshark log analysis)

[[File:Simulink pace error log.jpg|none|pace error log]]

The cheat: as i couldn't manage a stable 30fps from simulink, i simply recorded the netfdm packets (with wireshark) and replayed them at 30fps with a network tool (tcpreplay on linux)

*Both together:
With a steady netfdm flow of 30 fps and FlightGear running on a time grid, the result was way better, there are still some frame a bit late to create some stuttering, but this is something to expect on my old dual core.
FlightGear run on a time grid passing by the full second, but that's not the case for tcpreplay witch start at random (no clue for simulink) so you may need a bit of tries before having a good result, if both run at nearly the same time, you'll notice more jitter!
There's a simple way to see how both are interacting: the record of the network packets involved (wireshark or similar is your friend) if you make FlightGear send mp packet each frame, you'll see how in synchronization they are.

== Video tips ==

Using a network tool to record and replay a simulink session allow to separate the Simulink run, from the video run, you can eg run Simulink and Flightgear at 1 fps and then replay the netfdm at 30 fps for the video.

== using 2 planes ==

Next was the need for a video with two planes interacting, Both FlightGear were driven from simulink netfdm, and they are seeing the other by the mp protocol. the idea here is for each frame to run Simulink , a bit later FG1 and a bit later FG2, this way FG1 will see FG2's plane one frame late, but the result should be visually good on FG2, if it manage to receive FG1's position from the same frame.
We will use a time offset in Flightgear, to shift the time grid a bit, using:
<code>--prop:/sim/time/steady-offset-ms=16.6</code>

here it shift the time grid by 16.6 ms, half the 33.3333ms length of a 30fps frame, so FG1 and FG2 will run one after the other every 16.6ms, and depending when Simulink play in the loop, you'll have a nice video on FG1 or FG2, after probably some bad tries (Simulink/tcpreplay nearly at the same time of one of FG making jitter planes)
Notice that the clock used for the time grid is initialized by the wall clock, so having multiple FlightGear on multiple computers running on a precise timing is possible if the wall clocks are in synchronization, ntp, ptp or alike are your friends ...

== more than 2 planes ==

the best solution for displaying more than 2 planes from Simulink in FlightGear would be to directly send the mp protocol, one FlightGear displaying all the multiplayers at once.
an other way is to have each plane running a FlightGear, each sending the mp protocol to a FlightGear session used as final visual.

If Simulink doesn't output the mp protocol, a little soft to convert netfdm to mp protocol would do the job, but there's the need for a timestamp in the netfdm protocol.

If you use multiple Flightgear on the same computer, be sure they use different CPU cores, here it run always in the first core and i need to change the core affinity once started. Notice that disk access can make the frames a bit longer, so avoiding log files or the like is better, and a ssd HD is probably a good choice.

== possible improvements ==

The current netfdm protocol doesn't include a timestamp, so we lack the time information for the position we have, that's why running both in synchronization is the only way to have a smooth result as of now (end 2021)

With a timestamp and the planes's velocity added (was not sent from the project i used), the jitter between two planes externally driven could be removed as this allow to predict the mp plane's position, but that would need a bit of work in FlightGear to use this timestamp as time in the mp protocol to display the other planes, and to send in the mp packets.

[[Category:Interfacing protocols| ]]

A simulink exploration

2021-10-23T09:27:01Z

Jano: /* more than 2 planes */ possible improvements

{{stub}}

=== A Simulink exploration (one month free trial :) ===

If you can't wait, see [https://forum.flightgear.org/viewtopic.php?f=27&t=38421 the topic on the forum] :P
here are some tips to improve visuals for simulink driven Flightgear videos.
I only used the month free trial for matlab, so i can't run it anymore, but i still have netfdm packets recorded during some simulink run.

== simulink - fg synchronization ==

I used the asbhl20 project from simulink, which is a spacial ship short final landing. Flighgear use the netfdm input, with something like:

<code>''' --native-fdm=socket,in,30,,5502,udp --fdm=null '''</code>

You need to set the destination ip and port for the netfdm output from Simulink to match Flightgear ip and port, and if you're lucky, the plane fly to land safely in Flightgear when you start a simulink run !

Notice the netfdm protocol had some change after 2020.3 FG version (some tank props were added at jsbsim request), so using a more recent Flightgear with the same simulink i used ( 2020) will end as failure.

== a not so smooth visual ==

I didn't put a video example here, but the visual flow of an externally driven Flightgear is not a fluid experience, it's more like a compilation of frozen state played one after an other.
The current netfdm implementation consist simply in having the plane's position slaved to the last netfdm position received. Depending simulink and FG fps, we can have some frames having the same position, waiting from a netfdm update, or netfdm position being skipped because FG was to slow to render or load 3D stuff.

It's bearable when the plane is solo in the air, but this lead to big jitter if we try to have two planes together (or more) and is obvious close to the ground for solo plane.

== trying to run FlightGear and Simulink in synchronization ==

Later 2020.3 releases include a way to run Flightgear more précisely on a time grid, we'll use this to have a better synchronization, where each simulink frame is displayed, and Flightgear use each simulink positions for only one frame.

* FlightGear:
Using the throttle frame rate option in the rendering menu make Flightgear starting each frame on a time grid (after 2020.3.??), if it run fast enough, and if not, it will start on the next step given from the model-hz option, so to use a time grid at 30fps, we need both the model-hz and the throttle frame rate at 30:

<code>--model-hz=30 --prop:/sim/frame-rate-model-hz=30</code>

You can see the error between the time grid and the real time the frame started with /sim/time/dt-remainder-sec in the properties, if you want to log it and see if FG is stable enough.

* Simulink:
Sadly i didn't manage to have simulink run and send the netfdm packet at a nice 30fps (maybe because of the computer old age ? or the need of the real time mode not available to me), if you need to use feedback information from FlightGear, this is something to look at, notice Simulink provide a log of the offset between the sim running time and the real running time (add a scope to the output of the "set pace" module)
i didn't need feedback from Flightgear, so i changed the block waiting for it to use a record (iirc, otherwise Simulink was waiting nearly 5s for FG packet (not sent) before starting)

Here's what my log looked like: 1s late early, closing on real time, then in a 50ms window, not precise enough for my need (the result was the same from wireshark log analysis)

[[File:Simulink pace error log.jpg|none|pace error log]]

The cheat: as i couldn't manage a stable 30fps from simulink, i simply recorded the netfdm packets (with wireshark) and replayed them at 30fps with a network tool (tcpreplay on linux)

*Both together:
With a steady netfdm flow of 30 fps and FlightGear running on a time grid, the result was way better, there are still some frame a bit late to create some stuttering, but this is something to expect on my old dual core.
FlightGear run on a time grid passing by the full second, but that's not the case for tcpreplay witch start at random (no clue for simulink) so you may need a bit of tries before having a good result, if both run at nearly the same time, you'll notice more jitter!
There's a simple way to see how both are interacting: the record of the network packets involved (wireshark or similar is your friend) if you make FlightGear send mp packet each frame, you'll see how in synchronization they are.

== Video tips ==

Using a network tool to record and replay a simulink session allow to separate the Simulink run, from the video run, you can eg run Simulink and Flightgear at 1 fps and then replay the netfdm at 30 fps for the video.

== using 2 planes ==

Next was the need for a video with two planes interacting, Both FlightGear were driven from simulink netfdm, and they are seeing the other by the mp protocol. the idea here is for each frame to run Simulink , a bit later FG1 and a bit later FG2, this way FG1 will see FG2's plane one frame late, but the result should be visually good on FG2, if it manage to receive FG1's position from the same frame.
We will use a time offset in Flightgear, to shift the time grid a bit, using:
<code>--prop:/sim/time/steady-offset-ms=16.6</code>

here it shift the time grid by 16.6 ms, half the 33.3333ms length of a 30fps frame, so FG1 and FG2 will run one after the other every 16.6ms, and depending when Simulink play in the loop, you'll have a nice video on FG1 or FG2, after probably some bad tries (Simulink/tcpreplay nearly at the same time of one of FG making jitter planes)
Notice that the clock used for the time grid is initialized by the wall clock, so having multiple FlightGear on multiple computers running on a precise timing is possible if the wall clocks are in synchronization, ntp, ptp or alike are your friends ...

== more than 2 planes ==

the best solution for displaying more than 2 planes from Simulink in FlightGear would be to directly send the mp protocol, one FlightGear displaying all the multiplayers at once.
an other way is to have each plane running a FlightGear, each sending the mp protocol to a FlightGear session used as final visual.

If Simulink doesn't output the mp protocol, a little soft to convert netfdm to mp protocol would do the job, but there's the need for a timestamp in the netfdm protocol.

== possible improvements ==

The current netfdm protocol doesn't include a timestamp, so we lack the time information for the position we have, that's why running both in synchronization is the only way to have a smooth result as of now (end 2021)

With a timestamp and the planes's velocity added (was not sent from the project i used), the jitter between two planes externally driven could be removed as this allow to predict the mp plane's position, but that would need a bit of work in FlightGear to use this timestamp as time in the mp protocol to display the other planes, and to send in the mp packets.

[[Category:Interfacing protocols| ]]

A simulink exploration

2021-10-23T09:10:17Z

Jano: /* more than 2 planes */

{{stub}}

=== A Simulink exploration (one month free trial :) ===

If you can't wait, see [https://forum.flightgear.org/viewtopic.php?f=27&t=38421 the topic on the forum] :P
here are some tips to improve visuals for simulink driven Flightgear videos.
I only used the month free trial for matlab, so i can't run it anymore, but i still have netfdm packets recorded during some simulink run.

== simulink - fg synchronization ==

I used the asbhl20 project from simulink, which is a spacial ship short final landing. Flighgear use the netfdm input, with something like:

<code>''' --native-fdm=socket,in,30,,5502,udp --fdm=null '''</code>

You need to set the destination ip and port for the netfdm output from Simulink to match Flightgear ip and port, and if you're lucky, the plane fly to land safely in Flightgear when you start a simulink run !

Notice the netfdm protocol had some change after 2020.3 FG version (some tank props were added at jsbsim request), so using a more recent Flightgear with the same simulink i used ( 2020) will end as failure.

== a not so smooth visual ==

I didn't put a video example here, but the visual flow of an externally driven Flightgear is not a fluid experience, it's more like a compilation of frozen state played one after an other.
The current netfdm implementation consist simply in having the plane's position slaved to the last netfdm position received. Depending simulink and FG fps, we can have some frames having the same position, waiting from a netfdm update, or netfdm position being skipped because FG was to slow to render or load 3D stuff.

It's bearable when the plane is solo in the air, but this lead to big jitter if we try to have two planes together (or more) and is obvious close to the ground for solo plane.

== trying to run FlightGear and Simulink in synchronization ==

Later 2020.3 releases include a way to run Flightgear more précisely on a time grid, we'll use this to have a better synchronization, where each simulink frame is displayed, and Flightgear use each simulink positions for only one frame.

* FlightGear:
Using the throttle frame rate option in the rendering menu make Flightgear starting each frame on a time grid (after 2020.3.??), if it run fast enough, and if not, it will start on the next step given from the model-hz option, so to use a time grid at 30fps, we need both the model-hz and the throttle frame rate at 30:

<code>--model-hz=30 --prop:/sim/frame-rate-model-hz=30</code>

You can see the error between the time grid and the real time the frame started with /sim/time/dt-remainder-sec in the properties, if you want to log it and see if FG is stable enough.

* Simulink:
Sadly i didn't manage to have simulink run and send the netfdm packet at a nice 30fps (maybe because of the computer old age ? or the need of the real time mode not available to me), if you need to use feedback information from FlightGear, this is something to look at, notice Simulink provide a log of the offset between the sim running time and the real running time (add a scope to the output of the "set pace" module)
i didn't need feedback from Flightgear, so i changed the block waiting for it to use a record (iirc, otherwise Simulink was waiting nearly 5s for FG packet (not sent) before starting)

Here's what my log looked like: 1s late early, closing on real time, then in a 50ms window, not precise enough for my need (the result was the same from wireshark log analysis)

[[File:Simulink pace error log.jpg|none|pace error log]]

The cheat: as i couldn't manage a stable 30fps from simulink, i simply recorded the netfdm packets (with wireshark) and replayed them at 30fps with a network tool (tcpreplay on linux)

*Both together:
With a steady netfdm flow of 30 fps and FlightGear running on a time grid, the result was way better, there are still some frame a bit late to create some stuttering, but this is something to expect on my old dual core.
FlightGear run on a time grid passing by the full second, but that's not the case for tcpreplay witch start at random (no clue for simulink) so you may need a bit of tries before having a good result, if both run at nearly the same time, you'll notice more jitter!
There's a simple way to see how both are interacting: the record of the network packets involved (wireshark or similar is your friend) if you make FlightGear send mp packet each frame, you'll see how in synchronization they are.

== Video tips ==

Using a network tool to record and replay a simulink session allow to separate the Simulink run, from the video run, you can eg run Simulink and Flightgear at 1 fps and then replay the netfdm at 30 fps for the video.

== using 2 planes ==

Next was the need for a video with two planes interacting, Both FlightGear were driven from simulink netfdm, and they are seeing the other by the mp protocol. the idea here is for each frame to run Simulink , a bit later FG1 and a bit later FG2, this way FG1 will see FG2's plane one frame late, but the result should be visually good on FG2, if it manage to receive FG1's position from the same frame.
We will use a time offset in Flightgear, to shift the time grid a bit, using:
<code>--prop:/sim/time/steady-offset-ms=16.6</code>

here it shift the time grid by 16.6 ms, half the 33.3333ms length of a 30fps frame, so FG1 and FG2 will run one after the other every 16.6ms, and depending when Simulink play in the loop, you'll have a nice video on FG1 or FG2, after probably some bad tries (Simulink/tcpreplay nearly at the same time of one of FG making jitter planes)
Notice that the clock used for the time grid is initialized by the wall clock, so having multiple FlightGear on multiple computers running on a precise timing is possible if the wall clocks are in synchronization, ntp, ptp or alike are your friends ...

== more than 2 planes ==

the best solution for displaying more than 2 planes from Simulink in FlightGear would be to directly send the mp protocol, one FlightGear displaying all the multiplayers at once.
an other way is to have each plane running a FlightGear, each sending the mp protocol etc

==

[[Category:Interfacing protocols| ]]

A simulink exploration

2021-10-23T09:07:35Z

Jano: /* A Simulink exploration (one month free trial :) */

{{stub}}

=== A Simulink exploration (one month free trial :) ===

If you can't wait, see [https://forum.flightgear.org/viewtopic.php?f=27&t=38421 the topic on the forum] :P
here are some tips to improve visuals for simulink driven Flightgear videos.
I only used the month free trial for matlab, so i can't run it anymore, but i still have netfdm packets recorded during some simulink run.

== simulink - fg synchronization ==

I used the asbhl20 project from simulink, which is a spacial ship short final landing. Flighgear use the netfdm input, with something like:

<code>''' --native-fdm=socket,in,30,,5502,udp --fdm=null '''</code>

You need to set the destination ip and port for the netfdm output from Simulink to match Flightgear ip and port, and if you're lucky, the plane fly to land safely in Flightgear when you start a simulink run !

Notice the netfdm protocol had some change after 2020.3 FG version (some tank props were added at jsbsim request), so using a more recent Flightgear with the same simulink i used ( 2020) will end as failure.

== a not so smooth visual ==

I didn't put a video example here, but the visual flow of an externally driven Flightgear is not a fluid experience, it's more like a compilation of frozen state played one after an other.
The current netfdm implementation consist simply in having the plane's position slaved to the last netfdm position received. Depending simulink and FG fps, we can have some frames having the same position, waiting from a netfdm update, or netfdm position being skipped because FG was to slow to render or load 3D stuff.

It's bearable when the plane is solo in the air, but this lead to big jitter if we try to have two planes together (or more) and is obvious close to the ground for solo plane.

== trying to run FlightGear and Simulink in synchronization ==

Later 2020.3 releases include a way to run Flightgear more précisely on a time grid, we'll use this to have a better synchronization, where each simulink frame is displayed, and Flightgear use each simulink positions for only one frame.

* FlightGear:
Using the throttle frame rate option in the rendering menu make Flightgear starting each frame on a time grid (after 2020.3.??), if it run fast enough, and if not, it will start on the next step given from the model-hz option, so to use a time grid at 30fps, we need both the model-hz and the throttle frame rate at 30:

<code>--model-hz=30 --prop:/sim/frame-rate-model-hz=30</code>

You can see the error between the time grid and the real time the frame started with /sim/time/dt-remainder-sec in the properties, if you want to log it and see if FG is stable enough.

* Simulink:
Sadly i didn't manage to have simulink run and send the netfdm packet at a nice 30fps (maybe because of the computer old age ? or the need of the real time mode not available to me), if you need to use feedback information from FlightGear, this is something to look at, notice Simulink provide a log of the offset between the sim running time and the real running time (add a scope to the output of the "set pace" module)
i didn't need feedback from Flightgear, so i changed the block waiting for it to use a record (iirc, otherwise Simulink was waiting nearly 5s for FG packet (not sent) before starting)

Here's what my log looked like: 1s late early, closing on real time, then in a 50ms window, not precise enough for my need (the result was the same from wireshark log analysis)

[[File:Simulink pace error log.jpg|none|pace error log]]

The cheat: as i couldn't manage a stable 30fps from simulink, i simply recorded the netfdm packets (with wireshark) and replayed them at 30fps with a network tool (tcpreplay on linux)

*Both together:
With a steady netfdm flow of 30 fps and FlightGear running on a time grid, the result was way better, there are still some frame a bit late to create some stuttering, but this is something to expect on my old dual core.
FlightGear run on a time grid passing by the full second, but that's not the case for tcpreplay witch start at random (no clue for simulink) so you may need a bit of tries before having a good result, if both run at nearly the same time, you'll notice more jitter!
There's a simple way to see how both are interacting: the record of the network packets involved (wireshark or similar is your friend) if you make FlightGear send mp packet each frame, you'll see how in synchronization they are.

== Video tips ==

Using a network tool to record and replay a simulink session allow to separate the Simulink run, from the video run, you can eg run Simulink and Flightgear at 1 fps and then replay the netfdm at 30 fps for the video.

== using 2 planes ==

Next was the need for a video with two planes interacting, Both FlightGear were driven from simulink netfdm, and they are seeing the other by the mp protocol. the idea here is for each frame to run Simulink , a bit later FG1 and a bit later FG2, this way FG1 will see FG2's plane one frame late, but the result should be visually good on FG2, if it manage to receive FG1's position from the same frame.
We will use a time offset in Flightgear, to shift the time grid a bit, using:
<code>--prop:/sim/time/steady-offset-ms=16.6</code>

here it shift the time grid by 16.6 ms, half the 33.3333ms length of a 30fps frame, so FG1 and FG2 will run one after the other every 16.6ms, and depending when Simulink play in the loop, you'll have a nice video on FG1 or FG2, after probably some bad tries (Simulink/tcpreplay nearly at the same time of one of FG making jitter planes)
Notice that the clock used for the time grid is initialized by the wall clock, so having multiple FlightGear on multiple computers running on a precise timing is possible if the wall clocks are in synchronization, ntp, ptp or alike are your friends ...

== more than 2 planes ==

the best solution for displaying more than 2 planes from Simulink in FlightGear would be to directly send the mp protocol, one FlightGear displaying all the multiplayers at once.
an other way is to have each plane running a FlightGear, each sending the mp protocol etc (to be continued )

[[Category:Interfacing protocols| ]]

A simulink exploration

2021-10-23T09:02:06Z

Jano: /* trying to run FlightGear and Simulink in synchronization */

{{stub}}

=== A Simulink exploration (one month free trial :) ===

If you can't wait, see [[https://forum.flightgear.org/viewtopic.php?f=27&t=38421 | the topic on the forum]] :P
here are some tips to improve visuals for simulink driven Flightgear videos.
I only used the month free trial for matlab, so i can't run it anymore, but i still have netfdm packets recorded during some simulink run.

== simulink - fg synchronization ==

I used the asbhl20 project from simulink, which is a spacial ship short final landing. Flighgear use the netfdm input, with something like:

<code>''' --native-fdm=socket,in,30,,5502,udp --fdm=null '''</code>

You need to set the destination ip and port for the netfdm output from Simulink to match Flightgear ip and port, and if you're lucky, the plane fly to land safely in Flightgear when you start a simulink run !

Notice the netfdm protocol had some change after 2020.3 FG version (some tank props were added at jsbsim request), so using a more recent Flightgear with the same simulink i used ( 2020) will end as failure.

== a not so smooth visual ==

I didn't put a video example here, but the visual flow of an externally driven Flightgear is not a fluid experience, it's more like a compilation of frozen state played one after an other.
The current netfdm implementation consist simply in having the plane's position slaved to the last netfdm position received. Depending simulink and FG fps, we can have some frames having the same position, waiting from a netfdm update, or netfdm position being skipped because FG was to slow to render or load 3D stuff.

It's bearable when the plane is solo in the air, but this lead to big jitter if we try to have two planes together (or more) and is obvious close to the ground for solo plane.

== trying to run FlightGear and Simulink in synchronization ==

Later 2020.3 releases include a way to run Flightgear more précisely on a time grid, we'll use this to have a better synchronization, where each simulink frame is displayed, and Flightgear use each simulink positions for only one frame.

* FlightGear:
Using the throttle frame rate option in the rendering menu make Flightgear starting each frame on a time grid (after 2020.3.??), if it run fast enough, and if not, it will start on the next step given from the model-hz option, so to use a time grid at 30fps, we need both the model-hz and the throttle frame rate at 30:

<code>--model-hz=30 --prop:/sim/frame-rate-model-hz=30</code>

You can see the error between the time grid and the real time the frame started with /sim/time/dt-remainder-sec in the properties, if you want to log it and see if FG is stable enough.

* Simulink:
Sadly i didn't manage to have simulink run and send the netfdm packet at a nice 30fps (maybe because of the computer old age ? or the need of the real time mode not available to me), if you need to use feedback information from FlightGear, this is something to look at, notice Simulink provide a log of the offset between the sim running time and the real running time (add a scope to the output of the "set pace" module)
i didn't need feedback from Flightgear, so i changed the block waiting for it to use a record (iirc, otherwise Simulink was waiting nearly 5s for FG packet (not sent) before starting)

Here's what my log looked like: 1s late early, closing on real time, then in a 50ms window, not precise enough for my need (the result was the same from wireshark log analysis)

[[File:Simulink pace error log.jpg|none|pace error log]]

The cheat: as i couldn't manage a stable 30fps from simulink, i simply recorded the netfdm packets (with wireshark) and replayed them at 30fps with a network tool (tcpreplay on linux)

*Both together:
With a steady netfdm flow of 30 fps and FlightGear running on a time grid, the result was way better, there are still some frame a bit late to create some stuttering, but this is something to expect on my old dual core.
FlightGear run on a time grid passing by the full second, but that's not the case for tcpreplay witch start at random (no clue for simulink) so you may need a bit of tries before having a good result, if both run at nearly the same time, you'll notice more jitter!
There's a simple way to see how both are interacting: the record of the network packets involved (wireshark or similar is your friend) if you make FlightGear send mp packet each frame, you'll see how in synchronization they are.

== Video tips ==

Using a network tool to record and replay a simulink session allow to separate the Simulink run, from the video run, you can eg run Simulink and Flightgear at 1 fps and then replay the netfdm at 30 fps for the video.

== using 2 planes ==

Next was the need for a video with two planes interacting, Both FlightGear were driven from simulink netfdm, and they are seeing the other by the mp protocol. the idea here is for each frame to run Simulink , a bit later FG1 and a bit later FG2, this way FG1 will see FG2's plane one frame late, but the result should be visually good on FG2, if it manage to receive FG1's position from the same frame.
We will use a time offset in Flightgear, to shift the time grid a bit, using:
<code>--prop:/sim/time/steady-offset-ms=16.6</code>

here it shift the time grid by 16.6 ms, half the 33.3333ms length of a 30fps frame, so FG1 and FG2 will run one after the other every 16.6ms, and depending when Simulink play in the loop, you'll have a nice video on FG1 or FG2, after probably some bad tries (Simulink/tcpreplay nearly at the same time of one of FG making jitter planes)
Notice that the clock used for the time grid is initialized by the wall clock, so having multiple FlightGear on multiple computers running on a precise timing is possible if the wall clocks are in synchronization, ntp, ptp or alike are your friends ...

== more than 2 planes ==

the best solution for displaying more than 2 planes from Simulink in FlightGear would be to directly send the mp protocol, one FlightGear displaying all the multiplayers at once.
an other way is to have each plane running a FlightGear, each sending the mp protocol etc (to be continued )

[[Category:Interfacing protocols| ]]

A simulink exploration

2021-10-22T06:35:02Z

Jano: /* using 2 planes */ ntp part

{{stub}}

=== A Simulink exploration (one month free trial :) ===

If you can't wait, see [[https://forum.flightgear.org/viewtopic.php?f=27&t=38421 | the topic on the forum]] :P
here are some tips to improve visuals for simulink driven Flightgear videos.
I only used the month free trial for matlab, so i can't run it anymore, but i still have netfdm packets recorded during some simulink run.

== simulink - fg synchronization ==

I used the asbhl20 project from simulink, which is a spacial ship short final landing. Flighgear use the netfdm input, with something like:

<code>''' --native-fdm=socket,in,30,,5502,udp --fdm=null '''</code>

You need to set the destination ip and port for the netfdm output from Simulink to match Flightgear ip and port, and if you're lucky, the plane fly to land safely in Flightgear when you start a simulink run !

Notice the netfdm protocol had some change after 2020.3 FG version (some tank props were added at jsbsim request), so using a more recent Flightgear with the same simulink i used ( 2020) will end as failure.

== a not so smooth visual ==

I didn't put a video example here, but the visual flow of an externally driven Flightgear is not a fluid experience, it's more like a compilation of frozen state played one after an other.
The current netfdm implementation consist simply in having the plane's position slaved to the last netfdm position received. Depending simulink and FG fps, we can have some frames having the same position, waiting from a netfdm update, or netfdm position being skipped because FG was to slow to render or load 3D stuff.

It's bearable when the plane is solo in the air, but this lead to big jitter if we try to have two planes together (or more) and is obvious close to the ground for solo plane.

== trying to run FlightGear and Simulink in synchronization ==

Later 2020.3 releases include a way to run Flightgear more précisely on a time grid, we'll use this to have a better synchronization, where each simulink frame is displayed, and Flightgear use each simulink positions for only one frame.

* FlightGear:
Using the throttle frame rate option in the rendering menu make Flightgear starting each frame on a time grid (after 2020.3.??), if it run fast enough, and if not, it will start on the next step given from the model-hz option, so to use a time grid at 30fps, we need both the model-hz and the throttle frame rate at 30:
<code>--model-hz=30 --prop:/sim/frame-rate-model-hz=30</code>

You can see the error between the time grid and the real time the frame started with /sim/time/dt-remainder-sec in the properties, if you want to log it and see if FG is stable enough.

* Simulink:
Sadly i didn't manage to have simulink run and send the netfdm packet at a nice 30fps (maybe because of the computer old age ? or the need of the real time mode not available to me), if you need to use feedback information from FlightGear, this is something to look at, notice Simulink provide a log of the offset between the sim running time and the real running time (add a scope to the output of the "set pace" module)
i didn't need feedback from Flightgear, so i changed the block waiting for it to use a record (iirc, otherwise Simulink was waiting nearly 5s for FG packet (not sent) before starting)

Here's what my log looked like: 1s late early, closing on real time, then in a 50ms window, not precise enough for my need (the result was the same from wireshark log analysis)

[[File:Simulink pace error log.jpg|none|pace error log]]

The cheat: as i couldn't manage a stable 30fps from simulink, i simply recorded the netfdm packets (with wireshark) and replayed them at 30fps with a network tool (tcpreplay on linux)

*Both together:
With a steady netfdm flow of 30 fps and FlightGear running on a time grid, the result was way better, there are still some frame a bit late to create some stuttering, but this is something to expect on my old dual core.
FlightGear run on a time grid passing by the full second, but that's not the case for tcpreplay witch start at random (no clue for simulink) so you may need a bit of tries before having a good result, if both run at nearly the same time, you'll notice more jitter!
There's a simple way to see how both are interacting: the record of the network packets involved (wireshark or similar is your friend) if you make FlightGear send mp packet each frame, you'll see how in synchronization they are.

== Video tips ==

Using a network tool to record and replay a simulink session allow to separate the Simulink run, from the video run, you can eg run Simulink and Flightgear at 1 fps and then replay the netfdm at 30 fps for the video.

== using 2 planes ==

Next was the need for a video with two planes interacting, Both FlightGear were driven from simulink netfdm, and they are seeing the other by the mp protocol. the idea here is for each frame to run Simulink , a bit later FG1 and a bit later FG2, this way FG1 will see FG2's plane one frame late, but the result should be visually good on FG2, if it manage to receive FG1's position from the same frame.
We will use a time offset in Flightgear, to shift the time grid a bit, using:
<code>--prop:/sim/time/steady-offset-ms=16.6</code>

here it shift the time grid by 16.6 ms, half the 33.3333ms length of a 30fps frame, so FG1 and FG2 will run one after the other every 16.6ms, and depending when Simulink play in the loop, you'll have a nice video on FG1 or FG2, after probably some bad tries (Simulink/tcpreplay nearly at the same time of one of FG making jitter planes)
Notice that the clock used for the time grid is initialized by the wall clock, so having multiple FlightGear on multiple computers running on a precise timing is possible if the wall clocks are in synchronization, ntp, ptp or alike are your friends ...

== more than 2 planes ==

the best solution for displaying more than 2 planes from Simulink in FlightGear would be to directly send the mp protocol, one FlightGear displaying all the multiplayers at once.
an other way is to have each plane running a FlightGear, each sending the mp protocol etc (to be continued )

[[Category:Interfacing protocols| ]]

A simulink exploration

2021-10-21T23:29:57Z

Jano: /* trying to run FlightGear and Simulink in synchronization */

{{stub}}

=== A Simulink exploration (one month free trial :) ===

If you can't wait, see [[https://forum.flightgear.org/viewtopic.php?f=27&t=38421 | the topic on the forum]] :P
here are some tips to improve visuals for simulink driven Flightgear videos.
I only used the month free trial for matlab, so i can't run it anymore, but i still have netfdm packets recorded during some simulink run.

== simulink - fg synchronization ==

I used the asbhl20 project from simulink, which is a spacial ship short final landing. Flighgear use the netfdm input, with something like:

<code>''' --native-fdm=socket,in,30,,5502,udp --fdm=null '''</code>

You need to set the destination ip and port for the netfdm output from Simulink to match Flightgear ip and port, and if you're lucky, the plane fly to land safely in Flightgear when you start a simulink run !

Notice the netfdm protocol had some change after 2020.3 FG version (some tank props were added at jsbsim request), so using a more recent Flightgear with the same simulink i used ( 2020) will end as failure.

== a not so smooth visual ==

I didn't put a video example here, but the visual flow of an externally driven Flightgear is not a fluid experience, it's more like a compilation of frozen state played one after an other.
The current netfdm implementation consist simply in having the plane's position slaved to the last netfdm position received. Depending simulink and FG fps, we can have some frames having the same position, waiting from a netfdm update, or netfdm position being skipped because FG was to slow to render or load 3D stuff.

It's bearable when the plane is solo in the air, but this lead to big jitter if we try to have two planes together (or more) and is obvious close to the ground for solo plane.

== trying to run FlightGear and Simulink in synchronization ==

Later 2020.3 releases include a way to run Flightgear more précisely on a time grid, we'll use this to have a better synchronization, where each simulink frame is displayed, and Flightgear use each simulink positions for only one frame.

* FlightGear:
Using the throttle frame rate option in the rendering menu make Flightgear starting each frame on a time grid (after 2020.3.??), if it run fast enough, and if not, it will start on the next step given from the model-hz option, so to use a time grid at 30fps, we need both the model-hz and the throttle frame rate at 30:
<code>--model-hz=30 --prop:/sim/frame-rate-model-hz=30</code>

You can see the error between the time grid and the real time the frame started with /sim/time/dt-remainder-sec in the properties, if you want to log it and see if FG is stable enough.

* Simulink:
Sadly i didn't manage to have simulink run and send the netfdm packet at a nice 30fps (maybe because of the computer old age ? or the need of the real time mode not available to me), if you need to use feedback information from FlightGear, this is something to look at, notice Simulink provide a log of the offset between the sim running time and the real running time (add a scope to the output of the "set pace" module)
i didn't need feedback from Flightgear, so i changed the block waiting for it to use a record (iirc, otherwise Simulink was waiting nearly 5s for FG packet (not sent) before starting)

Here's what my log looked like: 1s late early, closing on real time, then in a 50ms window, not precise enough for my need (the result was the same from wireshark log analysis)

[[File:Simulink pace error log.jpg|none|pace error log]]

The cheat: as i couldn't manage a stable 30fps from simulink, i simply recorded the netfdm packets (with wireshark) and replayed them at 30fps with a network tool (tcpreplay on linux)

*Both together:
With a steady netfdm flow of 30 fps and FlightGear running on a time grid, the result was way better, there are still some frame a bit late to create some stuttering, but this is something to expect on my old dual core.
FlightGear run on a time grid passing by the full second, but that's not the case for tcpreplay witch start at random (no clue for simulink) so you may need a bit of tries before having a good result, if both run at nearly the same time, you'll notice more jitter!
There's a simple way to see how both are interacting: the record of the network packets involved (wireshark or similar is your friend) if you make FlightGear send mp packet each frame, you'll see how in synchronization they are.

== Video tips ==

Using a network tool to record and replay a simulink session allow to separate the Simulink run, from the video run, you can eg run Simulink and Flightgear at 1 fps and then replay the netfdm at 30 fps for the video.

== using 2 planes ==

Next was the need for a video with two planes interacting, Both FlightGear were driven from simulink netfdm, and they are seeing the other by the mp protocol. the idea here is for each frame to run Simulink , a bit later FG1 and a bit later FG2, this way FG1 will see FG2's plane one frame late, but the result should be visually good on FG2, if it manage to receive FG1's position from the same frame.
We will use a time offset in Flightgear, to shift the time grid a bit, using:
<code>--prop:/sim/time/steady-offset-ms=16.6</code>

here it shift the time grid by 16.6 ms, half the 33.3333ms length of a 30fps frame, so FG1 and FG2 will run one after the other every 16.6ms, and depending when Simulink play in the loop, you'll have a nice video on FG1 or FG2, after probably some bad tries (Simulink/tcpreplay nearly at the same time of one of FG making jitter planes)

== more than 2 planes ==

the best solution for displaying more than 2 planes from Simulink in FlightGear would be to directly send the mp protocol, one FlightGear displaying all the multiplayers at once.
an other way is to have each plane running a FlightGear, each sending the mp protocol etc (to be continued )

[[Category:Interfacing protocols| ]]

A simulink exploration

2021-10-21T23:28:14Z

Jano: /* trying to run FlightGear and Simulink in synchronization */ add image

{{stub}}

=== A Simulink exploration (one month free trial :) ===

If you can't wait, see [[https://forum.flightgear.org/viewtopic.php?f=27&t=38421 | the topic on the forum]] :P
here are some tips to improve visuals for simulink driven Flightgear videos.
I only used the month free trial for matlab, so i can't run it anymore, but i still have netfdm packets recorded during some simulink run.

== simulink - fg synchronization ==

I used the asbhl20 project from simulink, which is a spacial ship short final landing. Flighgear use the netfdm input, with something like:

<code>''' --native-fdm=socket,in,30,,5502,udp --fdm=null '''</code>

You need to set the destination ip and port for the netfdm output from Simulink to match Flightgear ip and port, and if you're lucky, the plane fly to land safely in Flightgear when you start a simulink run !

Notice the netfdm protocol had some change after 2020.3 FG version (some tank props were added at jsbsim request), so using a more recent Flightgear with the same simulink i used ( 2020) will end as failure.

== a not so smooth visual ==

I didn't put a video example here, but the visual flow of an externally driven Flightgear is not a fluid experience, it's more like a compilation of frozen state played one after an other.
The current netfdm implementation consist simply in having the plane's position slaved to the last netfdm position received. Depending simulink and FG fps, we can have some frames having the same position, waiting from a netfdm update, or netfdm position being skipped because FG was to slow to render or load 3D stuff.

It's bearable when the plane is solo in the air, but this lead to big jitter if we try to have two planes together (or more) and is obvious close to the ground for solo plane.

== trying to run FlightGear and Simulink in synchronization ==

Later 2020.3 releases include a way to run Flightgear more précisely on a time grid, we'll use this to have a better synchronization, where each simulink frame is displayed, and Flightgear use each simulink positions for only one frame.

* FlightGear:
Using the throttle frame rate option in the rendering menu make Flightgear starting each frame on a time grid (after 2020.3.??), if it run fast enough, and if not, it will start on the next step given from the model-hz option, so to use a time grid at 30fps, we need both the model-hz and the throttle frame rate at 30:
<code>--model-hz=30 --prop:/sim/frame-rate-model-hz=30</code>

You can see the error between the time grid and the real time the frame started with /sim/time/dt-remainder-sec in the properties, if you want to log it and see if FG is stable enough.

* Simulink:
Sadly i didn't manage to have simulink run and send the netfdm packet at a nice 30fps (maybe because of the computer old age ? or the need of the real time mode not available to me), if you need to use feedback information from FlightGear, this is something to look at, notice Simulink provide a log of the offset between the sim running time and the real running time (add a scope to the output of the "set pace" module)
i didn't need feedback from Flightgear, so i changed the block waiting for it to use a record (iirc, otherwise Simulink was waiting nearly 5s for FG packet (not sent) before starting)

Here's what my log looked like: 1s late early, closing on real time, then in a 50ms window, not precise enough for my need

[[File:Simulink pace error log.jpg|none|pace error log]]

The cheat: as i couldn't manage a stable 30fps from simulink, i simply recorded the netfdm packets (with wireshark) and replayed them at 30fps with a network tool (tcpreplay on linux)

*Both together:
With a steady netfdm flow of 30 fps and FlightGear running on a time grid, the result was way better, there are still some frame a bit late to create some stuttering, but this is something to expect on my old dual core.
FlightGear run on a time grid passing by the full second, but that's not the case for tcpreplay witch start at random (no clue for simulink) so you may need a bit of tries before having a good result, if both run at nearly the same time, you'll notice more jitter!
There's a simple way to see how both are interacting: the record of the network packets involved (wireshark or similar is your friend) if you make FlightGear send mp packet each frame, you'll see how in synchronization they are.

== Video tips ==

Using a network tool to record and replay a simulink session allow to separate the Simulink run, from the video run, you can eg run Simulink and Flightgear at 1 fps and then replay the netfdm at 30 fps for the video.

== using 2 planes ==

Next was the need for a video with two planes interacting, Both FlightGear were driven from simulink netfdm, and they are seeing the other by the mp protocol. the idea here is for each frame to run Simulink , a bit later FG1 and a bit later FG2, this way FG1 will see FG2's plane one frame late, but the result should be visually good on FG2, if it manage to receive FG1's position from the same frame.
We will use a time offset in Flightgear, to shift the time grid a bit, using:
<code>--prop:/sim/time/steady-offset-ms=16.6</code>

here it shift the time grid by 16.6 ms, half the 33.3333ms length of a 30fps frame, so FG1 and FG2 will run one after the other every 16.6ms, and depending when Simulink play in the loop, you'll have a nice video on FG1 or FG2, after probably some bad tries (Simulink/tcpreplay nearly at the same time of one of FG making jitter planes)

== more than 2 planes ==

the best solution for displaying more than 2 planes from Simulink in FlightGear would be to directly send the mp protocol, one FlightGear displaying all the multiplayers at once.
an other way is to have each plane running a FlightGear, each sending the mp protocol etc (to be continued )

[[Category:Interfacing protocols| ]]

File:Simulink pace error log.jpg

2021-10-21T23:20:21Z

Jano: Uploaded own work with UploadWizard

=={{int:filedesc}}==
{{Information
|description={{fr|1=log d'erreur entre temps de simulation et temps réel de simulink, la simulation prend une seconde de retard au début, refais son retard, puis est calé a 50ms près,}}
|date=2021-10-06 10:16:20
|source={{own}}
|author=[[User:Jano|Jano]]
|permission=
|other versions=
}}

=={{int:license-header}}==
{{self|cc-by-sa-4.0}}

A simulink exploration

2021-10-21T23:13:46Z

Jano: /* using 2 planes */

{{stub}}

=== A Simulink exploration (one month free trial :) ===

If you can't wait, see [[https://forum.flightgear.org/viewtopic.php?f=27&t=38421 | the topic on the forum]] :P
here are some tips to improve visuals for simulink driven Flightgear videos.
I only used the month free trial for matlab, so i can't run it anymore, but i still have netfdm packets recorded during some simulink run.

== simulink - fg synchronization ==

I used the asbhl20 project from simulink, which is a spacial ship short final landing. Flighgear use the netfdm input, with something like:

<code>''' --native-fdm=socket,in,30,,5502,udp --fdm=null '''</code>

You need to set the destination ip and port for the netfdm output from Simulink to match Flightgear ip and port, and if you're lucky, the plane fly to land safely in Flightgear when you start a simulink run !

Notice the netfdm protocol had some change after 2020.3 FG version (some tank props were added at jsbsim request), so using a more recent Flightgear with the same simulink i used ( 2020) will end as failure.

== a not so smooth visual ==

I didn't put a video example here, but the visual flow of an externally driven Flightgear is not a fluid experience, it's more like a compilation of frozen state played one after an other.
The current netfdm implementation consist simply in having the plane's position slaved to the last netfdm position received. Depending simulink and FG fps, we can have some frames having the same position, waiting from a netfdm update, or netfdm position being skipped because FG was to slow to render or load 3D stuff.

It's bearable when the plane is solo in the air, but this lead to big jitter if we try to have two planes together (or more) and is obvious close to the ground for solo plane.

== trying to run FlightGear and Simulink in synchronization ==

Later 2020.3 releases include a way to run Flightgear more précisely on a time grid, we'll use this to have a better synchronization, where each simulink frame is displayed, and Flightgear use each simulink positions for only one frame.

* FlightGear:
Using the throttle frame rate option in the rendering menu make Flightgear starting each frame on a time grid (after 2020.3.??), if it run fast enough, and if not, it will start on the next step given from the model-hz option, so to use a time grid at 30fps, we need both the model-hz and the throttle frame rate at 30:
<code>--model-hz=30 --prop:/sim/frame-rate-model-hz=30</code>

You can see the error between the time grid and the real time the frame started with /sim/time/dt-remainder-sec in the properties, if you want to log it and see if FG is stable enough.

* Simulink:
Sadly i didn't manage to have simulink run and send the netfdm packet at a nice 30fps (maybe because of the computer old age ? or the need of the real time mode not available to me), if you need to use feedback information from FlightGear, this is something to look at, notice Simulink provide a log of the offset between the sim running time and the real running time (add a scope to the output of the "set pace" module)
i didn't need feedback from Flightgear, so i changed the block waiting for it to use a record (iirc, otherwise Simulink was waiting nearly 5s for FG packet (not sent) before starting)

The cheat: as i couldn't manage a stable 30fps from simulink, i simply recorded the netfdm packets (with wireshark) and replayed them at 30fps with a network tool (tcpreplay on linux)

*Both together:
With a steady netfdm flow of 30 fps and FlightGear running on a time grid, the result was way better, there are still some frame a bit late to create some stuttering, but this is something to expect on my old dual core.
FlightGear run on a time grid passing by the full second, but that's not the case for tcpreplay witch start at random (no clue for simulink) so you may need a bit of tries before having a good result, if both run at nearly the same time, you'll notice more jitter!
There's a simple way to see how both are interacting: the record of the network packets involved (wireshark or similar is your friend) if you make FlightGear send mp packet each frame, you'll see how in synchronization they are.

== Video tips ==

Using a network tool to record and replay a simulink session allow to separate the Simulink run, from the video run, you can eg run Simulink and Flightgear at 1 fps and then replay the netfdm at 30 fps for the video.

== using 2 planes ==

Next was the need for a video with two planes interacting, Both FlightGear were driven from simulink netfdm, and they are seeing the other by the mp protocol. the idea here is for each frame to run Simulink , a bit later FG1 and a bit later FG2, this way FG1 will see FG2's plane one frame late, but the result should be visually good on FG2, if it manage to receive FG1's position from the same frame.
We will use a time offset in Flightgear, to shift the time grid a bit, using:
<code>--prop:/sim/time/steady-offset-ms=16.6</code>

here it shift the time grid by 16.6 ms, half the 33.3333ms length of a 30fps frame, so FG1 and FG2 will run one after the other every 16.6ms, and depending when Simulink play in the loop, you'll have a nice video on FG1 or FG2, after probably some bad tries (Simulink/tcpreplay nearly at the same time of one of FG making jitter planes)

== more than 2 planes ==

the best solution for displaying more than 2 planes from Simulink in FlightGear would be to directly send the mp protocol, one FlightGear displaying all the multiplayers at once.
an other way is to have each plane running a FlightGear, each sending the mp protocol etc (to be continued )

[[Category:Interfacing protocols| ]]

MATLAB

2021-10-21T23:12:27Z

Jano: /* Wiki articles */

'''MATLAB''' (matrix laboratory) is a numerical computing environment and fourth-generation programming language.<ref>{{cite web |url=http://en.wikipedia.org/wiki/MATLAB |title=MATLAB |publisher=Wikipedia |acessdate=17 August 2012 }}</ref>

== Simulink Aerospace Blockset ==
Since 2005, the Aerospace Blockset extension for Simulink includes an interface to visualize flight paths in [[FlightGear]].<ref>{{cite web |url=http://www10.mcadcafe.com/nbc/articles/view_article.php?section=CorpNews&articleid=192193 |title=The MathWorks Announces Aerospace Blockset 2 |date=11 July 2005 |acessdate=17 August 2012 }}</ref> Data is transmitted via UDP network packets.

When using the Generate Run Script, you may have to manually adjust the paths in the created .bat file to match your FlightGear setup.

=== Bug in Generate FlightGear Run Script ===
The Generate FlightGear Run Script block in the Aerospace Blockset creates a faulty FlightGear command line when the "Select FlightGear data flow" option is set to "Send". It is correct when set to either "Receive" or "Send-Receive". The faulty command is
--fdm=network,localhost,5501,5502,5503
This is not a valid FlightGear command and is rejected by the program. The correct commands for this would be:
--fdm=null --native-fdm=socket,in,30,localhost,5502,udp
Replace the faulty command in the generated batch file in order to run your simulation in FlightGear.

== Related content ==
=== Wiki articles ===
* [[Interfacing FlightGear]]
* [[A simulink exploration]]

=== Forum topics ===
* {{forum link|p=373898|title=Re: Created Quadcopter Model}} - Using the UFO FDM, so that the 3D model checked in FlightGear without running Simulink.

== External links ==
* [http://www.mathworks.com/products/matlab/ MATLAB]
* [http://www.mathworks.com/products/simulink/ Simulink]
* [http://www.mathworks.com/products/aeroblks/ Aerospace Blockset]
** [http://www.mathworks.com/help/aeroblks/introducing-the-flight-simulator-interface.html Aerospace Blockset FlightGear interface]
** [http://www.mathworks.nl/help/aeroblks/working-with-the-flight-simulator-interface.html Work with the Flight Simulator Interface]
* [http://www.mathworks.com/discovery/euler-angles.html Euler Angles] animated with MATLAB and FlightGear
* [http://www.mathworks.com/discovery/quaternion.html Quaternion] animated with MATLAB and FlightGear
{{appendix}}

[[Category:Software]]

MATLAB

2021-10-21T23:11:28Z

Jano: /* Wiki articles */ link to a simulink related wiki page

'''MATLAB''' (matrix laboratory) is a numerical computing environment and fourth-generation programming language.<ref>{{cite web |url=http://en.wikipedia.org/wiki/MATLAB |title=MATLAB |publisher=Wikipedia |acessdate=17 August 2012 }}</ref>

== Simulink Aerospace Blockset ==
Since 2005, the Aerospace Blockset extension for Simulink includes an interface to visualize flight paths in [[FlightGear]].<ref>{{cite web |url=http://www10.mcadcafe.com/nbc/articles/view_article.php?section=CorpNews&articleid=192193 |title=The MathWorks Announces Aerospace Blockset 2 |date=11 July 2005 |acessdate=17 August 2012 }}</ref> Data is transmitted via UDP network packets.

When using the Generate Run Script, you may have to manually adjust the paths in the created .bat file to match your FlightGear setup.

=== Bug in Generate FlightGear Run Script ===
The Generate FlightGear Run Script block in the Aerospace Blockset creates a faulty FlightGear command line when the "Select FlightGear data flow" option is set to "Send". It is correct when set to either "Receive" or "Send-Receive". The faulty command is
--fdm=network,localhost,5501,5502,5503
This is not a valid FlightGear command and is rejected by the program. The correct commands for this would be:
--fdm=null --native-fdm=socket,in,30,localhost,5502,udp
Replace the faulty command in the generated batch file in order to run your simulation in FlightGear.

== Related content ==
=== Wiki articles ===
* [[Interfacing FlightGear]]
* [[A_simulink_exploration]]

=== Forum topics ===
* {{forum link|p=373898|title=Re: Created Quadcopter Model}} - Using the UFO FDM, so that the 3D model checked in FlightGear without running Simulink.

== External links ==
* [http://www.mathworks.com/products/matlab/ MATLAB]
* [http://www.mathworks.com/products/simulink/ Simulink]
* [http://www.mathworks.com/products/aeroblks/ Aerospace Blockset]
** [http://www.mathworks.com/help/aeroblks/introducing-the-flight-simulator-interface.html Aerospace Blockset FlightGear interface]
** [http://www.mathworks.nl/help/aeroblks/working-with-the-flight-simulator-interface.html Work with the Flight Simulator Interface]
* [http://www.mathworks.com/discovery/euler-angles.html Euler Angles] animated with MATLAB and FlightGear
* [http://www.mathworks.com/discovery/quaternion.html Quaternion] animated with MATLAB and FlightGear
{{appendix}}

[[Category:Software]]

A simulink exploration

2021-10-21T22:59:11Z

Jano: /* more than 2 planes */ add categorie

{{stub}}

=== A Simulink exploration (one month free trial :) ===

If you can't wait, see [[https://forum.flightgear.org/viewtopic.php?f=27&t=38421 | the topic on the forum]] :P
here are some tips to improve visuals for simulink driven Flightgear videos.
I only used the month free trial for matlab, so i can't run it anymore, but i still have netfdm packets recorded during some simulink run.

== simulink - fg synchronization ==

I used the asbhl20 project from simulink, which is a spacial ship short final landing. Flighgear use the netfdm input, with something like:

<code>''' --native-fdm=socket,in,30,,5502,udp --fdm=null '''</code>

You need to set the destination ip and port for the netfdm output from Simulink to match Flightgear ip and port, and if you're lucky, the plane fly to land safely in Flightgear when you start a simulink run !

Notice the netfdm protocol had some change after 2020.3 FG version (some tank props were added at jsbsim request), so using a more recent Flightgear with the same simulink i used ( 2020) will end as failure.

== a not so smooth visual ==

I didn't put a video example here, but the visual flow of an externally driven Flightgear is not a fluid experience, it's more like a compilation of frozen state played one after an other.
The current netfdm implementation consist simply in having the plane's position slaved to the last netfdm position received. Depending simulink and FG fps, we can have some frames having the same position, waiting from a netfdm update, or netfdm position being skipped because FG was to slow to render or load 3D stuff.

It's bearable when the plane is solo in the air, but this lead to big jitter if we try to have two planes together (or more) and is obvious close to the ground for solo plane.

== trying to run FlightGear and Simulink in synchronization ==

Later 2020.3 releases include a way to run Flightgear more précisely on a time grid, we'll use this to have a better synchronization, where each simulink frame is displayed, and Flightgear use each simulink positions for only one frame.

* FlightGear:
Using the throttle frame rate option in the rendering menu make Flightgear starting each frame on a time grid (after 2020.3.??), if it run fast enough, and if not, it will start on the next step given from the model-hz option, so to use a time grid at 30fps, we need both the model-hz and the throttle frame rate at 30:
<code>--model-hz=30 --prop:/sim/frame-rate-model-hz=30</code>

You can see the error between the time grid and the real time the frame started with /sim/time/dt-remainder-sec in the properties, if you want to log it and see if FG is stable enough.

* Simulink:
Sadly i didn't manage to have simulink run and send the netfdm packet at a nice 30fps (maybe because of the computer old age ? or the need of the real time mode not available to me), if you need to use feedback information from FlightGear, this is something to look at, notice Simulink provide a log of the offset between the sim running time and the real running time (add a scope to the output of the "set pace" module)
i didn't need feedback from Flightgear, so i changed the block waiting for it to use a record (iirc, otherwise Simulink was waiting nearly 5s for FG packet (not sent) before starting)

The cheat: as i couldn't manage a stable 30fps from simulink, i simply recorded the netfdm packets (with wireshark) and replayed them at 30fps with a network tool (tcpreplay on linux)

*Both together:
With a steady netfdm flow of 30 fps and FlightGear running on a time grid, the result was way better, there are still some frame a bit late to create some stuttering, but this is something to expect on my old dual core.
FlightGear run on a time grid passing by the full second, but that's not the case for tcpreplay witch start at random (no clue for simulink) so you may need a bit of tries before having a good result, if both run at nearly the same time, you'll notice more jitter!
There's a simple way to see how both are interacting: the record of the network packets involved (wireshark or similar is your friend) if you make FlightGear send mp packet each frame, you'll see how in synchronization they are.

== Video tips ==

Using a network tool to record and replay a simulink session allow to separate the Simulink run, from the video run, you can eg run Simulink and Flightgear at 1 fps and then replay the netfdm at 30 fps for the video.

== using 2 planes ==

Next was the need for a video with two planes interacting, Both FlightGear were driven from simulink netfdm, and they are seeing the other by the mp protocol. the idea here is for each frame to run Simulink , a bit later FG1 and a bit later FG2, this way FG1 will see FG2's plane one frame late, but the result should be visually good on FG2, if it manage to receive FG1's position from the same frame.
We will use a time offset in Flightgear, to shift the time grid a bit, using:
<code>--prop:/sim/time/steady-offset-ms=16.6</code>
here it shift the time grid by 16.6 ms, half the 33.3333ms length of a 30fps frame, so FG1 and FG2 will run one after the other every 16.6ms, and depending when Simulink play in the loop, you'll have a nice video on FG1 or FG2, after probably some bad tries (Simulink/tcpreplay nearly at the same time of one of FG making jitter planes)

== more than 2 planes ==

the best solution for displaying more than 2 planes from Simulink in FlightGear would be to directly send the mp protocol, one FlightGear displaying all the multiplayers at once.
an other way is to have each plane running a FlightGear, each sending the mp protocol etc (to be continued )

[[Category:Interfacing protocols| ]]

A simulink exploration

2021-10-21T22:46:53Z

Jano: /* a not so smooth visual */

{{stub}}

=== A Simulink exploration (one month free trial :) ===

If you can't wait, see [[https://forum.flightgear.org/viewtopic.php?f=27&t=38421 | the topic on the forum]] :P
here are some tips to improve visuals for simulink driven Flightgear videos.
I only used the month free trial for matlab, so i can't run it anymore, but i still have netfdm packets recorded during some simulink run.

== simulink - fg synchronization ==

I used the asbhl20 project from simulink, which is a spacial ship short final landing. Flighgear use the netfdm input, with something like:

<code>''' --native-fdm=socket,in,30,,5502,udp --fdm=null '''</code>

You need to set the destination ip and port for the netfdm output from Simulink to match Flightgear ip and port, and if you're lucky, the plane fly to land safely in Flightgear when you start a simulink run !

Notice the netfdm protocol had some change after 2020.3 FG version (some tank props were added at jsbsim request), so using a more recent Flightgear with the same simulink i used ( 2020) will end as failure.

== a not so smooth visual ==

I didn't put a video example here, but the visual flow of an externally driven Flightgear is not a fluid experience, it's more like a compilation of frozen state played one after an other.
The current netfdm implementation consist simply in having the plane's position slaved to the last netfdm position received. Depending simulink and FG fps, we can have some frames having the same position, waiting from a netfdm update, or netfdm position being skipped because FG was to slow to render or load 3D stuff.

It's bearable when the plane is solo in the air, but this lead to big jitter if we try to have two planes together (or more) and is obvious close to the ground for solo plane.

== trying to run FlightGear and Simulink in synchronization ==

Later 2020.3 releases include a way to run Flightgear more précisely on a time grid, we'll use this to have a better synchronization, where each simulink frame is displayed, and Flightgear use each simulink positions for only one frame.

* FlightGear:
Using the throttle frame rate option in the rendering menu make Flightgear starting each frame on a time grid (after 2020.3.??), if it run fast enough, and if not, it will start on the next step given from the model-hz option, so to use a time grid at 30fps, we need both the model-hz and the throttle frame rate at 30:
<code>--model-hz=30 --prop:/sim/frame-rate-model-hz=30</code>

You can see the error between the time grid and the real time the frame started with /sim/time/dt-remainder-sec in the properties, if you want to log it and see if FG is stable enough.

* Simulink:
Sadly i didn't manage to have simulink run and send the netfdm packet at a nice 30fps (maybe because of the computer old age ? or the need of the real time mode not available to me), if you need to use feedback information from FlightGear, this is something to look at, notice Simulink provide a log of the offset between the sim running time and the real running time (add a scope to the output of the "set pace" module)
i didn't need feedback from Flightgear, so i changed the block waiting for it to use a record (iirc, otherwise Simulink was waiting nearly 5s for FG packet (not sent) before starting)

The cheat: as i couldn't manage a stable 30fps from simulink, i simply recorded the netfdm packets (with wireshark) and replayed them at 30fps with a network tool (tcpreplay on linux)

*Both together:
With a steady netfdm flow of 30 fps and FlightGear running on a time grid, the result was way better, there are still some frame a bit late to create some stuttering, but this is something to expect on my old dual core.
FlightGear run on a time grid passing by the full second, but that's not the case for tcpreplay witch start at random (no clue for simulink) so you may need a bit of tries before having a good result, if both run at nearly the same time, you'll notice more jitter!
There's a simple way to see how both are interacting: the record of the network packets involved (wireshark or similar is your friend) if you make FlightGear send mp packet each frame, you'll see how in synchronization they are.

== Video tips ==

Using a network tool to record and replay a simulink session allow to separate the Simulink run, from the video run, you can eg run Simulink and Flightgear at 1 fps and then replay the netfdm at 30 fps for the video.

== using 2 planes ==

Next was the need for a video with two planes interacting, Both FlightGear were driven from simulink netfdm, and they are seeing the other by the mp protocol. the idea here is for each frame to run Simulink , a bit later FG1 and a bit later FG2, this way FG1 will see FG2's plane one frame late, but the result should be visually good on FG2, if it manage to receive FG1's position from the same frame.
We will use a time offset in Flightgear, to shift the time grid a bit, using:
<code>--prop:/sim/time/steady-offset-ms=16.6</code>
here it shift the time grid by 16.6 ms, half the 33.3333ms length of a 30fps frame, so FG1 and FG2 will run one after the other every 16.6ms, and depending when Simulink play in the loop, you'll have a nice video on FG1 or FG2, after probably some bad tries (Simulink/tcpreplay nearly at the same time of one of FG making jitter planes)

== more than 2 planes ==

the best solution for displaying more than 2 planes from Simulink in FlightGear would be to directly send the mp protocol, one FlightGear displaying all the multiplayers at once.
an other way is to have each plane running a FlightGear, each sending the mp protocol etc (to be continued )

A simulink exploration

2021-10-21T22:45:59Z

Jano: /* Video tips */ 2 planes

{{stub}}

=== A Simulink exploration (one month free trial :) ===

If you can't wait, see [[https://forum.flightgear.org/viewtopic.php?f=27&t=38421 | the topic on the forum]] :P
here are some tips to improve visuals for simulink driven Flightgear videos.
I only used the month free trial for matlab, so i can't run it anymore, but i still have netfdm packets recorded during some simulink run.

== simulink - fg synchronization ==

I used the asbhl20 project from simulink, which is a spacial ship short final landing. Flighgear use the netfdm input, with something like:

<code>''' --native-fdm=socket,in,30,,5502,udp --fdm=null '''</code>

You need to set the destination ip and port for the netfdm output from Simulink to match Flightgear ip and port, and if you're lucky, the plane fly to land safely in Flightgear when you start a simulink run !

Notice the netfdm protocol had some change after 2020.3 FG version (some tank props were added at jsbsim request), so using a more recent Flightgear with the same simulink i used ( 2020) will end as failure.

== a not so smooth visual ==

I didn't put a video example here, but the visual flow of an externally driven Flightgear is not a fluid experience, it's more like a compilation of frozen state played one after an other, let's have a look !
The current netfdm implementation consist simply in having the plane's position slaved to the last netfdm position received. Depending simulink and FG fps, we can have some frames having the same position, waiting from a netfdm update, or netfdm position being skipped because FG was to slow to render or load 3D stuff.

It's bearable when the plane is solo in the air, but this lead to big jitter if we try to have two planes together (or more) and is obvious close to the ground for solo plane.

== trying to run FlightGear and Simulink in synchronization ==

Later 2020.3 releases include a way to run Flightgear more précisely on a time grid, we'll use this to have a better synchronization, where each simulink frame is displayed, and Flightgear use each simulink positions for only one frame.

* FlightGear:
Using the throttle frame rate option in the rendering menu make Flightgear starting each frame on a time grid (after 2020.3.??), if it run fast enough, and if not, it will start on the next step given from the model-hz option, so to use a time grid at 30fps, we need both the model-hz and the throttle frame rate at 30:
<code>--model-hz=30 --prop:/sim/frame-rate-model-hz=30</code>

You can see the error between the time grid and the real time the frame started with /sim/time/dt-remainder-sec in the properties, if you want to log it and see if FG is stable enough.

* Simulink:
Sadly i didn't manage to have simulink run and send the netfdm packet at a nice 30fps (maybe because of the computer old age ? or the need of the real time mode not available to me), if you need to use feedback information from FlightGear, this is something to look at, notice Simulink provide a log of the offset between the sim running time and the real running time (add a scope to the output of the "set pace" module)
i didn't need feedback from Flightgear, so i changed the block waiting for it to use a record (iirc, otherwise Simulink was waiting nearly 5s for FG packet (not sent) before starting)

The cheat: as i couldn't manage a stable 30fps from simulink, i simply recorded the netfdm packets (with wireshark) and replayed them at 30fps with a network tool (tcpreplay on linux)

*Both together:
With a steady netfdm flow of 30 fps and FlightGear running on a time grid, the result was way better, there are still some frame a bit late to create some stuttering, but this is something to expect on my old dual core.
FlightGear run on a time grid passing by the full second, but that's not the case for tcpreplay witch start at random (no clue for simulink) so you may need a bit of tries before having a good result, if both run at nearly the same time, you'll notice more jitter!
There's a simple way to see how both are interacting: the record of the network packets involved (wireshark or similar is your friend) if you make FlightGear send mp packet each frame, you'll see how in synchronization they are.

== Video tips ==

Using a network tool to record and replay a simulink session allow to separate the Simulink run, from the video run, you can eg run Simulink and Flightgear at 1 fps and then replay the netfdm at 30 fps for the video.

== using 2 planes ==

Next was the need for a video with two planes interacting, Both FlightGear were driven from simulink netfdm, and they are seeing the other by the mp protocol. the idea here is for each frame to run Simulink , a bit later FG1 and a bit later FG2, this way FG1 will see FG2's plane one frame late, but the result should be visually good on FG2, if it manage to receive FG1's position from the same frame.
We will use a time offset in Flightgear, to shift the time grid a bit, using:
<code>--prop:/sim/time/steady-offset-ms=16.6</code>
here it shift the time grid by 16.6 ms, half the 33.3333ms length of a 30fps frame, so FG1 and FG2 will run one after the other every 16.6ms, and depending when Simulink play in the loop, you'll have a nice video on FG1 or FG2, after probably some bad tries (Simulink/tcpreplay nearly at the same time of one of FG making jitter planes)

== more than 2 planes ==

the best solution for displaying more than 2 planes from Simulink in FlightGear would be to directly send the mp protocol, one FlightGear displaying all the multiplayers at once.
an other way is to have each plane running a FlightGear, each sending the mp protocol etc (to be continued )

A simulink exploration

2021-10-21T22:23:22Z

Jano: /* trying to run FlightGear and Simulink in synchronization */ main content

{{stub}}

=== A Simulink exploration (one month free trial :) ===

If you can't wait, see [[https://forum.flightgear.org/viewtopic.php?f=27&t=38421 | the topic on the forum]] :P
here are some tips to improve visuals for simulink driven Flightgear videos.
I only used the month free trial for matlab, so i can't run it anymore, but i still have netfdm packets recorded during some simulink run.

== simulink - fg synchronization ==

I used the asbhl20 project from simulink, which is a spacial ship short final landing. Flighgear use the netfdm input, with something like:

<code>''' --native-fdm=socket,in,30,,5502,udp --fdm=null '''</code>

You need to set the destination ip and port for the netfdm output from Simulink to match Flightgear ip and port, and if you're lucky, the plane fly to land safely in Flightgear when you start a simulink run !

Notice the netfdm protocol had some change after 2020.3 FG version (some tank props were added at jsbsim request), so using a more recent Flightgear with the same simulink i used ( 2020) will end as failure.

== a not so smooth visual ==

I didn't put a video example here, but the visual flow of an externally driven Flightgear is not a fluid experience, it's more like a compilation of frozen state played one after an other, let's have a look !
The current netfdm implementation consist simply in having the plane's position slaved to the last netfdm position received. Depending simulink and FG fps, we can have some frames having the same position, waiting from a netfdm update, or netfdm position being skipped because FG was to slow to render or load 3D stuff.

It's bearable when the plane is solo in the air, but this lead to big jitter if we try to have two planes together (or more) and is obvious close to the ground for solo plane.

== trying to run FlightGear and Simulink in synchronization ==

Later 2020.3 releases include a way to run Flightgear more précisely on a time grid, we'll use this to have a better synchronization, where each simulink frame is displayed, and Flightgear use each simulink positions for only one frame.

* FlightGear:
Using the throttle frame rate option in the rendering menu make Flightgear starting each frame on a time grid (after 2020.3.??), if it run fast enough, and if not, it will start on the next step given from the model-hz option, so to use a time grid at 30fps, we need both the model-hz and the throttle frame rate at 30:
<code>--model-hz=30 --prop:/sim/frame-rate-model-hz=30</code>

You can see the error between the time grid and the real time the frame started with /sim/time/dt-remainder-sec in the properties, if you want to log it and see if FG is stable enough.

* Simulink:
Sadly i didn't manage to have simulink run and send the netfdm packet at a nice 30fps (maybe because of the computer old age ? or the need of the real time mode not available to me), if you need to use feedback information from FlightGear, this is something to look at, notice Simulink provide a log of the offset between the sim running time and the real running time (add a scope to the output of the "set pace" module)
i didn't need feedback from Flightgear, so i changed the block waiting for it to use a record (iirc, otherwise Simulink was waiting nearly 5s for FG packet (not sent) before starting)

The cheat: as i couldn't manage a stable 30fps from simulink, i simply recorded the netfdm packets (with wireshark) and replayed them at 30fps with a network tool (tcpreplay on linux)

*Both together:
With a steady netfdm flow of 30 fps and FlightGear running on a time grid, the result was way better, there are still some frame a bit late to create some stuttering, but this is something to expect on my old dual core.
FlightGear run on a time grid passing by the full second, but that's not the case for tcpreplay witch start at random (no clue for simulink) so you may need a bit of tries before having a good result, if both run at nearly the same time, you'll notice more jitter!
There's a simple way to see how both are interacting: the record of the network packets involved (wireshark or similar is your friend) if you make FlightGear send mp packet each frame, you'll see how in synchronization they are.

== Video tips ==

Using a network tool to record and replay a simulink session allow to separate the Simulink run, from the video run, you can eg run Simulink and Flightgear at 1 fps and then replay the netfdm at 30 fps for the video.

A simulink exploration

2021-10-21T21:37:26Z

Jano: /* FG model-hz option */ running fg and simulink in synch

A simulink exploration

2021-10-12T05:17:34Z

Jano: /* a not so smooth visual */ main content

A simulink exploration

2021-10-12T04:45:30Z

Jano: netfdm version

A simulink exploration

2021-10-12T04:36:34Z

Jano: initial content

A simulink exploration

2021-10-12T04:32:01Z

Jano: command line

A simulink exploration

2021-10-12T03:58:50Z

Jano: /* A Simulink exploration (one month free trial :) */

A simulink exploration

2021-10-06T09:49:01Z

Jano: /* A Simulink exploration (one month free trial :) */

A simulink exploration

2021-01-15T22:35:32Z

Jano: /* A Simulink exploration (one month free trial :) */

{{stub}}

=== A Simulink exploration (one month free trial :) ===

If you can't wait, see [[https://forum.flightgear.org/viewtopic.php?f=27&t=38421 | the topic on the forum]] :P

A simulink exploration

2021-01-15T22:26:34Z

Jano: /* A Simulink exploration (one month free trial :) */

=== A Simulink exploration (one month free trial :) ===

If you can't wait, see [[https://forum.flightgear.org/viewtopic.php?f=27&t=38421 | the topic on the forum]] :P

A simulink exploration

2021-01-15T22:24:30Z

Jano: title

=== A Simulink exploration (one month free trial :) ===

Clock story in TimeManager

2021-01-15T22:20:33Z

Jano: /* 2021 (hopefully) */ main text

{{stub}}

===Clock story, an exploration of timeManager.cxx===

We'll make a travel in the clock's country, to understand what's happening in [https://sourceforge.net/p/flightgear/flightgear/ci/next/tree/src/Time/TimeManager.cxx TimeManager.cxx]. Initially only dedicated to calculate the frame time increment (dt) passed to lot's of subsystems in the update loops, but now having a responsibility in time synchronization between different FG session.

=Prior 2018=

The simulation time: the simulated time clock was something basic, we started at 0, do some clamping on the delta, (sim/max-simtime-per-frame) then only allow dt to be multiple of 1/modelHz (120 by default) this give a "dtRemainder" difference between the time we mesured, and the time we keep as timestamp;
Then is applied time acceleration, giving a sim time unrelated to real time flow, for time accel != 1

we have a clock :
*having 1/modelHz as basic step (before time accel)
*able to pause
*suffering for acceleration if time accel is used

This simulation time was used for the flight simulation as intended, but was the timestamp reference for mp protocol.
That's what lead to the 2018 change, the need to a clock for the mp protocol, not based on simulated time, but a "real time" timestamp.

= 2018 - 2020 =

The multiplayer achitecture (multi server), and the slow mp packet sending rate made impossible a good synchronization with only a server ping, so we moved to a "real time" way of addressing lag issues, helped by the introduction of 2 clocks in TimeManager.

We still have the same way of providing the simulation time dt, with the same characteristics as before, but we added:
* a "steady" clock: this clock start synchronized to the wall clock, then follow the "dt" increment, without pausing and accelerating, using a monotonic timer. The goal is to have the wall clock tuned by ntp or ptp, and the different mp pilot are in time with each other by relying on dedicated network time service.
We provide a way to know if the steady clock is still on time with the wall clock, setting "sim/time/compute-clock-drift" to "true" give the current difference in "sim/time/steady-clock-drift-ms"

* the mpprotocol clock: if the ntp works well, it's the same as the steady clock, but to cope with player not sing ntp, or eg flying with a recorded flight, there's an offset to adjust this mpprotocol clock relative to the steady clock (sim/time/mp-clock-offset_sec). this clock give us the timestamp used in the mp protocol.
This offset is exposed in the lag menu, if you need it :) (more to see [[Real_Time_manual | here]]).

This was a bit of a headhache to add in the TimeManager, but the result is quite good (for who know how to set it up, still need improvement for the end user)

Maybe you've noticed that those changes were made only during winter, so the question is, what needed to be changed this winter ?

=2021 (hopefully)=

To summarize, we have a time step multiple of 1/modelHz, a steady clock based on the wall clock, and a good concept for the multiplayer lag compensation based on time concordance. So what could be wrong?

We won't deal this time with the multiplayers, but with something a bit more delicate, ie the net external protocol, and that lead to investigate precisely what was possible from a [[flightgear]] perspective.

The question was: can we make each [[flightgear]] loop stick to a time grid with a fixed step ?

This would allow multiple FG sessions to be delayed inside the frame, and in synch with some 3rd party software ([[simulink]], [[JSBSim]] standalone etc...)
Assuming the wall clocks are in synch (ntp over internet or ptp in local networks or on the same pc) there were 2 problem in Flightgear:
* the prop "/sim/frame-rate-throttle-hz" allow FG to run at a specified frame rate, but the tests showed inegal spacing between frame because:
** we have minimal step equal to 1/modelHz and if modelHz is not a multiple of Throttle-frame-rate, we'll have different length of frame time reported, even if the throttle is equally spaced.
**the waiting implementation was based on the last timestamp from the system, if we are a bit late, this accumulate and make us loose frame in the long time.

* the steady clock was initialised with the wall clock time at startup, that make it random.

there are 2 parts for the "hopefully" 2021 winter change:

* we don't start the steady clock with the value from the wall clock directly, but we are finding the timestamp just before on a time grid of 1/modelHz, and the difference goes in dtRemainder (the one we introduced in the first topic :) ) this make the steady clock pass by the full second. To allow a volontary shift of the steady clock there's an offset (sim/time/frame-time-offset-ms) used to delay in time two FG sessions.
* Throttle-frame rate has 2 change:
** we wait not a fixed time, but until the next step on the time grid (1/modelHz). If the load allow this work very well (you can log "sim/time/dt-remainder" to have an idea of far you are from the time grid.
** We don't try to follow the value of throttle-frame-rate precisely, but find the closest value giving equally spaced frame on a 1/modelHz step, ie for modelHz=120, we can have 120,60,40,30,24,20,15,10,8,6,5,3,1

Other values are possible but need a change of modelHz.

For now this work well only if we set the different prop at start, changing modelHz in sim will lead to randon steady clock offset, and the frame-time-offset-ms is only applied on the initial run and stay the same. To ensure a steady frame rate stick to precise grid, you should give the same value at modelHz and throttle-frame-rate.

Possible improvements:
* allow only a limited time windows for the timestamp we start the loop, if we're too late, just skip a frame and start on time on the next grid step.
* make the frame time offset tunable, eg to find the best value when dealing with a source with unknown offset in frame (case with simulink packet replay with tcpreplay, wich allow strict time grid, but unknown start time)

I'll let you see this in action in this [[a_simulink_exploration | simulink exploration]], when it'll be done :)

[[Category:Core_developer_documentation]]

Clock story in TimeManager

2021-01-15T20:53:22Z

Jano: /* before 2021 */ main text

Clock story in TimeManager

2021-01-15T20:12:39Z

Jano: /* Prior 2018 */

Clock story in TimeManager

2021-01-15T20:10:50Z

Jano: /* Prior 2018 */ sim time

Clock story in TimeManager

2021-01-15T19:40:53Z

Jano: Created page with "{{stub}} ===Clock story, an exploration of timeManager.cxx=== We'll make a travel in the clock's country, to understand what's happening in [https://sourceforge.net/p/flight..."

Howto:Record, analyze and replay multiplayer flights with network tools

2020-03-25T19:05:08Z

Jano: /* Replay the flight to our FlightGear session */

You can '''record, analyze and replay flights with network tools''' by recording and working on UDP packets sent through the [[multiplayer protocol]] between your computer and a [[Howto:Multiplayer|multiplayer]] server.

This article will cover some uses in multiplayer problems diagnostics and a way to record (and replay) a multiplayer session.

The truth is I never found what I needed in FlightGear itself, but just a little work with some network tools made my day.

I agree this is not easy to use, need some time to master the tools, and root access to the network interface (so take care ;) ), but it works, as can be seen in [http://www.youtube.com/watch?v=kGHRDrc_n98 this video!]

== Some examples of utilization ==
* Debriefing/training of formation flight: Your aerobatic formation can review the flight, and find who broke the diamond figure in the top of the loop :) You can even fly again with the recorded multiplayer flight to improve your skills.
* Make a film by recording the planes one by one, without any Internet connection.
* Having some "multiplayer action files" available, for example to replay them in a FlightGear demo, or if you do not have internet but want some other planes than the silly AI models.
* Save a multiplayer event and do video capture after the event. This way you can make multiple video takes and do a nice editing.
* UDP traffic analysis, to see the effect of internet transmission on other multiplayer pilot's packets (loss/jitter)

== Network tools ==
* Wireshark: A network traffic sniffer/analyzer available for all platforms. We will use it to record the FlightGear UDP traffic.
* tcpreplay: A suit of programs available on Mac and Linux, and maybe on Windows by using Cygwin. It will be used to change some UDP packet properties and to send back the traffic to another FlightGear session.

== Captureing FlightGear network traffic ==
Assuming we are connected to a [[Howto:Multiplayer#Servers|multiplayer server]] and that the server and your FlightGear session receive on UDP port 5000, we can record all the UDP packets going to a port 5000.

This way we will record our outgoing packets and the ingoing packets from the other multiplayer pilots.

We start Wireshark, then select the network interface used, and finally set a filter and apply it, for example:
UDP.dstport == 5000 && !(icmp.type)

Finally, we stop the capture and save the file as a <code>.cap</code> file.

What do we have now? We have the flight recorded at the network level!

With wireshark we can now extract some pilot(s), use a certain time interval of traffic, or see certain traffic properties.

With the help of tcpreplay/tcprewrite, we can send the multiplayer traffic back to a running FlightGear session.

== Extracting files for individual pilots ==
In Wireshark it is possible to extract each pilot flight individually. Using the filter looks like:

frame contains "callsign"

and to exclude a pilot:

!frame contains "callsign"

This way you can have each pilot's flight separated. If needed you can merge again 2 or more pilots in a file, if you for example you have recorded a diamond formation and you want to practice the left wingman position. Either you remove the existing one for the <code>.cap</code> file, or you merge the 3 others again, after having done individual <code>.cap</code> files.

Wireshark come with mergecap to merge multiple <code>.cap</code> files. Taking care the chronological order is respected. The provided editcap can be used too if you need to add a time offset to a plane to address recorded latency issues.

It is sometimes difficult to have lot of matching schedules when you are training in an aerobatic team, and we have here a good way to train with other mp pilots without being connected. You can even practice by file exchange: The leader make a flight, send the <code>.cap</code> file to the second pilot which add his flight, then pass to the third pilot etc.

== Analyzing UDP packets loss/jitter ==
[[File:Wireshark mp rate analyse.png|600px|thumb|Received Multiplayer packet rate]]
Here is a wireshark analysis showing the multiplayer packet rate for different multiplayer pilots from a recorded multiplayer session.
* The black curve got a nice 10 packets/s, our internet link was good and is maybe connected to the same multiplayer server.
* The red and pink were pretty chaotic. Their packets are often lost and I guess they are jittery in FlightGear. They were probably connected to an other multiplayer server.
* The blue and green show less than 1 packet/s, this can be explained by some connection losses between servers. By default the multiplayer server send at 1 packet/s to the other multiplayer server just to "inform" them which planes they host, and start relaying all the packets only once other multiplayer server send them planes in vicinity. In this case our multiplayer server probably did not relay to their server, and we only got the "lazy relay" from their server.{{-}}

== Replay the flight to our FlightGear session ==
Ok, we have a recorded flight saved as <code>.cap</code> file and want to see it again in FlightGear. That's where the tricky parts start. :)

=== Limitations of tcpreplay ===
To make things short, tcpreplay can not send directly to FlightGear if we are on the same computer.

Until now I either use an other PC to send traffic back to my FlightGear PC, or i send to the internet box's external IP address, with a redirection to my pc for the port used in its firewall.

According to the tcpreplay FAQ, it could work if tcpreplay runs in a virtualized machine, but I did not give it a go.

=== tcprewrite magic ===
Next is that the UDP packets. To come back to our PC, we need some editing in the UDP packets:
* The MAC address of the sender must match the MAC of the PC sending the recorded flight.
* The destination MAC address must match the receiving PC, or depending the case the MAC address of the router.
* The destination and sender IP address must match the IP address of the 2 PC involved.

We make the change in one go with tcprewrite, here's a practical example. We have:
* The computer running FlightGear, MAC address 00:11:22:33:44:55, IP address 192.168.1.12 .
* The computer we will use to send the recorded packets: MAC address 99:88:77:66:55:44, IP address 192.168.1.17

It is up to you to find your MAC and IP address. You can use wireshark on some network traffic between the two, or ifconfig on Linux etc. Here is the "magic" line,
tcprewrite -C --enet-smac=99:88:77:66:55:44 --enet-dmac=00:11:22:33:44:55
-S 0.0.0.0/0:192.168.1.17 -D 0.0.0.0/0:192.168.1.12 -i recorded.cap -o readytosend.cap

where:
* -C force a checksum recalculation. If not the packets are not valid and are not sent.
* --enet-smac gives the mac address of the sender
* --enet-dmac same with the receiver
* -S changes source IP address
* -D does the same for destination
* -i input file (the previously recorded flight)
* -o output file

tcprewrite do not need to be root to be used, but using tcpreplay to see the recorded traffic again need root access to use the network interface.

Now we start FlightGear with Multiplayer enabled by sending to a server (it can be a MPserver or the address of a valid local host on your network, for example 192.168.0.17).

Finally we start replaying the flight, on the dedicaced PC:
tcpreplay -i eth0 readytosend.cap

Be sure to use the correct name for the network interface. ifconfig/ipconfig is your friend here!

Tcpreplay works fine this way if we don't need a very good time flow sync, as it end being late, dépending on the cpu usage mostly. I didn't notice this issue at first, but this became evident using lag statistics available recently in next, so to keep a good sync between tcpreplay and flightgear, using a core affinity, setting a priority higher and preferably on a not overloaded PC/CPU core makes things better, so we got something like that on linux:

taskset 02 nice -n -15 tcpreplay -x 1.000065 -K -i enp0s10 /rab/refuel/test.pcapng

The "-x" option (time acceleration) ensure a good time sync over the replay (find yourself the best value for your config :) ), i just need to adjust the time offset for the mp clock, to have replayed planes in sync with my session, considered as "real time" buddys.

=== Give it a try ===
[[File:Fighters mont aiguille.jpg|thumb|The "mont aiguille", in the French Vercors Massif.]]
if you don't have time to make a record yourself, you can use this [http://janodesbois.free.fr/flightgear/cap_files/vercors_test.cap.bz2 flight in the French Vercors Massif] (3.4M). Start at LFLU, runway 01 with the [[UFO]] or in a plane capable of more than 400 kts, and follow the leading [[McDonnell F-4 Phantom II|F-4N]] in a Vercors Massif trip, followed by a [[English Electric Lightning|Lightning]], a [[General Dynamics F-16 Fighting Falcon|F-16]] and a [[Grumman F-14 Tomcat|F-14B]]. It is better with a detailed scenery like "La France d'el maxo".

Just decompress it, and use tcprewrite to match your config.{{-}}

== Related content ==
=== Wiki articles ===
* [[Generic protocol]] - ''A flexible way to log properties locally.''
* [[Interfacing FlightGear]] - ''Various ways to interface FlightGear.''
* [[JSBSim Logging]] - ''Logging JSBSim internal properties locally.''
* [[Logging properties]] - ''The simplest way to log properties locally.''

== External links ==
* {{cite web
| url = https://www.wireshark.org
| title = wireshark.org
}}

[[Category:Development]]
[[Category:Multiplayer]]

Howto:Record, analyze and replay multiplayer flights with network tools

2020-03-25T19:01:03Z

Jano: /* Limitations of tcpreplay */ added use of nat box

You can '''record, analyze and replay flights with network tools''' by recording and working on UDP packets sent through the [[multiplayer protocol]] between your computer and a [[Howto:Multiplayer|multiplayer]] server.

This article will cover some uses in multiplayer problems diagnostics and a way to record (and replay) a multiplayer session.

The truth is I never found what I needed in FlightGear itself, but just a little work with some network tools made my day.

I agree this is not easy to use, need some time to master the tools, and root access to the network interface (so take care ;) ), but it works, as can be seen in [http://www.youtube.com/watch?v=kGHRDrc_n98 this video!]

== Some examples of utilization ==
* Debriefing/training of formation flight: Your aerobatic formation can review the flight, and find who broke the diamond figure in the top of the loop :) You can even fly again with the recorded multiplayer flight to improve your skills.
* Make a film by recording the planes one by one, without any Internet connection.
* Having some "multiplayer action files" available, for example to replay them in a FlightGear demo, or if you do not have internet but want some other planes than the silly AI models.
* Save a multiplayer event and do video capture after the event. This way you can make multiple video takes and do a nice editing.
* UDP traffic analysis, to see the effect of internet transmission on other multiplayer pilot's packets (loss/jitter)

== Network tools ==
* Wireshark: A network traffic sniffer/analyzer available for all platforms. We will use it to record the FlightGear UDP traffic.
* tcpreplay: A suit of programs available on Mac and Linux, and maybe on Windows by using Cygwin. It will be used to change some UDP packet properties and to send back the traffic to another FlightGear session.

== Captureing FlightGear network traffic ==
Assuming we are connected to a [[Howto:Multiplayer#Servers|multiplayer server]] and that the server and your FlightGear session receive on UDP port 5000, we can record all the UDP packets going to a port 5000.

This way we will record our outgoing packets and the ingoing packets from the other multiplayer pilots.

We start Wireshark, then select the network interface used, and finally set a filter and apply it, for example:
UDP.dstport == 5000 && !(icmp.type)

Finally, we stop the capture and save the file as a <code>.cap</code> file.

What do we have now? We have the flight recorded at the network level!

With wireshark we can now extract some pilot(s), use a certain time interval of traffic, or see certain traffic properties.

With the help of tcpreplay/tcprewrite, we can send the multiplayer traffic back to a running FlightGear session.

== Extracting files for individual pilots ==
In Wireshark it is possible to extract each pilot flight individually. Using the filter looks like:

frame contains "callsign"

and to exclude a pilot:

!frame contains "callsign"

This way you can have each pilot's flight separated. If needed you can merge again 2 or more pilots in a file, if you for example you have recorded a diamond formation and you want to practice the left wingman position. Either you remove the existing one for the <code>.cap</code> file, or you merge the 3 others again, after having done individual <code>.cap</code> files.

Wireshark come with mergecap to merge multiple <code>.cap</code> files. Taking care the chronological order is respected. The provided editcap can be used too if you need to add a time offset to a plane to address recorded latency issues.

It is sometimes difficult to have lot of matching schedules when you are training in an aerobatic team, and we have here a good way to train with other mp pilots without being connected. You can even practice by file exchange: The leader make a flight, send the <code>.cap</code> file to the second pilot which add his flight, then pass to the third pilot etc.

== Analyzing UDP packets loss/jitter ==
[[File:Wireshark mp rate analyse.png|600px|thumb|Received Multiplayer packet rate]]
Here is a wireshark analysis showing the multiplayer packet rate for different multiplayer pilots from a recorded multiplayer session.
* The black curve got a nice 10 packets/s, our internet link was good and is maybe connected to the same multiplayer server.
* The red and pink were pretty chaotic. Their packets are often lost and I guess they are jittery in FlightGear. They were probably connected to an other multiplayer server.
* The blue and green show less than 1 packet/s, this can be explained by some connection losses between servers. By default the multiplayer server send at 1 packet/s to the other multiplayer server just to "inform" them which planes they host, and start relaying all the packets only once other multiplayer server send them planes in vicinity. In this case our multiplayer server probably did not relay to their server, and we only got the "lazy relay" from their server.{{-}}

== Replay the flight to our FlightGear session ==
Ok, we have a recorded flight saved as <code>.cap</code> file and want to see it again in FlightGear. That's where the tricky parts start. :)

=== Limitations of tcpreplay ===
To make things short, tcpreplay can not send directly to FlightGear if we are on the same computer.

Until now I either use an other PC to send traffic back to my FlightGear PC, or i send to the internet box's external IP address, with a redirection to my pc for the port used in its firewall.

According to the tcpreplay FAQ, it could work if tcpreplay runs in a virtualized machine, but I did not give it a go.

=== tcprewrite magic ===
Next is that the UDP packets. To come back to our PC, we need some editing in the UDP packets:
* The MAC address of the sender must match the MAC of the PC sending the recorded flight.
* The destination MAC address must match the receiving PC, or depending the case the MAC address of the router.
* The destination and sender IP address must match the IP address of the 2 PC involved.

We make the change in one go with tcprewrite, here's a practical example. We have:
* The computer running FlightGear, MAC address 00:11:22:33:44:55, IP address 192.168.1.12 .
* The computer we will use to send the recorded packets: MAC address 99:88:77:66:55:44, IP address 192.168.1.17

It is up to you to find your MAC and IP address. You can use wireshark on some network traffic between the two, or ifconfig on Linux etc. Here is the "magic" line,
tcprewrite -C --enet-smac=99:88:77:66:55:44 --enet-dmac=00:11:22:33:44:55
-S 0.0.0.0/0:192.168.1.17 -D 0.0.0.0/0:192.168.1.12 -i recorded.cap -o readytosend.cap

where:
* -C force a checksum recalculation. If not the packets are not valid and are not sent.
* --enet-smac gives the mac address of the sender
* --enet-dmac same with the receiver
* -S changes source IP address
* -D does the same for destination
* -i input file (the previously recorded flight)
* -o output file

tcprewrite do not need to be root to be used, but using tcpreplay to see the recorded traffic again need root access to use the network interface.

Now we start FlightGear with Multiplayer enabled by sending to a server (it can be a MPserver or the address of a valid local host on your network, for example 192.168.0.17).

Finally we start replaying the flight, on the dedicaced PC:
tcpreplay -i eth0 readytosend.cap

Be sure to use the correct name for the network interface. ifconfig/ipconfig is your friend here!

Tcpreplay works fine this way if we don't need a very good time flow sync, as it end being late, dépending on the cpu usage mostly. I didn't notice this issue at first, but this became evident using lag statistics available recently in next, so to keep a good sync between tcpreplay and flightgear, using a core affinity, setting a priority higher and preferably on a not overloaded PC/CPU core makes things better, so we got something like that on linux:

taskset 02 nice -n -15 tcpreplay -x 1.000065 -K -i enp0s10 /rab/refuel/test.pcapng

The "-x" option ensure a good time sync over the replay (find yourself the best value for your config :) ), i just need to adjust the time offset for the mp clock, to have replayed planes in sync with my session, considered as "real time" buddys.

=== Give it a try ===
[[File:Fighters mont aiguille.jpg|thumb|The "mont aiguille", in the French Vercors Massif.]]
if you don't have time to make a record yourself, you can use this [http://janodesbois.free.fr/flightgear/cap_files/vercors_test.cap.bz2 flight in the French Vercors Massif] (3.4M). Start at LFLU, runway 01 with the [[UFO]] or in a plane capable of more than 400 kts, and follow the leading [[McDonnell F-4 Phantom II|F-4N]] in a Vercors Massif trip, followed by a [[English Electric Lightning|Lightning]], a [[General Dynamics F-16 Fighting Falcon|F-16]] and a [[Grumman F-14 Tomcat|F-14B]]. It is better with a detailed scenery like "La France d'el maxo".

Just decompress it, and use tcprewrite to match your config.{{-}}

== Related content ==
=== Wiki articles ===
* [[Generic protocol]] - ''A flexible way to log properties locally.''
* [[Interfacing FlightGear]] - ''Various ways to interface FlightGear.''
* [[JSBSim Logging]] - ''Logging JSBSim internal properties locally.''
* [[Logging properties]] - ''The simplest way to log properties locally.''

== External links ==
* {{cite web
| url = https://www.wireshark.org
| title = wireshark.org
}}

[[Category:Development]]
[[Category:Multiplayer]]

Howto:Record, analyze and replay multiplayer flights with network tools

2020-03-25T18:47:38Z

Jano: /* tcprewrite magic */ added core affinity and nice to the replay

You can '''record, analyze and replay flights with network tools''' by recording and working on UDP packets sent through the [[multiplayer protocol]] between your computer and a [[Howto:Multiplayer|multiplayer]] server.

This article will cover some uses in multiplayer problems diagnostics and a way to record (and replay) a multiplayer session.

The truth is I never found what I needed in FlightGear itself, but just a little work with some network tools made my day.

I agree this is not easy to use, need some time to master the tools, and root access to the network interface (so take care ;) ), but it works, as can be seen in [http://www.youtube.com/watch?v=kGHRDrc_n98 this video!]

== Some examples of utilization ==
* Debriefing/training of formation flight: Your aerobatic formation can review the flight, and find who broke the diamond figure in the top of the loop :) You can even fly again with the recorded multiplayer flight to improve your skills.
* Make a film by recording the planes one by one, without any Internet connection.
* Having some "multiplayer action files" available, for example to replay them in a FlightGear demo, or if you do not have internet but want some other planes than the silly AI models.
* Save a multiplayer event and do video capture after the event. This way you can make multiple video takes and do a nice editing.
* UDP traffic analysis, to see the effect of internet transmission on other multiplayer pilot's packets (loss/jitter)

== Network tools ==
* Wireshark: A network traffic sniffer/analyzer available for all platforms. We will use it to record the FlightGear UDP traffic.
* tcpreplay: A suit of programs available on Mac and Linux, and maybe on Windows by using Cygwin. It will be used to change some UDP packet properties and to send back the traffic to another FlightGear session.

== Captureing FlightGear network traffic ==
Assuming we are connected to a [[Howto:Multiplayer#Servers|multiplayer server]] and that the server and your FlightGear session receive on UDP port 5000, we can record all the UDP packets going to a port 5000.

This way we will record our outgoing packets and the ingoing packets from the other multiplayer pilots.

We start Wireshark, then select the network interface used, and finally set a filter and apply it, for example:
UDP.dstport == 5000 && !(icmp.type)

Finally, we stop the capture and save the file as a <code>.cap</code> file.

What do we have now? We have the flight recorded at the network level!

With wireshark we can now extract some pilot(s), use a certain time interval of traffic, or see certain traffic properties.

With the help of tcpreplay/tcprewrite, we can send the multiplayer traffic back to a running FlightGear session.

== Extracting files for individual pilots ==
In Wireshark it is possible to extract each pilot flight individually. Using the filter looks like:

frame contains "callsign"

and to exclude a pilot:

!frame contains "callsign"

This way you can have each pilot's flight separated. If needed you can merge again 2 or more pilots in a file, if you for example you have recorded a diamond formation and you want to practice the left wingman position. Either you remove the existing one for the <code>.cap</code> file, or you merge the 3 others again, after having done individual <code>.cap</code> files.

Wireshark come with mergecap to merge multiple <code>.cap</code> files. Taking care the chronological order is respected. The provided editcap can be used too if you need to add a time offset to a plane to address recorded latency issues.

It is sometimes difficult to have lot of matching schedules when you are training in an aerobatic team, and we have here a good way to train with other mp pilots without being connected. You can even practice by file exchange: The leader make a flight, send the <code>.cap</code> file to the second pilot which add his flight, then pass to the third pilot etc.

== Analyzing UDP packets loss/jitter ==
[[File:Wireshark mp rate analyse.png|600px|thumb|Received Multiplayer packet rate]]
Here is a wireshark analysis showing the multiplayer packet rate for different multiplayer pilots from a recorded multiplayer session.
* The black curve got a nice 10 packets/s, our internet link was good and is maybe connected to the same multiplayer server.
* The red and pink were pretty chaotic. Their packets are often lost and I guess they are jittery in FlightGear. They were probably connected to an other multiplayer server.
* The blue and green show less than 1 packet/s, this can be explained by some connection losses between servers. By default the multiplayer server send at 1 packet/s to the other multiplayer server just to "inform" them which planes they host, and start relaying all the packets only once other multiplayer server send them planes in vicinity. In this case our multiplayer server probably did not relay to their server, and we only got the "lazy relay" from their server.{{-}}

== Replay the flight to our FlightGear session ==
Ok, we have a recorded flight saved as <code>.cap</code> file and want to see it again in FlightGear. That's where the tricky parts start. :)

=== Limitations of tcpreplay ===
To make things short, tcpreplay can not send directly to FlightGear if we are on the same computer.

Until now I have used a other PC to send traffic back to my FlightGear PC, if you find an other way, tell me. :)

According to the tcpreplay FAQ, it could work if tcpreplay runs in a virtualized machine, but I did not give it a go.

=== tcprewrite magic ===
Next is that the UDP packets. To come back to our PC, we need some editing in the UDP packets:
* The MAC address of the sender must match the MAC of the PC sending the recorded flight.
* The destination MAC address must match the receiving PC, or depending the case the MAC address of the router.
* The destination and sender IP address must match the IP address of the 2 PC involved.

We make the change in one go with tcprewrite, here's a practical example. We have:
* The computer running FlightGear, MAC address 00:11:22:33:44:55, IP address 192.168.1.12 .
* The computer we will use to send the recorded packets: MAC address 99:88:77:66:55:44, IP address 192.168.1.17

It is up to you to find your MAC and IP address. You can use wireshark on some network traffic between the two, or ifconfig on Linux etc. Here is the "magic" line,
tcprewrite -C --enet-smac=99:88:77:66:55:44 --enet-dmac=00:11:22:33:44:55
-S 0.0.0.0/0:192.168.1.17 -D 0.0.0.0/0:192.168.1.12 -i recorded.cap -o readytosend.cap

where:
* -C force a checksum recalculation. If not the packets are not valid and are not sent.
* --enet-smac gives the mac address of the sender
* --enet-dmac same with the receiver
* -S changes source IP address
* -D does the same for destination
* -i input file (the previously recorded flight)
* -o output file

tcprewrite do not need to be root to be used, but using tcpreplay to see the recorded traffic again need root access to use the network interface.

Now we start FlightGear with Multiplayer enabled by sending to a server (it can be a MPserver or the address of a valid local host on your network, for example 192.168.0.17).

Finally we start replaying the flight, on the dedicaced PC:
tcpreplay -i eth0 readytosend.cap

Be sure to use the correct name for the network interface. ifconfig/ipconfig is your friend here!

Tcpreplay works fine this way if we don't need a very good time flow sync, as it end being late, dépending on the cpu usage mostly. I didn't notice this issue at first, but this became evident using lag statistics available recently in next, so to keep a good sync between tcpreplay and flightgear, using a core affinity, setting a priority higher and preferably on a not overloaded PC/CPU core makes things better, so we got something like that on linux:

taskset 02 nice -n -15 tcpreplay -x 1.000065 -K -i enp0s10 /rab/refuel/test.pcapng

The "-x" option ensure a good time sync over the replay (find yourself the best value for your config :) ), i just need to adjust the time offset for the mp clock, to have replayed planes in sync with my session, considered as "real time" buddys.

=== Give it a try ===
[[File:Fighters mont aiguille.jpg|thumb|The "mont aiguille", in the French Vercors Massif.]]
if you don't have time to make a record yourself, you can use this [http://janodesbois.free.fr/flightgear/cap_files/vercors_test.cap.bz2 flight in the French Vercors Massif] (3.4M). Start at LFLU, runway 01 with the [[UFO]] or in a plane capable of more than 400 kts, and follow the leading [[McDonnell F-4 Phantom II|F-4N]] in a Vercors Massif trip, followed by a [[English Electric Lightning|Lightning]], a [[General Dynamics F-16 Fighting Falcon|F-16]] and a [[Grumman F-14 Tomcat|F-14B]]. It is better with a detailed scenery like "La France d'el maxo".

Just decompress it, and use tcprewrite to match your config.{{-}}

== Related content ==
=== Wiki articles ===
* [[Generic protocol]] - ''A flexible way to log properties locally.''
* [[Interfacing FlightGear]] - ''Various ways to interface FlightGear.''
* [[JSBSim Logging]] - ''Logging JSBSim internal properties locally.''
* [[Logging properties]] - ''The simplest way to log properties locally.''

== External links ==
* {{cite web
| url = https://www.wireshark.org
| title = wireshark.org
}}

[[Category:Development]]
[[Category:Multiplayer]]

Real Time manual

2020-01-31T12:22:31Z

Jano: /* result and limitations */

Following the work in the [[Mp-patch]], this page is intended to help setup fg to use the real time lag compensation.
 

[[File:707-TT refuelinfg a ch53e.jpg|799px|mp session où le ch53e se ravitaille dérrière le 707-TT, qui a des tuyaux qui se déplacent au point de contact.]]
 
the ch53e, the tanker and the ufo taking the screen are all mp pilots.
 

== how it works ==

préviously, we tried to set manually a lag value, usually the mpserver ping, and use that lag correction to display the planes in the futur.

Now, we will trust the pc clocks, and use the same time to display all the mp plane, if the times they show are close enough.

The advantage is that the pilots don't need to do anything in most the case.
 

As a way to investigate network issues, there are now the lag, and pps (packet per second) values available for each mp pilots, and a dedicated page is included in pilot list, just click on the unit to get it.

== step by step ==

(will probably change :) ): first, set the master switch and apply to close mp to on, in the lag correction menu

Next is to open pilot list, and display the "lag" page (by clicking on the unit (IM => lag => SI)

- lag-pps is the number of packets received per second. if all pilots are sending at 10 (work quite well for the public mpservers) they should have 10 or quite close for this value, something equal or inferior to 1 pps show usually a different mpserver used, wich don't properly relai the traffic, letting only the "lazy relai" at 1 pps working.
Change the mpserver you're on to have a solid 10, if that still don't works, there are sometimes pilots with bad internet connexion dropping too much FG traffic.

- lag-ms This is the time in ms between our FG clock, and the time present in the packet received from the mp pilots, negative is mp pilots are late. This is basically the travel time from the mp pilot's pc to our FG, a bit pessimistic because of delay waiting to be processed at the next iteration.
From recent mp session, i get in the -50 ~ -70ms range for mp pilots lag, who have a clock synchronized to the internet time servers (probably european pilots, like me).

- If the values are nearly the same from each side, our clocks are in synchro, and all is left is to fly. Different fps lead to a bit better or worst lag display so 10 or so ms difference is not a big deal.
- For high lag values, it can be a very bad internet pipe (eg australia <=> europe) specially if both side see something in the same range (and with the same "-" sign :) )
Or it can be simply a clock not synchronised over internet.
You can add an offset to the mp clock, (the clock offset in the lag menu)
beware this value is in seconds, and the lag in ms !
If a player using ntp have 15260 lag in pilot list, using 15.320 in the offset should give -60ms lag and if the value is in the same range on his side, you should be good.

The other solution is to use ntp, wich should do the job permanently, install ntp on linux, similar on mac ?? and there's a ntp program from Meinberg for windows users (need confirmation)

== result and limitations ==

Video of a mp session, where the tanker move the drogue to the plane's refuel point:
[https://www.youtube.com/watch?v=qIRI8glSl7s mp planes refueling]

the flight is now very smooth, for the speed and altitudes used by the planes.

I tried it with the ufo at high speed and altitude, and that's not good enough, so this not to be used for a space rendez-vous :( the tip in this case is for the chasing space vehicule to tune the offset to avoid prediction, eg have a iss lag in the 150ms range (positive value).

the very obvious advantage is that if you're using ntp, nothing is needed to have a perfect sync, and the offset allow you to tune easily your clock. Note that this works well for Clocks having the same speed, we had a case with a pilot who was playing with the ntp settings a drifting lag over time, but this was the only case (Older FG version before the change in the mp clock behave like that too).

Real Time manual

2020-01-31T00:21:30Z

Jano: /* how it works */

Following the work in the [[Mp-patch]], this page is intended to help setup fg to use the real time lag compensation.
 

[[File:707-TT refuelinfg a ch53e.jpg|799px|mp session où le ch53e se ravitaille dérrière le 707-TT, qui a des tuyaux qui se déplacent au point de contact.]]
 
the ch53e, the tanker and the ufo taking the screen are all mp pilots.
 

== how it works ==

préviously, we tried to set manually a lag value, usually the mpserver ping, and use that lag correction to display the planes in the futur.

Now, we will trust the pc clocks, and use the same time to display all the mp plane, if the times they show are close enough.

The advantage is that the pilots don't need to do anything in most the case.
 

As a way to investigate network issues, there are now the lag, and pps (packet per second) values available for each mp pilots, and a dedicated page is included in pilot list, just click on the unit to get it.

== step by step ==

(will probably change :) ): first, set the master switch and apply to close mp to on, in the lag correction menu

Next is to open pilot list, and display the "lag" page (by clicking on the unit (IM => lag => SI)

- lag-pps is the number of packets received per second. if all pilots are sending at 10 (work quite well for the public mpservers) they should have 10 or quite close for this value, something equal or inferior to 1 pps show usually a different mpserver used, wich don't properly relai the traffic, letting only the "lazy relai" at 1 pps working.
Change the mpserver you're on to have a solid 10, if that still don't works, there are sometimes pilots with bad internet connexion dropping too much FG traffic.

- lag-ms This is the time in ms between our FG clock, and the time present in the packet received from the mp pilots, negative is mp pilots are late. This is basically the travel time from the mp pilot's pc to our FG, a bit pessimistic because of delay waiting to be processed at the next iteration.
From recent mp session, i get in the -50 ~ -70ms range for mp pilots lag, who have a clock synchronized to the internet time servers (probably european pilots, like me).

- If the values are nearly the same from each side, our clocks are in synchro, and all is left is to fly. Different fps lead to a bit better or worst lag display so 10 or so ms difference is not a big deal.
- For high lag values, it can be a very bad internet pipe (eg australia <=> europe) specially if both side see something in the same range (and with the same "-" sign :) )
Or it can be simply a clock not synchronised over internet.
You can add an offset to the mp clock, (the clock offset in the lag menu)
beware this value is in seconds, and the lag in ms !
If a player using ntp have 15260 lag in pilot list, using 15.320 in the offset should give -60ms lag and if the value is in the same range on his side, you should be good.

The other solution is to use ntp, wich should do the job permanently, install ntp on linux, similar on mac ?? and there's a ntp program from Meinberg for windows users (need confirmation)

== result and limitations ==

If i find a good cameraman, will do a better one than this hastily made (and speeded x1.5) video of a mp session, where the tanker move the drogue to the plane's refuel point:
[https://www.youtube.com/watch?v=nITb3y18us0 mp planes refueling]

the flight is now very smooth, for the speed and altitudes used by the planes.

I tried it with the ufo at high speed and altitude, and that's not good enough, so this not to be used for a space rendez-vous :( the tip in this case is for the chasing space vehicule to tune the offset to avoid prediction, eg have a iss lag in the 150ms range (positive value).

the very obvious advantage is that if you're using ntp, nothing is needed to have a perfect sync, and the offset allow you to tune easily your clock. Note that this works well for Clocks having the same speed, we had a case with a pilot who was playing with the ntp settings a drifting lag over time, but this was the only case (Older FG version before the change in the mp clock behave like that too).

Real Time manual

2020-01-31T00:19:26Z

Jano:

Following the work in the [[Mp-patch]], this page is intended to help setup fg to use the real time lag compensation.
 

[[File:707-TT refuelinfg a ch53e.jpg|799px|mp session où le ch53e se ravitaille dérrière le 707-TT, qui a des tuyaux qui se déplacent au point de contact.]]
 
the ch53e, the tanker and the ufo taking the screen are all mp pilots.
 

== how it works ==

préviously, we tried to guess manually a lag value, and use that lag correction to display the planes in the futur.

Now, we will trust the pc clocks, and use the same time to display all the mp plane, if the times they show are close enough.

The advantage is that the pilots don't need to do anything in most the case.
 

As a way to investigate network issues, there are now the lag, and pps (packet per second) values available for each mp pilots, and a dedicated page is included in pilot list, just click on the unit to get it.

== step by step ==

(will probably change :) ): first, set the master switch and apply to close mp to on, in the lag correction menu

Next is to open pilot list, and display the "lag" page (by clicking on the unit (IM => lag => SI)

- lag-pps is the number of packets received per second. if all pilots are sending at 10 (work quite well for the public mpservers) they should have 10 or quite close for this value, something equal or inferior to 1 pps show usually a different mpserver used, wich don't properly relai the traffic, letting only the "lazy relai" at 1 pps working.
Change the mpserver you're on to have a solid 10, if that still don't works, there are sometimes pilots with bad internet connexion dropping too much FG traffic.

- lag-ms This is the time in ms between our FG clock, and the time present in the packet received from the mp pilots, negative is mp pilots are late. This is basically the travel time from the mp pilot's pc to our FG, a bit pessimistic because of delay waiting to be processed at the next iteration.
From recent mp session, i get in the -50 ~ -70ms range for mp pilots lag, who have a clock synchronized to the internet time servers (probably european pilots, like me).

- If the values are nearly the same from each side, our clocks are in synchro, and all is left is to fly. Different fps lead to a bit better or worst lag display so 10 or so ms difference is not a big deal.
- For high lag values, it can be a very bad internet pipe (eg australia <=> europe) specially if both side see something in the same range (and with the same "-" sign :) )
Or it can be simply a clock not synchronised over internet.
You can add an offset to the mp clock, (the clock offset in the lag menu)
beware this value is in seconds, and the lag in ms !
If a player using ntp have 15260 lag in pilot list, using 15.320 in the offset should give -60ms lag and if the value is in the same range on his side, you should be good.

The other solution is to use ntp, wich should do the job permanently, install ntp on linux, similar on mac ?? and there's a ntp program from Meinberg for windows users (need confirmation)

== result and limitations ==

If i find a good cameraman, will do a better one than this hastily made (and speeded x1.5) video of a mp session, where the tanker move the drogue to the plane's refuel point:
[https://www.youtube.com/watch?v=nITb3y18us0 mp planes refueling]

the flight is now very smooth, for the speed and altitudes used by the planes.

I tried it with the ufo at high speed and altitude, and that's not good enough, so this not to be used for a space rendez-vous :( the tip in this case is for the chasing space vehicule to tune the offset to avoid prediction, eg have a iss lag in the 150ms range (positive value).

the very obvious advantage is that if you're using ntp, nothing is needed to have a perfect sync, and the offset allow you to tune easily your clock. Note that this works well for Clocks having the same speed, we had a case with a pilot who was playing with the ntp settings a drifting lag over time, but this was the only case (Older FG version before the change in the mp clock behave like that too).

Real Time manual

2020-01-31T00:18:33Z

Jano:

Following the work in the [[Mp-patch]], this page is intended to help setup fg to use the real time lag compensation, WIP;
 

[[File:707-TT refuelinfg a ch53e.jpg|799px|mp session où le ch53e se ravitaille dérrière le 707-TT, qui a des tuyaux qui se déplacent au point de contact.]]
 
the ch53e, the tanker and the ufo taking the screen are all mp pilots.
 

== how it works ==

préviously, we tried to guess manually a lag value, and use that lag correction to display the planes in the futur.

Now, we will trust the pc clocks, and use the same time to display all the mp plane, if the times they show are close enough.

The advantage is that the pilots don't need to do anything in most the case.
 

As a way to investigate network issues, there are now the lag, and pps (packet per second) values available for each mp pilots, and a dedicated page is included in pilot list, just click on the unit to get it.

== step by step ==

(will probably change :) ): first, set the master switch and apply to close mp to on, in the lag correction menu

Next is to open pilot list, and display the "lag" page (by clicking on the unit (IM => lag => SI)

- lag-pps is the number of packets received per second. if all pilots are sending at 10 (work quite well for the public mpservers) they should have 10 or quite close for this value, something equal or inferior to 1 pps show usually a different mpserver used, wich don't properly relai the traffic, letting only the "lazy relai" at 1 pps working.
Change the mpserver you're on to have a solid 10, if that still don't works, there are sometimes pilots with bad internet connexion dropping too much FG traffic.

- lag-ms This is the time in ms between our FG clock, and the time present in the packet received from the mp pilots, negative is mp pilots are late. This is basically the travel time from the mp pilot's pc to our FG, a bit pessimistic because of delay waiting to be processed at the next iteration.
From recent mp session, i get in the -50 ~ -70ms range for mp pilots lag, who have a clock synchronized to the internet time servers (probably european pilots, like me).

- If the values are nearly the same from each side, our clocks are in synchro, and all is left is to fly. Different fps lead to a bit better or worst lag display so 10 or so ms difference is not a big deal.
- For high lag values, it can be a very bad internet pipe (eg australia <=> europe) specially if both side see something in the same range (and with the same "-" sign :) )
Or it can be simply a clock not synchronised over internet.
You can add an offset to the mp clock, (the clock offset in the lag menu)
beware this value is in seconds, and the lag in ms !
If a player using ntp have 15260 lag in pilot list, using 15.320 in the offset should give -60ms lag and if the value is in the same range on his side, you should be good.

The other solution is to use ntp, wich should do the job permanently, install ntp on linux, similar on mac ?? and there's a ntp program from Meinberg for windows users (need confirmation)

== result and limitations ==

If i find a good cameraman, will do a better one than this hastily made (and speeded x1.5) video of a mp session, where the tanker move the drogue to the plane's refuel point:
[https://www.youtube.com/watch?v=nITb3y18us0 mp planes refueling]

the flight is now very smooth, for the speed and altitudes used by the planes.

I tried it with the ufo at high speed and altitude, and that's not good enough, so this not to be used for a space rendez-vous :( the tip in this case is for the chasing space vehicule to tune the offset to avoid prediction, eg have a iss lag in the 150ms range (positive value).

the very obvious advantage is that if you're using ntp, nothing is needed to have a perfect sync, and the offset allow you to tune easily your clock. Note that this works well for Clocks having the same speed, we had a case with a pilot who was playing with the ntp settings a drifting lag over time, but this was the only case (Older FG version before the change in the mp clock behave like that too).

Real Time manual

2020-01-31T00:16:14Z

Jano: /* how it works */

Following the work in the [[Mp-patch]], this page is intended to help setup fg to use the real time lag compensation, WIP;
 

[[File:707-TT refuelinfg a ch53e.jpg|799px|mp session où le ch53e se ravitaille dérrière le 707-TT, qui a des tuyaux qui se déplacent au point de contact.]]
 

== how it works ==

préviously, we tried to guess manually a lag value, and use that lag correction to display the planes in the futur.

Now, we will trust the pc clocks, and use the same time to display all the mp plane, if the times they show are close enough.

The advantage is that the pilots don't need to do anything in most the case.
 

As a way to investigate network issues, there are now the lag, and pps (packet per second) values available for each mp pilots, and a dedicated page is included in pilot list, just click on the unit to get it.

== step by step ==

(will probably change :) ): first, set the master switch and apply to close mp to on, in the lag correction menu

Next is to open pilot list, and display the "lag" page (by clicking on the unit (IM => lag => SI)

- lag-pps is the number of packets received per second. if all pilots are sending at 10 (work quite well for the public mpservers) they should have 10 or quite close for this value, something equal or inferior to 1 pps show usually a different mpserver used, wich don't properly relai the traffic, letting only the "lazy relai" at 1 pps working.
Change the mpserver you're on to have a solid 10, if that still don't works, there are sometimes pilots with bad internet connexion dropping too much FG traffic.

- lag-ms This is the time in ms between our FG clock, and the time present in the packet received from the mp pilots, negative is mp pilots are late. This is basically the travel time from the mp pilot's pc to our FG, a bit pessimistic because of delay waiting to be processed at the next iteration.
From recent mp session, i get in the -50 ~ -70ms range for mp pilots lag, who have a clock synchronized to the internet time servers (probably european pilots, like me).

- If the values are nearly the same from each side, our clocks are in synchro, and all is left is to fly. Different fps lead to a bit better or worst lag display so 10 or so ms difference is not a big deal.
- For high lag values, it can be a very bad internet pipe (eg australia <=> europe) specially if both side see something in the same range (and with the same "-" sign :) )
Or it can be simply a clock not synchronised over internet.
You can add an offset to the mp clock, (the clock offset in the lag menu)
beware this value is in seconds, and the lag in ms !
If a player using ntp have 15260 lag in pilot list, using 15.320 in the offset should give -60ms lag and if the value is in the same range on his side, you should be good.

The other solution is to use ntp, wich should do the job permanently, install ntp on linux, similar on mac ?? and there's a ntp program from Meinberg for windows users (need confirmation)

== result and limitations ==

If i find a good cameraman, will do a better one than this hastily made (and speeded x1.5) video of a mp session, where the tanker move the drogue to the plane's refuel point:
[https://www.youtube.com/watch?v=nITb3y18us0 mp planes refueling]

the flight is now very smooth, for the speed and altitudes used by the planes.

I tried it with the ufo at high speed and altitude, and that's not good enough, so this not to be used for a space rendez-vous :( the tip in this case is for the chasing space vehicule to tune the offset to avoid prediction, eg have a iss lag in the 150ms range (positive value).

the very obvious advantage is that if you're using ntp, nothing is needed to have a perfect sync, and the offset allow you to tune easily your clock. Note that this works well for Clocks having the same speed, we had a case with a pilot who was playing with the ntp settings a drifting lag over time, but this was the only case (Older FG version before the change in the mp clock behave like that too).

Real Time manual

2020-01-31T00:13:08Z

Jano:

Following the work in the [[Mp-patch]], this page is intended to help setup fg to use the real time lag compensation, WIP;
 

[[File:707-TT refuelinfg a ch53e.jpg|799px|mp session où le ch53e se ravitaille dérrière le 707-TT, qui a des tuyaux qui se déplacent au point de contact.]]
 

== how it works ==

préviously, we tried to guess manually a lag value, and use that lag correction to display the planes in the futur.

Now, we will trust the pc clocks, and use the same time to display all the mp plane, if the times they show are close enough.

The advantage is that the pilots don't need to do anything in most the case.

there are now the lag value, and pps (packet per second) available for each mp pilots, and a dedicaced page is included in pilot list.

== step by step ==

(will probably change :) ): first, set the master switch and apply to close mp to on, in the lag correction menu

Next is to open pilot list, and display the "lag" page (by clicking on the unit (IM => lag => SI)

- lag-pps is the number of packets received per second. if all pilots are sending at 10 (work quite well for the public mpservers) they should have 10 or quite close for this value, something equal or inferior to 1 pps show usually a different mpserver used, wich don't properly relai the traffic, letting only the "lazy relai" at 1 pps working.
Change the mpserver you're on to have a solid 10, if that still don't works, there are sometimes pilots with bad internet connexion dropping too much FG traffic.

- lag-ms This is the time in ms between our FG clock, and the time present in the packet received from the mp pilots, negative is mp pilots are late. This is basically the travel time from the mp pilot's pc to our FG, a bit pessimistic because of delay waiting to be processed at the next iteration.
From recent mp session, i get in the -50 ~ -70ms range for mp pilots lag, who have a clock synchronized to the internet time servers (probably european pilots, like me).

- If the values are nearly the same from each side, our clocks are in synchro, and all is left is to fly. Different fps lead to a bit better or worst lag display so 10 or so ms difference is not a big deal.
- For high lag values, it can be a very bad internet pipe (eg australia <=> europe) specially if both side see something in the same range (and with the same "-" sign :) )
Or it can be simply a clock not synchronised over internet.
You can add an offset to the mp clock, (the clock offset in the lag menu)
beware this value is in seconds, and the lag in ms !
If a player using ntp have 15260 lag in pilot list, using 15.320 in the offset should give -60ms lag and if the value is in the same range on his side, you should be good.

The other solution is to use ntp, wich should do the job permanently, install ntp on linux, similar on mac ?? and there's a ntp program from Meinberg for windows users (need confirmation)

== result and limitations ==

If i find a good cameraman, will do a better one than this hastily made (and speeded x1.5) video of a mp session, where the tanker move the drogue to the plane's refuel point:
[https://www.youtube.com/watch?v=nITb3y18us0 mp planes refueling]

the flight is now very smooth, for the speed and altitudes used by the planes.

I tried it with the ufo at high speed and altitude, and that's not good enough, so this not to be used for a space rendez-vous :( the tip in this case is for the chasing space vehicule to tune the offset to avoid prediction, eg have a iss lag in the 150ms range (positive value).

the very obvious advantage is that if you're using ntp, nothing is needed to have a perfect sync, and the offset allow you to tune easily your clock. Note that this works well for Clocks having the same speed, we had a case with a pilot who was playing with the ntp settings a drifting lag over time, but this was the only case (Older FG version before the change in the mp clock behave like that too).

Real Time manual

2020-01-31T00:11:21Z

Jano:

[Following the work in the [[Mp-patch]], this page is intended to help setup fg to use the real time lag compensation, WIP;
 

[File:707-TT refuelinfg a ch53e.jpg|mp session où le ch53e se ravitaille dérrière le 707-TT, qui a des tuyaux qui se déplacent au point de contact.]]
 

== how it works ==

préviously, we tried to guess manually a lag value, and use that lag correction to display the planes in the futur.

Now, we will trust the pc clocks, and use the same time to display all the mp plane, if the times they show are close enough.

The advantage is that the pilots don't need to do anything in most the case.

there are now the lag value, and pps (packet per second) available for each mp pilots, and a dedicaced page is included in pilot list.

== step by step ==

(will probably change :) ): first, set the master switch and apply to close mp to on, in the lag correction menu

Next is to open pilot list, and display the "lag" page (by clicking on the unit (IM => lag => SI)

- lag-pps is the number of packets received per second. if all pilots are sending at 10 (work quite well for the public mpservers) they should have 10 or quite close for this value, something equal or inferior to 1 pps show usually a different mpserver used, wich don't properly relai the traffic, letting only the "lazy relai" at 1 pps working.
Change the mpserver you're on to have a solid 10, if that still don't works, there are sometimes pilots with bad internet connexion dropping too much FG traffic.

- lag-ms This is the time in ms between our FG clock, and the time present in the packet received from the mp pilots, negative is mp pilots are late. This is basically the travel time from the mp pilot's pc to our FG, a bit pessimistic because of delay waiting to be processed at the next iteration.
From recent mp session, i get in the -50 ~ -70ms range for mp pilots lag, who have a clock synchronized to the internet time servers (probably european pilots, like me).

- If the values are nearly the same from each side, our clocks are in synchro, and all is left is to fly. Different fps lead to a bit better or worst lag display so 10 or so ms difference is not a big deal.
- For high lag values, it can be a very bad internet pipe (eg australia <=> europe) specially if both side see something in the same range (and with the same "-" sign :) )
Or it can be simply a clock not synchronised over internet.
You can add an offset to the mp clock, (the clock offset in the lag menu)
beware this value is in seconds, and the lag in ms !
If a player using ntp have 15260 lag in pilot list, using 15.320 in the offset should give -60ms lag and if the value is in the same range on his side, you should be good.

The other solution is to use ntp, wich should do the job permanently, install ntp on linux, similar on mac ?? and there's a ntp program from Meinberg for windows users (need confirmation)

== result and limitations ==

If i find a good cameraman, will do a better one than this hastily made (and speeded x1.5) video of a mp session, where the tanker move the drogue to the plane's refuel point:
[https://www.youtube.com/watch?v=nITb3y18us0 mp planes refueling]

the flight is now very smooth, for the speed and altitudes used by the planes.

I tried it with the ufo at high speed and altitude, and that's not good enough, so this not to be used for a space rendez-vous :( the tip in this case is for the chasing space vehicule to tune the offset to avoid prediction, eg have a iss lag in the 150ms range (positive value).

the very obvious advantage is that if you're using ntp, nothing is needed to have a perfect sync, and the offset allow you to tune easily your clock. Note that this works well for Clocks having the same speed, we had a case with a pilot who was playing with the ntp settings a drifting lag over time, but this was the only case (Older FG version before the change in the mp clock behave like that too).