High-Level Architecture
 Screenshot showing HLA prototype at LOWI | |
| Started in | 05/2009 |
|---|---|
| Description | Implementing support for the High Level Architecture to modularize FlightGear |
| Contributor(s) | Mathias Fröhlich[1][2], James Turner[3], Stuart Buchanan[4], Richard Harrison [5] |
| Status | stalled (active 2009-2016) |
| Changelog | |
| Multicore |
|---|
|
Configuration |
|
Ongoing Efforts |
|
Proposals & RFCs |
|
Background |
|
For developers |
High-Level Architecture (HLA) is a general purpose architecture for distributed computer simulation systems.
Rather than have the entire simulation within a single executable, the simulation is split into different Federates, which interact with each other by a Run-Time Infrastructure (RTI, a message bus that handles serialization of messages, events and objects), with federates typically running in their own threads/processes and each federate process having access to the full virtual process address space provided by the OS instead of having to share it with other subsystems (i.e. 32 bit platforms may make better use of virtual RAM that way).
There are three big advantages to this over a monolithic simulation (e.g. FlightGear V3.6):
- It provides a robust environment to make the simulator multi-threaded, taking advantage of computers with multiple cores, or indeed running different parts of the simulation on different computers (including even different platforms and operating systems).
- It allows us to split out parts of the simulator such as AI (by Decoupling the AI Traffic System), the FDM, Nasal scripting [1] and Renderer from each other and less time-critical sub-systems such as weather so that we can get consistent (and perhaps higher) frame-rates (i.e. reduced Nasal GC impact on frame rate).
- It provides a very good framework to allow anyone to create components that interact with FlightGear using programming languages other than C/C++ (think Ada, Java, Python etc), which may be running in their own threads, and reside in separate binaries[2], which will be also easier to debug/troubleshoot (think regression testing, i.e. running a self-contained subsystem in a dedicated gdb/valgrind session), without having to know how to modify/patch and rebuild FlightGear.
A good overview of how the HLA architecture works can be found here http://www.pitchtechnologies.com/wp-content/uploads/2014/04/TheHLAtutorial.pdf
Design
The planned overall design is as follows:
- We intend to use OpenRTI as the underlying RTI. This is an open source IEEE 1516 standard RTI, written by Mathias Fröhlich and available from https://sourceforge.net/projects/openrti/.
- Mathias has also written an open source toolkit to act as an interface library, with a working name of SimKit. This sits on top of the RTI and simplifies interfacing with the RTI massively. It also has excellent Python integration, making it very easy to write scripted Federates. It's available from: https://sourceforge.net/projects/sim-kit/
Using SimKit, the integration point is within the FlightGear code (rather than SimGear), in particular the code under $FG_SRC/Network/HLA.
Federate Object Model
A key part of the design is writing the Federation Object Model (FOM), which defines the objects and updates that are published by the RTI. While it might at first glance seem a good idea to use the FOM to share the internal property tree across multiple federates, this is probably the wrong way to use HLA as the granularity is too low [3][4][5] and it's likely to lead to synchonization issues. Instead, we'll need to make explicit decisions about the data models to communicate.
The FOM is a set of XML files in fgdata/HLA/
The orientation data in the FOM (SGOrientationWGS84) are Quaternions (https://en.wikipedia.org/wiki/Quaternion), and those are the i,j,k values. From memory, the orientation is relative to a earth-centered-frame, and there are functions in SimGear to convert to heading/pitch/roll in combination with the object's position. [6] [7]
Richard's Emesary system would also probably work quite well with the HLA FOM to give us a way for models to communicate with other models.[8]
References
|
Federates
Federates can be written in most languages, but SimKit provides very good hooks for writing them in C++ and Python.
For some Subsystems split off from the existing FlightGear source, it's fairly easy to create an executable with its own property tree[1] and have shared C++ code to map FOM objects to property values. However, this is an implementation detail - the whole point of HLA and the FOM is that it makes no assumptions about what Federates do with the data.
Regarding weather, in a HLA/RTI context the way to do this would be to have a weather engine as an RTI Federate. This would run any required weather simulation, and pass (very) local weather conditions to each of the aircraft/tower/windsock in the simulation, plus publish position information on clouds for use by visualiation engines.[2]
Current Status
Last updated: (10/2022)
Stuart did some prototyping work to federate FlightGear within HLA a number of years ago, using OpenRTI and SimKit. There should be some reference in the mailing list archive if you care to look. It is certainly possible, and one use case was to decouple the IG (image generator) and enable work to start using VSG (vulcan scene graph) while continuing to support OSG.
FlightGear is used regularly as a decoupled Image Generator (IG) for both academic and industry projects, through the many flexible I/O options available. They just aren't publicised much.
There has been some work on how we might move to VulkanSceneGraph, but last I heard there were some technical problems we couldn't overcome. [3]
Stuart is cautiously optimistic that he may be able to provide the start of a HLA implementation quite soon, as one of the blocking factors may be resolved soon.[4]
Currently, there is some very old HLA support in SimGear. This is very out of date and should be ignored.
Stuart has HLA support using the latest OpenRTI and SimKit working on a local build, but is waiting for Mathias to officially publish his SimKit before he pushes his changes. So all of the following is local-only, but included here for information.
The Flightgear core currently supports HLA as follows.
- SimKit integration, reading the SimKit FOM and connecting to an OpenRTI RTI.
- Instantiating MP AI objects so users can view objects published over the RTI by other Federates. This is currently somewhat unsatisfactory as it overloads the MP code, where really these objects are more basic.
- Pick up Environment Objects for this instance and use them to set the local METAR string for weather generation.
We currently have the following other Federates:
- fgogel - An AI model written in python the publishes over to the RTI. Part of SimKit, but handy nevertheless!
- fgtraffic - to run an AI Scenario externally to the FG Core
- fgmetar - written in python that retrieves the closest METAR station for other published Federates and publishes the METAR for them to pick up. This could be expanded to provide a general Weather Engine.
Stuart now has the AI Manager working as an external binary, and publishing updates over the RTI which are displayed as MP Aircraft. Stuart has created a little python script that looks for objects with locations being published over the RTI and publishes the closest METAR to them, then modified FG to pick up the appropriate METAR string for this instance and use it as the real-time input for weather generation.
The python script could be fairly easily enhanced into a more substantial weather engine allowing a single coherent weather environment to be used across the simulation. Stuart also started work modifying FGViewer so that it works with SimKit. This is a fairly big deal and quite a bit of fairly dull plumbing work but would allow us at least have a view of the simulation entirely separate from the running simulation. [5]
Separately, Erik has been doing some preparatory work suitable for supporting HLA in JSBSim by adopting so called PropertyObjects to hopefully get rid of tied properties (http://sourceforge.net/p/jsbsim/mailman/message/34720784/)
- ↑ https://sourceforge.net/p/flightgear/mailman/message/34632142/
- ↑ Stuart Buchanan (Jan 10th, 2017). Re: [Flightgear-devel] Traffic 2020: Towards a new Development Model for FlightGear AI Traffic .
- ↑ https://sourceforge.net/p/flightgear/mailman/message/37726603/
- ↑ Stuart Buchanan (Jan 10th, 2017). Re: [Flightgear-devel] Traffic 2020: Towards a new Development Model for FlightGear AI Traffic .
- ↑ Stuart Buchanan (Feb 23rd, 2016). [Flightgear-devel] HLA Status Update 23/2/2016.
Related
For additional information, please see:
- Technical_Reports#A_New_Architecture_for_FlightGear_Flight_Simulator
- Decoupling the AI Traffic System (currently work-in progress, highlights the limitations of the current AI system related to distributed/multi-instance and multiplayer usage)
- FlightGear HLA support (High Level Architecture) (outdated, documenting the original HLA infrastructure code)
- FlightGear CIGI Support (Common Image Generator Interface).
- Modularizing, parallelizing and distributing FlightGear (background info/discussions pre-dating HLA adoption in FlightGear)
- Distributed Interactive Simulation (DIS specific review of the FlightGear multiplayer system/protocol and its limitations, which can be addressed via HLA)
External links
- High-level architecture (simulation)
on Wikipedia