Python in FlightGear

From FlightGear wiki
Revision as of 13:41, 31 December 2015 by Hooray (talk | contribs) (→‎Objective)
Jump to navigation Jump to search
This article is a stub. You can help the wiki by expanding it.

Background

Cquote1.png This is a follow up of a discussion that I have with one of the devs at fg weekend 2015.

My propose is that we include python 3 as an alternative for nasal. This mean not that nasal is bad, but I can writing code in python faster than nasal. (this is only my opinion and experience)

Another plus for python re can create a standalone interface that connect to fgfs or play a flight from an recode session for cockpit building or testing new code. What is your considers about Python as an nasal alternative?

Sorry this is short post, more in the future.
Cquote2.png
Cquote1.png to explore building a prototype interface to python, and I can't see any harm in doing some exploration and testing.
Cquote2.png
Cquote1.png We have someone (maybe a few people) that are interested in doing some exploration. They think there could be some interesting things to be found.
Cquote2.png
Cquote1.png I think the first hurdle towards python integration would be an interface to the property system. I have another project (a UAV autopilot system) that is built using the FlightGear properties, and I have a longer term interest in python integration there too. For the FlightGear project specifically, it would be important to proceed in a way that doesn't create headaches for the build system ... i.e. we need to be able to build on Windows, Mac, and Linux. Typically we are adverse to adding new build dependencies. We are less adverse to adding 'optional' build dependencies.
Cquote2.png
Cquote1.png there are no promises ahead of time that we will roll python integration into the core source code ... especially before seeing actual code and seeing how the integration looks, understanding what parts of the code and build system need to be touched, making sure it can be supported on all platforms, and pondering some of the bigger issues brought up in this thread. But there are a lot of python fans floating around the FlightGear community so I think you could build up some interest, especially if you were able to get some early prototype integration working and show some examples. Maybe in the process you'll stumble on a really compelling or creative use for python that nasal can't match? Maybe in the process you'll stumble on some show stopper problem and we'll learn some lessons. We do have some precident for including optionally compiled features in FlightGear, so if you can come up with a clean integration, but larger objections remain, perhaps we can make it an optionally compiled feature so those with interest can pursue it, and those with concerns can avoid it ... at least for the near term.
Cquote2.png

Status

RFC

Objective

Cquote1.png For the idea of prototyping, you could set up a builtin Python package which provides a framework to create a new FlightGear subsystem. There's no point arguing if this is doable or not, because I know that anyone who knows Python, C, and the FG internals could probably embed Python, add read+write access to the property tree, and create this subsystem framework package in an extremely short timeframe (with real dedication, a week would not be unreasonable). I could probably do this myself in this timeframe, if it wasn't for my other projects (1, 2). You can expose as much of the C++ internals as you wish with the Python-C API.

Once you have a Python interface that allows for code prototyping, then this opens up the FlightGear internals to a much larger group of people. For example uni students, as a large number already use Python for prototyping ideas. Once the idea is tested, the code can then be ported to C++. Or if the performance impact is low, the Python subsystem can be integrated into the core without modification.

Python is the perfect prototyping language for lower level languages, as it is one of the easiest to learn. Syntax-wise it is the simplest language. The prototyping potential of FG embedded Python is such an awesome feature, that it makes any arguments against it moot.
Cquote2.png

Approach

Cquote1.png The first thing is to embed the interpreter. The second is to access the property tree. The third might be to wait for a HLA compatible solution for the property tree to be added to FG, then hooking the Python interpreter into that. I also don't understand the HLA push here, because HLA is a slow technology (just like most multi-processor solutions). I have extensive experience with clustering, and I am well aware of the disadvantages of HLA. However certain built-in modules could provide HLA access to certain functionality. This can all be done down the line - planning for it now is really too much. Finally, this would all have to be done in C. From the main program, you interact with embedded Python only using C code, via the embedded Python C interface. Therefore the person implementing this needs to either know C, or is willing to learn C.
Cquote2.png

Motivation

Cquote1.png Python support would be a great feature I think. It is used in many products (e.g. in Blender) and that is a plus. However it's not guaranteed that it is free of such problems until you port all current Nasal scripts to Python and test it. What I am thinking about is a possibility to convert Nasal scripts to C or C++ and compile them as shared objects (.so .dll). Then we could load them dynamically at fgfs runtime. So in the end we get raw C/C++ performance to our modules. Is this possible or am I dreaming of something impossible here? What do you think?
— Robert (Mar 26th, 2011). Re: [Flightgear-devel] Stuttering at 1 Hz rate.
(powered by Instant-Cquotes)
Cquote2.png

Open Issues

HLA

Cquote1.png there is a sketch of an AI module working on top of hla objects, this sketch also shows how you can roll on ground - at least for solid ground. Also fgviewer gained a lot of unfinished hla code that enables fgviewer to sit on a hla configured view - more or less. At least a python script that I regularily use for development providing this works fine.
— Mathias Fröhlich (Jan 20th, 2013). Re: [Flightgear-devel] 2.10.0 Changelog - help required.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png I am for a long time running here also a brief python script which does about the same than fgai. While a scripting language is not fast enough to drive a lot of ai traffic, it is rather nice to do only a few such objects without much worries.
— Mathias Fröhlich (Dec 24th, 2012). Re: [Flightgear-devel] OpenRTI / HLA.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png I now have FlightGear reading other aircraft in the RTI Federation (RTI speak for a simulation environment) and publishing it's own position information for other clients (in RTI known as Federates) to read. There's a screenshot showing it all here: http://www.nanjika.co.uk/flightgear/hla1.png There are a number of different components in play here: 1) A python script Federate which represents the KC-135 tanker. It publishes position updates a bit like we have for MP aircraft. 2) FlightGear itself as a Federate. It too is publishing position updates, but is also subscribing to updates and displaying them if appropriate. In this case it is displaying the KC-135 Federate. 3) A python script simulating a radar screen. This is receiving updates from both the KC-135 Federate and the FlightGear Federate and displaying them. The python scripts and all the clever stuff are courtesy of Mathias. The actual flightgear code changes amount to only 1.2kloc so far.
— Stuart Buchanan (Dec 14th, 2015). [Flightgear-devel] HLA Update.
(powered by Instant-Cquotes)
Cquote2.png

Footprint

Cquote1.png And you don't have to buy in the Lua way, there is no such way really, unlike Python and other scripting languages where the proper way to embed is embedding your app in the language and not embedding the language in your app. Lua embeds itself, like you want it to :) It's also brilliant, smaller (runs on cellphones) and faster than nasal (that's an opinion, but I really can't see how anyone says Nasal is fast, at least from my experience so far)
Cquote2.png

Performance

Cquote1.png As far as speed goes, the last time I was doing any benchmarking Nasal was about as fast as Perl 5 or Python 2.2 at most things. It's garbage collector was faster, its symbol lookup about the same or slightly faster, and its bytecode interpreter somewhat slower.
Cquote2.png

Dependency Hell?

Cquote1.png a C API that allows loading of C/C++ modules through Lua (similar to python dlls) : that's a free, runtime usable plug-in system.
Cquote2.png
Cquote1.png Having the training session as a separate network app instead of being integrated into FG is going to put off a lot of less technically minded people. It requires that the user first install Python + pyao + pyogg + pyvorvis. Then they have to install the training packages and try to start FG together with the simulator. I'd much rather code it in Nasal have it part of the FG package itself or as an addon that can be unipped into the FG tree and run as is.
— Paul Surgeon (Jan 5th, 2006). Re: [Flightgear-devel] Lessons in FlightGear.
(powered by Instant-Cquotes)
Cquote2.png

Garbage Collection

Cquote1.png the issue with scanning for unreachable objects is that it scales with the number of data elements in memory. Even when you aren't touching the data structures, they still affect performance. Reference counting itself doesn't guarantee real-time behavior, since removing any reference may result in deleting large data structures. But it's much more predictable and can be influence in the code. When certain code doesn't touch anything large, then it isn't affected. There are real-time systems which have successfully integrated a Python engine because of this. Though it doesn't guarantee hard real-time "by the book", it still works very, very well, and only require few limitations (i.e. you have to avoid cyclic data structures, or accept that they are leaking).
— ThorstenB (Mar 26th, 2011). Re: [Flightgear-devel] Stuttering at 1 Hz rate.
(powered by Instant-Cquotes)
Cquote2.png
Cquote1.png Does anybody of you know how garbage collectors of Java and Python and other GPL'd script engines compare to each other? (atomic step character)
Cquote2.png
Cquote1.png Reference counting doesn't provide a strong performance advantage over conventional garbage collection, and a reference-counting scheme can take an unbounded amount of time. Reference counting schemes that do have real-time or bounded behavior are very complicated. I don't know much about our Nasal implementation, but I suspect that the garbage collector could be changed to trace only a portion of Nasal's heap at each invocation, at the risk of increased memory use.
— Tim Moore (Mar 26th, 2011). Re: [Flightgear-devel] Stuttering at 1 Hz rate.
(powered by Instant-Cquotes)
Cquote2.png

Replacing Nasal ?

Cquote1.png But changing current scripts would be a huge, huge project. And it'd would require much more maintenance than our tiny Nasal engine. It's a better option to improve the existing garbage collector (i.e. use reference counting, improve its speed, or make it happen less often). But that would also be a very complex change in a very sensitive area of our sources.
— ThorstenB (Mar 26th, 2011). Re: [Flightgear-devel] Stuttering at 1 Hz rate.
(powered by Instant-Cquotes)
Cquote2.png

Converting Scripts

Cquote1.png Such things may look simple at first. Easy to convert a simple "hello world". But it's very complex when supporting all the features of an interpreted script language. And the funny thing is: you'd still need to worry about automatic garbage collection and count references (though that'd be a lesser issue compared to many others then). So, time wake up...
— ThorstenB (Mar 26th, 2011). Re: [Flightgear-devel] Stuttering at 1 Hz rate.
(powered by Instant-Cquotes)
Cquote2.png