Howto:Optimizing FlightGear for mobile devices

From FlightGear wiki
Jump to navigation Jump to search
Portability & Compatibility
Performance
Efforts

Background

Also see: Flightgear On Android.

Cquote1.png There are a few users wanting to run FlightGear on mobile devices or gaming consoles - others want to use it for UAV stuff, and even others want to run it in "headless" mode, without displaying any graphics at all (certainly not Rembrandt/ALS or other effect/shader stuff) - all of those are valid use-cases, as is running FlightGear on an outdated OS, including even on 10-year old hardware with a nvidia 7600 GPU - ideally, FlightGear should "just work", using what's available - analogous to installing a modern Linux 3.0 distro on a 586 PC with 133 mhz and 32 mb of RAM - give it a try, it actually works reasonably well
— Hooray (Wed Apr 01). Re: Cloning fgdata with GIT submodules.
(powered by Instant-Cquotes)
 Cquote2.png

Increasingly, people want to try running FlightGear on mobile devices, like the Apple IPhone/IPad, Android phones or Nokia Maemo devices. FlightGear isn't currently optimized for such devices, so that it would require a fair amount of customizations to make this happen, not just optimization-wise, but also related to the user-interface, i.e. lack of keyboard/mouse/yoke and screen space.

Status (09/2013)

Cquote1.png far more useful would be to get ARM working so we can run parts of the stack on Pis, Pandaboards and so on. This would be materially useful for various add-on functions, especially the canvas and fgcom. (I spend an increasing amount of my work time on OpenGL on ARM platforms, they have plenty of power to run graphics, depending on which GPU is on the SoC)[1]
— James Turner
 Cquote2.png
Cquote1.png I'm currently busy on other areas but it will be either Raspbian or a custom buildroot Linux deployment. Although, since my current buildroot uses ulibc, that would mean discovering all the places in SG+FG where we've assumed glibc. Note we can't run FG itself without major work, since these platforms only support GLES1 or GLES2, and there's many legacy areas of the renderer which would not work. (BTW I believe OSG compiles out of the box on ARM targets now, Android at least, but I didn't try it myself yet)[2]
— James Turner
 Cquote2.png
  1. ↑ James Turner (Sun, 08 Sep 2013 22:30:54 -0700). Re: [Flightgear-devel] portability of simgear.
  2. ↑ James Turner (Thu, 12 Sep 2013 23:14:47 -0700). Re: [Flightgear-devel] portability of simgear.

Motivation

This an interesting concept. Such a version would have the potential to greatly expand the ability to use FG, and increase public awareness of the project. However, it would would require a well coordinated project with people who really understand both flightgear and mobile devices.

Thats a good point about the publicity that an app would give FG, however about the graphics, I don't know what specs an iPhone has butit is obviously very limited, so maybe the graphics can't be improved, or any better than the XPlane app.

Problem

Basically, it doesn't really matter if you want to run FG on an IPhone, Android device or on Maemo - there are certain shared challenges and issues for each target. In fact, FlightGear isn't currently too well suited for less powerful systems, like netbooks or old computers.

Currently flightgear development results in software oriented towards high performance desktops/graphics cards and increased simulation accuracy. Given the limited resources of the project, I think the bigger issue would be finding people interesting in working on such a 'mobile flightgear', or iphone specific one.

I would be interested in running such a mobile version on my laptop as well, as the regular versions system requirements leave me unable to run it. There is a very large gap in hardware between fg 1 and the iphone though, and I think even very old late 90s versions would be too high.

They key to orchestrating a version of FG for mobile devices would be finding a technically skilled person with a interest in the concept, who could evaluate how a mobile versions could be written and what direction to take.

Some educated guesses about would be required would be a stripped down FDM, and limited terrain data. With the original FG, the there was NASA laRCsim for the FDM, and OpenGL. However, FG is written in C/C++.

Also, the system requirement of mobile devices are very low compared to PC and also very narrow. Someone with more technical expertise would have to evaluate how feasible a mobile fg would be.

Issues

  • For instance, maybe for the interface have the 3d cockpit buttons be clickable with a touch screen directly.
  • The idea is good - but immensely surrealistic. It's a phone, not a PC. And I don't even want to imagine the interface (keyboard and mouse wise) such a version of FG would need.
  • Wouldn't that be waaaaaaay complicated for the controls though? flaps, magnetos, etc. Unless you're putting out a simplified version.
  • Flight simulators are kinda hard to play with small screens. I currently play Leo's flight simulator on my pocketPC and it works pretty well, some of the controls are a bit hard to manage, but it's still quite the feat they accomplished.
  • if you can delete the extra goodies such as HIGH detailed models, HIGH detailed textures and things like that it would cut those high specs down i am trying to make an iphone app so im not sure if ill pursue this as my friend is a developer for the iphone and from his personal experience porting 3d into iphone and ipod touch and or ipad is a helluva job and requires ALOT of patience

Linux-based Targets (Android, Maemo etc)

Now, Android/Maemo support should be easier to implement than Playstation, Nintendo Wii, IPhone and all the other platforms that people keep asking about - simply, because Android/Maemo is basically a downstripped Linux version, so compiling/building FG would certainly be possible. However, running would still be "interesting", there are MANY potential issues here, even on a "full" Linux-System like Android: http://stackoverflow.com/questions/653453/how-would-i-go-about-making-a-flight-gear-port-for-wiibrew

Cross-compiling FlightGear for Android devices is not that problematic. However, running FlightGear on an Android device, requires a fair share of customizations, i.e. a downstripped base package, and custom aircraft modifications (key bindings, mouse vs. touch screen support etc). Also, many FlightGear defaults would be unnecessarily high and would not work "as is", these could be reduced (scenery complexity, shaders, rendering settings).

OpenGL vs. OpenGL ES

If FG could be made to run with opengl ES (supported by OSG), it could potentially run on mobile devices that support it and have high enough resources, so long as the rest of the FDM, terrain, and models would work too. However, how to compile the code would depend on the OS, and it seems the iphone uses a cut-down embedded version of OS X. Regular FG can work with regular OS X, so it seems plausible that it could be compiled for the modified OSX (maybe the iphone sdk would help here).

I have hope this is possible if we can find somone with more expertise in programming. I mean, we know FG can run on OS X, and that it needs Open GL. The iphone can use OpenGl ES and seems to use a cut down OS X. If the differences between them can be over come, then it would be a matter of getting FG system requirements low enough. There is enough hard drive space for even the full version, so its a matter of running it (if the software issues can be overcome). 128 mb memory and the processor seems low, but rendering hvga or qvga is less demanding too.

Also see: http://forum.flightgear.org/viewtopic.php?f=71&t=21964

Embedded Platforms

Anybody interested in this could just as well start out by optimizing FlightGear for smaller devices, like netbooks, or older computers. That work would also be useful for any other/future mobile/embedded target platforms and wouldn't be specific to just a certain target.

Keep in mind that the processing power of modern mobile devices and many other "phones" are more advanced then PCs of some time ago, and the platform is not as different as it once was. The current generation Iphone supports Open GL ES and runs a stripped down/modified OS X- it is foreseeable (or at least plausible) that a future phone with greater hardware specifications could run fg 1.0.

These days, a mobile platform may have multi-core CPUs running at 1-2 ghz with 512+ MB of RAM and hardware-accelerated rendering using OpenGL/OpenGL ES. These platforms are probably at least as powerful as PCs were a decade ago.

Embedded technology has come a lot further. But still, this is a 3D game which even runs with limited fps (80 without 3D clouds) on my rather modern PC (quadcore 2.5GHz, Radeon HD 3650, 4GB ram). The iPhone doesn't nearly have these specifications - especially the video ram is going to be a *big* problem. Maybe in a few years this concept will become interesting, but for now the hardware won't be able to keep up. And like I said earlier, I don't even want to imagine the kind of tiny micro movements I'd need to turn left without crashing. The current generation of phones won't be able to do this - open this topic again in a year or two.

My earlier thoughts about mobile FG were if its hardware demands could be radically reduced by the lower resolution, lower textures, lower vector on terrain, and perhaps simpler FDM it was plausible- in essence taking it back to lower hardware requirements of past versions (less video memory).

Base Package

There are lots of useful customizations that can be done at the base package-level, which won't require any coding experience at all. If you are familiar with developing FlightGear resources (textures, aircraft, scenery) or just editing XML files, you can get involved in helping with this part of the effort.

GUI

However, the challenges remain - you could just as well try running FG on a netbook with Intel GMA graphics, while it works, there are certain challenges: my IPhone has about 1/5 of the screen estate that my netbook has ...

Then, a custom FG package for Android could probably just as well need its own custom set of GUI dialogs, because most of the default dialogs are not intended to work on small screens - just try running FG with low settings, i.e. 800x600 or even 640x480, and you'll see for yourself that there are many issues here. Many of these issues also affect other devices, like netbooks for example - so it would be great to come up with a solution here.

Until FlightGear 2.8, the GUI is PUI-based and thus cannot be scaled dynamically. However, the new upcoming Canvas Widgets based GUI will use a vector-driven approach, so that the GUI can be fully scaled as required.

Textures

Cquote1.png The original structure/design was in support of video cards that could

only do 256x256 max texture sizes (or video cards with limited memory that
couldn't handle a complete set of textures with higher resolutions.) We
are easily a decade beyond those days, so if this causes anyone a problem,
maybe they are trying to run flightgear on a phone?


Cquote2.png

Scenery

To learn more about creating a customized and downstripped materials.xml file, see [2] This is a link to the FlightGear forum..

It might even be an option to come up with a custom scenery build for Android devices, specifically aimed at low resource usage - i.e. just the KSFO region, and some basic aircraft, which can be easily adapted for non-PC devices, such as the ufo.

Yes, it's worth pointing out that even the X-Plane "IPhone App" is a significantly customized and down-stripped version of the original program and of the original data sets (scenery and aircraft-wise). While it would be trivial to come up with custom GUI dialogs and simple aircraft like the ufo, creating custom scenery tiles using TerraGear is a little more involved.

As memory space is limited on an Android, perhaps there should be a basic landform with a detuned visual scenery just showing mountain ranges, savanah, forest and desert, basicaly like a semi 3d world map. then downloads of airports for continents ie Africa divided by three with ten airports per sector. The reason I suggest this is because lets face it most people like to fly in specific areas ie Africa or North America etc. Lets face it are we flying or looking at scenery? the average android screen, while a lot bigger then a cell phone is still not a desktop. Personaly I only want to fly and scenery is secondary, besides having some auto generated mountains and hills around would suffice for low and slow, just suggestions mind you, probably impractical as I dont know enough about the program and I am just talking off the top of my head. At the end of the day this idea will send Flightgear where FS never really went

In fact the perfect scenery for your phone (and FG in general) is NO SCENERY, i.e. OCEAN TILES You could reduce the CPU/GPU load by customizing the scenery tile, using custom resolutions and textures.

XML/Property Tree level optimizations

Just so that you know: I just finished building and running FG 2.8 on an old (5 years) computer, and the default settings left FG basically unusable (even when using just the ufo), the largest impact had disabling the AI traffic/model system (which was causing severe stutter, which also showed up in the system monitor) and ALL the shader support. Starting FlightGear in a no-scenery location gives me about 350 fps in 800x600, starting at KSFO is well above 60 fps (frame throttling disabled), usually between 50-90 fps and frame spacing between 35-45 ms.

So, I'd suggest to customize your FG version accordingly and keep these settings disabled for starters. There are probably a number of other opportunities to customize FG without touching the C++ code (even though optimizing the C++ code would definitely seem like a good idea in my opinion). Custom texture packs and custom-built scenery tiles were already mentioned. Making use of frame throttling and adjusting the FDM update interval (model-hz) would also seem to make sense.


I didn't check the specs of the Android phones, but I'd also make sure to set up the threading mode properly.

In other words, even if you don't have access to an Android phone, there are certain things that can be done, and which would help (using an Android emulator via VirtualBox would be another option). Just customizing FG for limited targets like netbooks would also help, and if you have an old computer with an nvidia 6x/7x generation card, you could just as well try running FG.

Minimal Startup Profile

Basically, the point of the following settings is to disable EVERYTHING that could have an effect on performance, to ensure that we get a working FlightGear window up and running with no eye candy at all, and maximum frame rate.

Once that is working, features can be re-enabled step by step. Features should only be re-enabled after evaluating their impact on performance, using the frame rate counter, frame spacing indicator, the built-in system monitor (can be inspected via telnet/http using --telnet5900) and the OSG on-screen stats.

Depending on the hardware specs of the target platform, adjusting the threading mode of FG/OSG may also help: Howto:Activate multi core and multi GPU support.


  • --airport=ksfo
  • --offset-distance=4000
  • --offset-azimuth=90
  • --altitude=500
  • --heading=0
  • --model-hz=60
  • --bpp=16
  • --enable-wireframe
  • --disable-random-objects
  • --prop:/sim/rendering/quality-level=0
  • --prop:/sim/rendering/shaders/quality-level=0
  • --disable-ai-traffic
  • --prop:/sim/ai/enabled=0
  • --aircraft=ufo
  • --disable-sound
  • --prop:/sim/rendering/random-vegetation=0
  • --prop:/sim/rendering/random-buildings=0
  • --disable-specular-highlight
  • --disable-ai-models
  • --disable-clouds
  • --disable-clouds3d
  • --disable-skyblend
  • --disable-textures
  • --fog-fastest
  • --visibility=5000
  • --disable-distance-attenuation
  • --disable-enhanced-lighting
  • --disable-real-weather-fetch

Using these settings, I am getting frame rates between 400-500 fps and a steady frame spacing (latency) of 3-11 ms, on a notebook from 2007 (1024x768). Disabling wireframe mode lowers the framerate by about 50-80 fps.

FlightGear bare bones

Additional resources can be freed by flying in areas without scenery (ocean tiles), and by using simple aircraft like the ufo. And by using a HUD instead of a 2D/3D cockpit panel. In addition, custom texture packs may help to reduce the memory footprint. Further memory savings can be achieved by reducing the size of the NavDB in $FG_ROOT

Also, it helps not to use subsystems like instruments and/or the autopilot system, by not loading the via the aircraft-set.xml

So, custom scenery/aircraft packs, with customized textures or rebuilding scenery via TerraGear may help.

Disabling Nasal modules

Nasal sub modules like local weather can be "disabled" by removing the sub folders in $FG_ROOT/Nasal. Individual Nasal modules in $FG_ROOT/Nasal can be disabled by changing the *.nas file extension.

Customizing and Disabling Effects/Shaders

Currently, all shaders are disabled by default. However, shaders may actually help improve performance, i.e. using the stencil test to discard geometry that's invisible or by using a dynamic LOD algorithm: http://rastergrid.com/blog/2010/10/gpu-based-dynamic-geometry-lod/

Overall, some shaders could probably be ported to OpenGL ES GLSL, while others should remain disabled. Some new shaders with optimizations may help reduce the runtime footprint even further.


Effects and shaders could be used to simplify and optimize the FG scenery, such that there's less complexity, and simpler textures used - to reduce the total runtime footprint of FG. The idea is to use really simple graphics, like in the early 90s of flight simulators - so that FG can also be run on old hardware, such as for example shown here.

Fred said: "You may want to use the same shader for all materials and proceed like Thorsten did for the procedural shading. Then you put the complexity you want in that shader. The geometry is here to stay anyhow."

Optimizing 3D Models and Textures

Please see [3] for reference on how to optimize 3D models.

Optimizing at the Source Code Level

Preparing FlightGear to be run on mobile platforms will require some heavy customizations, not just at the XML/property tree level, but also the source code level.

That being said, it would make sense for people to team up with other FlightGear users who are interested in running FlightGear on such devices (mobile phones, netbooks, game consoles), if we manage to cooperate, we could actually make this work, regardless of the target platform - simply because these customizations will be useful for all related efforts.

First of all, this is about reducing the FlightGear resource footprint (CPU, GPU, RAM, disk space), but also targeting FlightGear for devices which may not have a real screen, or keyboard/mouse support will be necessary.

Obviously, you will need to be able to build FG from source, probably using a cross compiler. Ideally, you would be familiar with using Virtualization (e.g via VirtualBox or VMWare) or emulators, so that you can actually run the binaries yourself. Obviously, having access to real hardware would be just as good.

So if you know C++, and especially OpenGL/OSG, you could definitely help optimize FlightGear such that it's less resource hungry. As was said in the Android thread, there are a number of resource-hungry subsystems that could be simply disabled individually, such as the AI traffic system, or shader support.

Being able to disable subsystems individually will be useful for the whole simulator, not just for this effort: FlightGear Run Levels.

There are many other subsystems which are started by default and which cannot be currently disabled, so more CPU/RAM resources can be saved by editing $FG_SRC/src/Main/fg_init.cxx#l1043 and by making more subsystems optional there, using a property to decide if the subsystem shall be started or not, e.g.: 

std::string target_mode = fgGetString("/startup/target", "default");
if (target_mode == "default") {
 // default code
}
else if (target_mode =="android") {
 // android-specific code
}
else if (target_mode =="maemo") {
 // maemo-specific code
}

To test the non-default code path, just start up with something like --prop:/startup/target=android

This is really just intended to get us started. Ideally, subsystem-specific switches would be added, so that all subsystems can be individually disabled or customized, analogous to how other subsystems like the AI traffic system can already be disabled purely from the command line using a --prop:/foo/feature=true/false switch.

Some possible candidates include:

To check what else is initialized, see the "globals" files in $FG_SRC/Main:

While disabling these systems can be fairly easily accomplished using ifdefs, care must be taken that FlightGear still builds properly and that no other systems are trying to access disabled subsystems at runtime, so this will involve repeated edit/build/debugging sessions using gdb. But that doesn't need to be done on the target device, it could just as well be completed on a conventional PC.

To see how this can be done using a class template and listeners, please have a look at FlightGear Run Levels

Potential pitfalls are also discussed at FGCanvas#Open Issues.

Tracking Memory Utilization at the Subsystem-level

In order to help reduce the memory footprint of FlightGear, it might be beneficial to extend the built-in performance monitor system such that it also supports tracking memory consumption per subsystem.

The performance monitor is initialized in fg_init.cxx: https://sourceforge.net/p/flightgear/flightgear/ci/next/tree/src/Main/fg_init.cxx#l1083 The performance monitor is implemented as part of SimGear, <simgear/structure/SGPerfMon.hxx> As can be seen, it hooks into the SGSubsystem code, so that it gets invoked by all subsystems in order to create runtime samples.

The commits implementing the performance monitor are these:

The performance monitor dialog reads all data from the property tree and it is created dynamically using a Nasal module in $FG_ROOT/Nasal/performance_monitor/monitor.nas

It would probably suffice to use "placement new" (i.e. pool memory) here, and overload the new operator at the SGSubsystem level (i.e. in SimGear), so that all subsystems use the overloaded new operator automatically and write allocation reports to the performance monitor.

This would allow us to keep track of the number of memory allocations per subsystem, and also the amount of total RAM allocated by single subsystems. In addition, we could also track freeing via delete, too. And publish all the info to the property tree.

We could even provide a facility to register our own new/malloc operator for SimGear code, so that also non-FG/SG (subsystem) code, such as the Nasal GC, could be tracked memory-wise.

Once the performance monitor is extended like that, it should become obvious how much memory is used by each subsystem, so that it will be easier to do further optimizations, i.e. possibly using a lazy allocation scheme or by optimizing the underlying data structures.

Related

Retrieved from "https://wiki.flightgear.org/w/index.php?title=Howto:Optimizing_FlightGear_for_mobile_devices&oldid=118815"