Howto:Troubleshooting Aircraft Performance Issues
| This article is a stub. You can help the wiki by expanding it. |
| Note In its current form, this section/article is largely based on quotes collected from various related discussions/channels (devel list, forum etc) using the Instant-Cquotes script. Wiki users and other contributors are encouraged to help rewrite/edit contents accordingly to help get rid of unnecessary quoting (i.e. while the wiki is not intended to be a collection of quotes, quotes are sometimes the best/easiest way to bootstrap new articles, while also providing a good way to link back to related discussions in the archives).
While rewriting usually only entails changing first person speech to 3rd person. However, please try to retain references/links to the original discussion whenever possible. |
| Troubleshooting |
|---|
Motivation
We are seeing more and more complex aircraft/features developed by aircraft developers, with non-obvious impact on overall FlightGear performance. Many aircraft developers don't fully understand how to draw correct conclusions from interpreting their findings, i.e. if performance issues are related to the FDM, texturing/3D model complexity or other aspects like systems simulation (e.g. Nasal).
|
Someone who uses 4096x4096 texture sheets for aircraft usually knows pretty darn well that they cost lots of resources. He just thinks that that's how he wants to spend resources. |
|
|
|
most aircraft developers who are involved in 3D modeling and texturing, and not necessarily coders, i.e. they may not necessarily understand the impact on frame rate and frame spacing certain actions have - I have seen too many screenshots of highly detailed cockpits posted by aircraft developers, who seem to be also just getting 10-15 fps - so they apparently don't know how to solve the problem. Having a list of expensive resources (3D models, texture, scripts) would be useful to identify problematic components - I don't think most end-users would accept a 777 where it is obvious that 60% of the load is caused by heavy textures and 3D models with lots of vertices - likewise, Nasal scripts having 30-40% impact on frame spacing and latency would also not be accepted for very long. Ideally, this would be a part of a "review" process once aircraft are committed to fgdata, or at least if they are to be considered for inclusion into a release. |
|
Objective
Document the main steps to understand performance issues at the aircraft level, i.e. FDM, 3D modeling, texturing and Nasal scripting.
|
|
you can use the osg-stats and use osgviewer/fgviewer in standalone mode to draw your own conclusions |
|
|
|
you can only draw reliable conclusions once you do multiple tests doing different settings/configurations, which includes all tools at your disposal - including different model viewers like osgviewer/fgviewer. |
|
|
|
in conjunction with viewing the model in osgviewer, fgviewer and fgfs using different settings, we can draw reliable conclusions. |
|
|
|
osgviewer is a good way to exclude FDM, Nasal and Canvas altogether - so it provides an excellent baselineAnd that is not specific to the Su15, you could just as well do the same tests with the 777 or the extra500, and anybody interested in understanding FG performance issues, is well-advised to use such tests to determine if, how and where certain features are contributing to those |
|
Approach
|
|
you will obtain the "maximal theoretical performance" of each system (in isolation), so that you can check if/how complexity of the 3D model and/or FDM is playing a role here.
Next, you would do the same thing with Nasal code, i.e. by disabling it entirely and watching performance then. |
|
We will be testing different components, and features, of each aircraft in isolation, to exclude certain factors and determine if/how performance is affected by certain features.
For instance, when loading an aircraft in osgviewer, we can basically determine the maximal theoretical rendering speed that can be obtained (flightgear using osgviewer internally), without adding any flightgear specific subsystems/overhead (e.g. no scenery/terrain).
Equally, when running a FDM in standalone mode, we can look at the FDM overhead without graphics and/or scripting playing any role.
And once we load the same aircraft in fgviewer, we can look at it without Nasal playing any role.
|
|
We do have some fairly heavy/complex models, some unnecessarily so - i.e. in areas that add little to the overall realism (e.g. 777 or extra500).To see for yourself, you could dump the scene graph to a file using the debug menu, and inspect the whole thing in osgviewer. Which is also a good way to see how much of this is due to FlightGear itself.To see the impact of your model in relation to scenery/terrain etc, you can use the draw masks /sim/rendering/draw-masks and the minimal startup profile: Troubleshooting crashes#Minimal startup profileTo see the impact of enabling/disabling Rembrandt, you would obviously have to re-do the test accordingly.But overall, this should give you a fairly good idea for your hardware, especially in conjunction with using the osg stats: Howto:Use the system monitor#Interpreting the OSG on screen stats |
|
Metrics
The main metrics will be osg-stats (link) and frame rate/frame spacing when in FG, as well as ogging performance monitor metrics (link)
Performance
|
|
What is a too high polygon count depends on your graphics hardware, how the polygons are to be rendered technically (single draw vs. multiple draws) and how much else is in the scene.For comparison, the GPU on my old computer would render everything up to ~400.000 vertices fine and then framerate would collapse, my new computer manages about 4 million vertices without hard framerate drop and has still delivered acceptable results with 12 million vertices.So if you put a single million polygon model into the scene, it's not a massive problem for modern graphics cards, but it doesn't scale - if others get to see your model over MP (because you are perhaps not providing a lowres version) and there are ten such planes in the scene, it's misery for everyone. Likewise, if you happen to meet a user with a low performance GPU in MP, he is going to suffer.The rule is hence that a plane in FG should use as high polygon count as needed, not more.Beyond some point (usually well below 100.000) vertex count doesn't influence visual impression much any more, rather detailed texturing, normal maps, reflection effects and good interaction with the scene light become the dominating factors. So in my view, there is simply no point to drive vertex counts up to the millions, other than to impress users who are impressed by large numbers. A gifted modeler can achieve the same visuals with good design and other techniques which render much faster. |
|
Understanding framerate
|
|
What drags framerate is the number of model vertices in the view frustrum. When you don't look at the larger part of the cockpit because you're essentially looking out of the window with little of the plane in view, you get decent framerate. When you have the lower part of the cockpit and the forward part of the plane in view, you get lower framerate. When you have most of the model in view, you get lowest framerate.Doesn't really matter whether you actually see so much of the plane - vertices blocked by opaque surfaces still count in this game. Even if you see an instrument, the nose wheel a few meters before it will still kill your framerate even if you can't see it. |
|
Use Cases
We will be using aircraft that were tested on the forum, especially:
- Su-15 vs. F15
- extra500
- 777-2000
standalone FDM
|
|
usually, the FDM accounts only for a fraction of the execution time over the overall time spent creating the frame, and given the complexity of the 3D model and textures, I would assume that quite some time is spent rendering, not updating FDM state. To see for yourself, you can open the performance-monitor, and post a screen shot here - next, do the same thing with the osg-on-screen stats.If in doubt, JSBSim also provides a way to run in standalone mode, you can basically run a 2 hr flight within a few minutes, and watch the whole thing using a task monitor (top/htop on Linux).This should tell you immediately if you are FDM-bound (which really is a fancy term for saying that you'd be CPU-bound), which seems unlikely as far as I can see. |
|
osgviewer
osgviewer is the basic tool that you should use to measure performance of a 3d model, it comes with compiling Openscenegraph with the -DBUILD_OSG_APPLICATIONS option, it provides the basic OSG stats interface, FPS/Cull/Draw being the most important ones for the ultimate bearing load of your model on your machine.
If you want to compare models, for consistency, it is better to use a static camera view(default camera angle/pov is fine), with the wire-frame option enabled to measure the Draw for both visible and hidden faces, wait for a perfect moment and press 'c' to capture the screenshot. Also it is helpful to watch metrics as you disable/enable textures.
| Key combination | Action |
|---|---|
| 1 | Select 'Trackball' camera manipulator (default) |
| 2 | Select 'Flight' camera manipulator |
| 3 | Select 'Drive' camera manipulator |
| 4 | Select 'Terrain' camera manipulator |
| 5 | Select 'UFO' camera manipulator |
| Drive: Space | Reset the viewing position to home panel hotspots |
| Drive: a | Use mouse middle,right mouse buttons for speed |
| Drive: q | Use mouse y for controlling speed |
| Escape | Exit the application |
| Flight: Space | Reset the viewing position to home |
| Flight: a | No yaw when banked |
| Flight: q | Automatically yaw when banked (default) |
| Trackball: + | When in stereo, increase the fusion distance |
| Trackball: - | When in stereo, reduse the fusion distance |
| Trackball: Space | Reset the viewing position to home |
| Z | If recording camera path stop recording camera path, save to "saved_animation.path" |
| b | Toggle backface culling |
| c | Take screenshot and write to file. |
| f | Toggle fullscreen |
| h | Display help |
| l | Toggle lighting |
| s | Toggle instrumention/stats |
| t | Toggle texturing |
| v | Toggle block and vsync |
| w | Toggle polygon fill mode between fill, line (wire frame) and points |
| z | Start recording camera path. |
FGViewer
This tool is the easiest because you don't need to have osg binaries compiled, just navigate to where you installed Flightgear to find it, it is based on osgviewer but with some Simgear integration, it is not sophisticated and i don't think you can allow it to display aircraft and scenery together, use it as you would osgviewer, except you have to define FGROOT before using it.
Windows example
C:\Users\JonDue> "C:\Program Files\Flightgear 3.7.0\bin\fgviewer" C:\Location\Of\Aircraft\Model.acLinux example
vblank_mode=0 ./fgviewer ~/build/fgbuild/install/flightgear/fgdata/Aircraft/ufo/Models/ufo.ac
Disabling Nasal
Minimal startup profile
Using draw masks
|
|
people with very powerful hardware who are still getting only ~30 fps may want to hide the aircraft using /sim/rendering/draw-masks to see for themselves how much of an impact the 3D model/texturing vs. scenery/clouds etc has, note that this will not have any effect on running Nasal code |
|
flight recorder
|
|
Core Development
|
|
Those osg stats should really be dumped to the console during startup, maybe just the aircraft-specific branch, i.e. separated scenery/aircraft to write into the log file how complex an aircraft/airport (location) really is/was.I really believe that this is a useful metric, and the approach could also be suited to understand other issues, i.e. rendering a minimal view of a scene and removing/adding different elements using draw-mass to see if/how they impact overall scene complexity and performance.This would probably also help us understand Rembrandt issues, and it would allow us to identify opportunities for optimiing Canvas further - e.g. by allowing such stats to be gathered "per-canvas" (texture), querying the whole sub-graph for the corresponding cam, which is possible using existing osg APIs)In fact, aircraft/airport complexity I would even log to the splash screen right away, to provide some data to aircraft developers |
|