FlightGear and OpenGL ES

From FlightGear wiki
Jump to: navigation, search
This article is a stub. You can help the wiki by expanding it.
OpenGL ES compatible subset of FlightGear
Started in 04/2020
Description identify, patch and build a subset of FlightGear compatible with GLES
Contributor(s) talks
Status RFC
Open Tickets Compositor

Some FG users are very interested in seeing how FlightGear runs on lower end/embedded hardware (think RPi-style) using OpenGL ES This is a link to a Wikipedia article.

The bigger issue here is we need to ditch PUI (which is in progress) and some OpenGL 1.0 code (HUD, 2D panels especially - can be #ifdef for now) so we can enable Core profile on Mac - since Mac 4.x support (we only hit about 4.3 alas, but with some extensions to get in sight of 4.5) is Core profile only, no Compatability mode.

I *believe* the new open-source Intel-Mesa drivers on Linux (which are supposed to decent quality, and even fast) might be in the same situation.[1]

Probably to do it initially, GLES1 would have to be used - I think GLES2 has no fixed function - and simgear/flightgear would have to be patched to use only OSG calls (if possible), e.g. by excluding features/subsystems that still use legacy OpenGL code incompatible with GLES.

OSG can have GLES{1,2} support without windowing compiled in.

OpenGL 1 and 2 can be mixed but OpenGL ES 1 and OpenGL ES 2 can't. There is a performance price to pay for this backward compatibility. For this reason, we should get rid of PLib because this (old) code is all OpenGL 1 that may slowdown the all rendering pipeline.

For historical reasons, Flightgear and Simgear are mixing OpenGL 1 (old code) and OpenGL 2 (more recent code).[2]

Background

1rightarrow.png See Howto:Optimizing FlightGear for mobile devices for the main article about this subject.

The OpenSceneGraph port initiated in 2006 has never been fully completed, so that there is a certain amount of code making use of legacy OpenGL calls, which complicates modernizing the renderer.

In particular, this means that Supporting multiple renderers is unnecessarily complicated, unifying the 2D rendering back-end is a long standing challenge. It's only since just very recently, that supporting different renderers is being worked on thanks to the Compositor effort.

But even then, phasing out legacy code or porting it, still needs to be addressed sooner or later.

Being able to run fgfs on such, comparatively low-powered, systems using OpenGL ES can actually be a good thing for fgfs as a whole - it can help us understand bottlenecks that are hardly visible on typical gaming/developer rigs, but that may still show up over time (think leaking listeners/memory) - this sort of thing can also be considered the prerequisite for people wanting to target/build/run fgfs on other embedded hardware, such as thin clients with integrated GPUs or even mobile phones/tablets (think Android)

we need to remove PUI and change the Canvas not to use Shiva, to be ES2 compatible or Core-profile compatible.[3]

In other words, if the right people were to team up to specifically target such hardware, this could also mean significant performance improvements for people on powerful gaming rigs.

Approach

For starters, we can try the FlightGear Headless option to build a fgfs version without showing any graphics at all.

The next step will be identifying and excluding problematic sources (those containing legacy/raw OpenGL code, e.g. using glBegin() or glEnable() respectively):

https://sourceforge.net/p/flightgear/flightgear/ci/next/tree/src/Cockpit/render_area_2d.cxx

void RenderArea2D::RenderQuad( const SGVec2f *p) {
    glBegin(GL_QUADS);
        glNormal3f(0.0f, 0.0f, 0.0f);
        glVertex2fv( p[0].data() );
        glVertex2fv( p[1].data() );
        glVertex2fv( p[2].data() );
        glVertex2fv( p[3].data() );
    glEnd();
}

Whenever FlightGear sources contain such or similar code, it is pretty safe to assume that we will need to port/exclude such modules from compilation to ensure that no legacy OpenGL code is executed at runtime.

Instrumentation/HUD/HUD_runway.cxx
Instrumentation/HUD/HUD_tbi.cxx
Instrumentation/HUD/HUD.hxx
Instrumentation/HUD/HUD_instrument.cxx
Instrumentation/HUD/HUD_ladder.cxx
Instrumentation/HUD/HUD_dial.cxx
Instrumentation/HUD/HUD_tape.cxx
Cockpit/render_area_2d.cxx
Cockpit/panel.cxx
GUI/WaypointList.cxx
GUI/CanvasWidget.cxx
GUI/MapWidget.cxx

This means, we'll need to review/disable the compilation of the following folders in $FG_SRC and patch up all hard-coded references to these systems:

  • Instrumentation/HUD
  • Cockpit
  • GUI

In addition, there's Viewer/PUICamera.cxx which references glEnable()

Example: Disabling PUI

1rightarrow.png See Developing using CMake for the main article about this subject.

We should add separate build options to explicitly disable certain features individually.

The bigger issue here is we need to ditch PUI (which is in progress) and some OpenGL 1.0 code (HUD, 2D panels especially - can be #ifdef for now) so we can enable Core profile on Mac - since Mac 4.x support (we only hit about 4.3 alas, but with some extensions to get in sight of 4.5) is Core profile only, no Compatability mode.[4]

Knowing that PUI contains legacy OpenGL code and knowing it's scheduled to be removed anyway because it isn't compatible with modern OpenGL, we will disable it completely without reviewing/porting individual PUI files. This means opening $FG_SRC/CMakeLists.txt to add a new option to disable PUI.


Successfully disabling a feature means primarily:

  • adding a corresponding new build option to the top-level CMakeLists.txt (e.g. DISABLE_PUI)
  • opening fg_init.cxx and navigating to the lines where the feature/subsystem is initialized
  • wrapping the corresponding code in between #ifdef...#endif blocks
  • locating any remaining references to the subsystem in question and repeating the last step there to ensure that removed subsystems are not accessed at runtime

https://sourceforge.net/p/flightgear/flightgear/ci/next/tree/CMakeLists.txt

option(SYSTEM_CPPUNIT    "Set to ON to build Flightgear with the system's CppUnit library")
option(DISABLE_PUI    "Set to ON to build Flightgear without PUI support")

if(DISABLE_PUI)
      add_definitions(-DDISABLE_PUI)
endif(DISABLE_PUI)

We will be adding #ifdef macros to the sources in question, and update CMakeLists.txt accordingly.

Ideally, in conjunction with a feature-specific build option to disable the corresponding feature (think PUI or the HUD).

Thus, after editing CMakeLists.txt, we need to open fg_init.cxx to prevent initalization of PUI.


https://sourceforge.net/p/flightgear/flightgear/ci/next/tree/src/Main/fg_init.cxx#l1043

    ////////////////////////////////////////////////////////////////////
    // Create and register the XML GUI.
    ////////////////////////////////////////////////////////////////////
#ifndef DISABLE_PUI
    globals->add_subsystem("gui", new NewGUI, SGSubsystemMgr::INIT);
#endif


So, after editing fg_init.cxx to prevent the PUI GUI from getting initialized by FlightGear, we will need to find remaining hard-coded references to it, to fix those up and deal with PUI not being available. This means grepping $FG_SRC for any references to "pui" to locate remaining get_subsystem() calls. (FIXME: new subsystem lookups use templates), this will include code in unrelated modules, e.g. fgcommands (think menu bindings) accessing the GUI via something like get_subsystem("gui");

https://sourceforge.net/p/flightgear/flightgear/ci/next/tree/src/Main/fg_scene_commands.cxx#l277

    NewGUI * gui = (NewGUI *)globals->get_subsystem("gui");
    if (!gui) {
      return false;
    }

Thus, it makes sense to intercept registration of these fgcommands at the bottom of the file by wrapping these inside ifdef macros:

https://sourceforge.net/p/flightgear/flightgear/ci/next/tree/src/Main/fg_scene_commands.cxx#l497

/**
 * Table of built-in commands.
 *
 * New commands do not have to be added here; any module in the application
 * can add a new command using globals->get_commands()->addCommand(...).
 */
static struct {
  const char * name;
  SGCommandMgr::command_t command;
} built_ins [] = {
    { "exit", do_exit },
    { "reset", do_reset },
    { "reposition", do_reposition },
    { "switch-aircraft", do_switch_aircraft },
    { "panel-load", do_panel_load },
    { "preferences-load", do_preferences_load },
    { "toggle-fullscreen", do_toggle_fullscreen },
    { "screen-capture", do_screen_capture },
    { "hires-screen-capture", do_hires_screen_capture },
    { "tile-cache-reload", do_tile_cache_reload },
#ifndef DISABLE_PUI
    { "dialog-new", do_dialog_new },
    { "dialog-show", do_dialog_show },
    { "dialog-close", do_dialog_close },
    { "dialog-update", do_dialog_update },
    { "dialog-apply", do_dialog_apply },
    { "open-browser", do_open_browser },
    { "gui-redraw", do_gui_redraw },
#endif
    { "add-model", do_add_model },
    { "presets-commit", do_presets_commit },
    { "press-cockpit-button", do_press_cockpit_button },
    { "release-cockpit-button", do_release_cockpit_button },
    { "dump-scenegraph", do_dump_scene_graph },
    { "dump-terrainbranch", do_dump_terrain_branch },
    { "print-visible-scene", do_print_visible_scene_info },
    { "reload-shaders", do_reload_shaders },
    { "reload-materials", do_materials_reload },
    { "open-launcher", do_open_launcher },
    { 0, 0 }			// zero-terminated
};

This will ensure that fgcommands that are PUI related won't be available in a non-PUI build.

We will also need to look for other files accessing the "NewGUI" subsystem grep -nr "NewGUI" -l:

Main/subsystemFactory.cxx
Main/fg_scene_commands.cxx
ATC/atcdialog.cxx
ATC/atcdialog.hxx
Autopilot/route_mgr.cxx

Since we have excluded $FG_ROOT/GUI from the build and updated fg_init.cxx, we will only need to review/patch these files to ensure that there are no hard-coded references to PUI if it's not available, using the same ifdef based approach.

Obviously, fgdata level resources like menu bindings and/or Nasal code may still try to execute such bindings.

Finally, there may still be other PUI specific references in the source tree ($FG_SRC), so that it does help to check the other folders next: grep -nr "PUI" -l

Since we have excluded $FG_SRC/GUI from the compilation, we can safely ignore any references to that folder, with the remaining ones being:

Input/FGMouseInput.cxx
Main/fg_os.hxx
Main/bootstrap.cxx
Viewer/renderer_compositor.hxx
Viewer/PUICamera.cxx
Viewer/renderer_legacy.cxx
Viewer/CMakeLists.txt
Viewer/PUICamera.hxx
Viewer/renderer_compositor.cxx
Viewer/renderer_legacy.hxx
Viewer/GraphicsWindowQt5.cpp
Canvas/canvas_mgr.hxx

The renderer is basically just referencing the PUI camera so that the GUI can be drawn, whereas canvas_mgr merely acccesses the PUI subsystem to be able to render canvas based textures. In other words, both references can be easily removed for testing purposes.

Affected features and sources

Feature Directory Notes Status
PUI $FG_SRC/GUI trivial to disable, can be replaced via Phi 70}% completed
2D Panels $FG_SRC/Cockpit legacy 2D panels (could use Canvas), actually as of 04/2020, there is now a dedicated OpenGL ES based version of the standalone fgpanel port available which could be backported (mainly contains hard-coded shader updates) [5] See also merge request #191:HLA update + fgpanel on Raspberry Pi and the discussion [6]

There's also a branch (which is a few of years old) contains WIP on implementing 2D panels as canvas. It uses the same loading / updating logic in C++ (for 100% compatibility), but rather than building custom rendering, it builds up a Canvas element hierarchy in C++.[7][8]

Pending Pending
Canvas Path (Shiva) $SG_SRC/canvas needs some thinking, probably native OpenVG on RPi [9] [10] or an OpenGL based implementation like nanovg [11] Pending Pending
Effects $FG_ROOT/Effects not important for the time being, will include Shaders and probably involve a custom Compositor pipeline Pending Pending
$FG_ROOT/Docs/README.osgtext ... If you have a chance to replace OSGText by canvas - do so. OSGText (even with the custom OSG repository) does not play nice with our Effects and Shaders framework and will continue to have issues for all eternity.[12][13] [14] Pending Pending

In addition, SimGear needs special treatment, too [15]. Using the same heuristics as before (based on grepping $SG_SRC for glBegin and glEnable, we end up with the following sources (ignoring Canvas Path /shivavg, which is dealt with already above):

simgear/scene/tgdb/SGVasiDrawable.cxx:42:    glBegin(GL_POINTS);
simgear/scene/model/shadanim.cxx


Feature Directory Notes Status

Porting

OpenGL ES This is a link to a Wikipedia article

Vertex buffer objects

Canvas.Path (OpenVG)

Will likely need to replace shiva vg with an OpenGL 2.0 based implementation like nanovg [16].

Community talks

References

References
  1. https://sourceforge.net/p/flightgear/mailman/message/36341578/
  2. https://sourceforge.net/p/flightgear/mailman/message/36980923/
  3. https://sourceforge.net/p/flightgear/mailman/message/36194973/
  4. https://sourceforge.net/p/flightgear/mailman/message/36341578/
  5. https://sourceforge.net/p/flightgear/flightgear/ci/dc6fb535962eff8214bc5b1708886178cceb482e/log/?path=/utils/fgpanel
  6. https://sourceforge.net/p/flightgear/mailman/message/36978512/
  7. https://sourceforge.net/p/flightgear/mailman/message/36980724/
  8. https://sourceforge.net/p/flightgear/mailman/message/37042351/
  9. https://github.com/ajstarks/openvg
  10. http://mindchunk.blogspot.com/2012/09/openvg-on-raspberry-pi.html
  11. https://www.raspberrypi.org/forums/viewtopic.php?t=247586#p1516785
  12. https://forum.flightgear.org/viewtopic.php?f=37&t=36287&p=353351&#p353351
  13. https://sourceforge.net/p/flightgear/codetickets/2199/#f702
  14. https://sourceforge.net/p/flightgear/mailman/message/37042476/
  15. https://sourceforge.net/p/flightgear/mailman/message/36978098/
  16. https://github.com/memononen/nanovg

Related content