PUI

Caution  The feature documented below, in its current form, is currently being scheduled/expected (or discussed) to be significantly updated, or phased out, in future FlightGear versions 2024.2+ (rough estimate). This should be taken into account if you're interested in working on the feature or developing it further. If in doubt, please get in touch via the devel list first. (Plib/PUI is to be removed and replaced by a Canvas/Nasal based solution to help improve compatibility with OSG 3.6+ and OpenGL 4.x (OpenGL Core Profile, see 2022.X Release plan ) and improve performance by phasing out legacy OpenGL code. Therefore, people should be careful when extending the PUI based legacy GUI engine and coordinate any related work with the devel mailing list first-this applies particularly to adding any additional hard-coded PUI widgets.)

FlightGear GUI dialog (exit.xml)stored in $FG_ROOT/gui/dialogs

PUI is the standard GUI engine used in FlightGear, it is part of PLIB and is using raw, fixed-pipeline, OpenGL code internally (no OpenSceneGraph). As of mid 2021, FlightGear is in the process of working towards adopting the OpenGL Core Profile, this means that PUI is causing trouble and needs to be replaced/disabled or ported.

PUI provides a fairly basic, but robust, set of widgets. PUI is also used for rendering the Menubar.

FlightGear uses a GUI widget set that is implemented on top of raw (legacy) OpenGL. This has many advantages from a portability standpoint and from the standpoint of integrating with window systems. PUI doesn't have every feature under the sun, but it was never meant to. It's relatively small, lean, mean, and written on top of OpenGL which makes life *much* easier for us.

PUI already is a separate distinct library within Plib. It depends on some central utility stuff, but that's about it. So it is pretty stripped down and separate already.

We are to some extent hamstrung by the rather old GUI toolkit we use. However, replacing that is going to be non-trivial, and it would affect not just the core GUI but also all the dialog boxes that have been set up for particular aircraft.

Flightgear can look amazing when you use the HDR pipeline. We absolutely need this to stay relevant. The problem is that HDR is not compatible with OpenGL1 PUI. Work on getting rid of PUI has been going on for over a decade (longer than it took to implement it).

Replacement status

See also FlightGear_and_OpenGL_ES#Example:_disabling_PUI.

Note  Update 09/2022: The PUI replacement was going to be Qt but it started to get very complicated with changes in Qt 5.15 + Qt 6, so James is going with a more light-weight Canvas based approach now. Qt added support for Vulkan / Metal / D3D starting in 5.15, but FlightGear / OpenSceneGraph can’t support those, so integrating the two renderers went from being ‘complicated but ok’ to ‘very very complicated’. So now James is going with something much more lightweight using some C++ compatibility code, some Nasal for styling and the existing Canvas widget rendering from Thomas Geymayer (TheTom) with some extensions and additions, based on the plans originally discussed when the Canvas GUI system was added to FlightGear: some pieces are in FlightGear & FGData already. James has basic dialogs working okay but not the more complex ones and everything looks kind of ugly, he needs to improve the visual look before he shares screenshots to avoid everyone freaking out :) The disadvantage of this approach is James is far from expert at creating visual appearances this way, so it’s kind on unrewarding and slow for him. If someone likes messing with CSS-type styling, border-images and hover-states, ping him since we could probably move things also faster

Motivation

See Core Profile support for the main article about this subject.

The bigger issue here is we need to ditch PUI (which is in progress) and some OpenGL 1.0 code (HUD, 2D panels especially - can be #ifdef for now) so we can enable Core profile on Mac - since Mac 4.x support (we only hit about 4.3 alas, but with some extensions to get in sight of 4.5) is Core profile only, no Compatibility mode.

Improving the frame-rate and modernised 3D rendering, can’t be worked on until the PUI code, 2D panels and Shiva are removed, but doing so is a frustrating slow path

Note  There is now an alternative to ShivaVG in SimGear - ShaderVG, which is working pretty much as good as ShivaVG. It can be enabled by passing -DENABLE_SHADERVG to cmake when building SImGear from source.

Given that with many graphics drivers PUI doesn't render correctly when higher shader quality is on, many people are convinced PUI needs to be replaced.

Canvas Emulation

See also Canvas widget matrix.

Last updated: 04/2023

Gallery

• 

  about.xml PUI/XML dialog parsed/rendered via the Canvas GUI and the underlying XMLDialog.nas parser/translator (04-2023)

• 

  logging.xml PUI/XML dialog parsed/rendered via the Canvas GUI and the underlying XMLDialog.nas parser/translator (04-2023)

• 

  logging.xml PUI/XML dialog parsed/rendered via the Canvas GUI and the underlying XMLDialog.nas parser/translator (04-2023)

• 

  Screen shot showing the XMLDialog.nas bridge in use to parse/render the environment.xml PUI/XML dialog (unmodified)

Status

As of 01/2023, the replacement for PUI is making good progress.

In February 2022, James reported having something in progress locally, and that should even be something we can try in March/April 2022. Hopefully that’s quick enough, knowing it has been a long time coming.

There's now an XML to Nasal bridge, to keep PUI dialogs working (Disabled by a CMake option). This is implemented on top of the Canvas system.

This builds equivalent C++ objects to what the PUI dialogs build, with properties exposed to Nasal. Peer objects are created by Nasal callbacks, which can implement the various dialog functions needed to keep compatibility, especially the ‘update’ and ‘apply’ hooks.

For remaining work to be done, please refer to [1]

Note  The PUI Compatible Canvas GUI is a compile time option: Use the following cmake option to enable this code: -DENABLE_PUICOMPAT=ON (This requires latest SG/FG AND FGDATA requires next). But this is all highly fluid and most of it really developer only. At one point they will become default or be accessible in the launcher. If you compiled with PUICompatibility enabled, but still don't see any dialogs - only the menubar, you might need to enable the GUI module in defaults.xml:--prop:/nasal/gui/enabled=1

The new Compatibility files can now be found in $FG_SRC/GUI:

• flightgear/src/GUI/FGPUICompatDialog.cxx
• flightgear/src/GUI/FGPUICompatDialog.hxx
• flightgear/src/GUI/PUICompatObject.cxx

01/2023: James is slowly working on creating the pop-up menu and combo-box widgets [...]

One of the major things slowing down James' work on the popup-menu and combo box is time to restart the simulator for each change, he is looking for some solution to re-load the widget Nasal code independently.

James has done about 75% (hah!) of the C++ work to enable live reloading of modules this way, but unfortunately there are some code paths that would become crashy if you use the feature for the Canvas: because it would reference the ‘old’ (pre-reload) Nasal code, and not the new one, and therefore you’d get a weird mix of old and new, and then just crash the sim. To fix it properly I need to track down those places that store a reference to Canvas and give them a re-init method, so we don’t keep the stale references.

The general non-fun-ness of that kind of debugging is James has been making such slow progress on my PUI replacement widgets recently

Popup menus and combo box widgets are working now - also, the simulator doesn't have to be restarted anymore - any Nasal module (including the canvas module can now be reloaded live:

fgcommand("nasal-reload", {"module": "canvas"})

Todo

Nothing in particular: help debugging the grid layout (which is in simgear) would be good. For now, you could create a grid layout manually using some Nasal.

The grid layout is now mostly working (from my experience) but it behaves weirdly when there isn't enough place to accommodate all widgets.

Global menu keybindings won’t worry but don’t touch that, I have a separate plan for it.

As of 04/2023, remaining items on the todo list are:

• Grid layout
  • GridLayout needs debugged: After this, the basic UI is usable, would probably make it a switchable option on ‘next’ (I can already use the simulator with the new UI today, but all the sizes are off in the grid layouts which we use everywhere)

• list Done
• table (and tree ?) models corresponding views
• waypoint list
• Re-write the waypoint list and airport dialog to work again (since these use custom widgets):
  • log list
  • airport-list equivalents
• Nasal access to the menu i18n functions
• selection support for single- and multi-line text (which is being worked on -> CanvasPangoText)

Note  For the LineEdit widget, a hack has been implemented to provide more-or-less working selection highlighting while the PangoText element isn't done.

• Keyboard shortcut handling, although this is not really a PUI vs replacement issue, to me we should handle UI shortcuts in the same was normal keyboard.xml shortcuts
  • this previously worked because PUI got ‘first chance’ at handling key input. But we always had duplication between keyboard.xml and other places, so I would rather standardise on keyboard.xml, but allow a UI element to set a scope for shortcuts. (So that Copy/paste shortcuts etc can work in a text input, at least … but probably some other standard ones)
• Ensure all the places which use the PUI combo-box and list-widget work : there’s a few of these.

The CanvasPangoText stuff is needed but we can perfectly well run the UI on next without it.

Parser

See pui2canvas for the main article about this subject.

The corresponding Nasal/Canvas module to dynamically "translate" legacy PUI/XML dialogs into Canvas dialogs (at runtime), is to be found in fgdata/Nasal/gui/XMLDialog.nas

Widgets

See Howto:Creating a Canvas GUI Widget for the main article about this subject.

It’s split between Simgear (see classes with widget / layout in the name) and in FGData. (Eg widgets/Button.nas). To be able to use the existing dialog XML files un-modified (which is a design goal), James is extending the widget types with many additional ones (eg PUI has slider, dial, combo-box, checkbox, all of which need to be created, see Canvas widget matrix.

Thomas’s canvas widgets have a very good separation of API + state from appearance, so all styling is in its own file, and James is being very strict about maintaining this separation, so we also retain the re-styling feature of the PUI UI, which many people also rely on. This does make the process of adding new widgets more complex, however.

See also Canvas widgets.

Layouting

The other thing is to preserve all the layouting: James has added a grid layout to Simgear, since that is supported by the existing PUI code (even though the layouts are not actually part of PUI itself). The problem is getting the sizing / hinting of all the widgets to match the PUI values, so that dialogs look approximately the same under the new UI as they did with PUI; again this a design goal so that all existing dialogs in aircraft and add-ons, which we can’t update, continue to work and be usable. Debugging that is also proving quite tricky, since there’s all kinds of hard-coded assumptions built into PUI widgets about pixels, font-sizes etc which are not true in the new system.

Background

In FlightGear, PUI dialogs are standard PropertyList XML Files that are stored in $FG_ROOT/gui/dialogs, they can contain the widgets mentioned in $FG_ROOT/Docs/README.gui, using a simple layout engine discussed in $FG_ROOT/Docs/README.layout, and bindings using a combination of so called fgcommands operating on properties (see $FG_ROOT/Docs/README.commands) and custom Nasal (FlightGear scripting) code.

In addition, each PUI/XML dialog may contain Nasal script sections that are executed when opening/closing the dialog, a feature which is commonly used for procedurally creating/updating widgets using the cmdarg() API, which allows the dialog tree to be traversed and manipulated prior to the dialog being rendered. Widgets can be conditionally hidden/shown using a wrapper for SGCondition in props.nas The canvas widget also supports its own embedded Nasal code section to execute arbitrary widget specific Nasal code upon opening/closing the dialog/widget.

PUI/XML dialogs can be loaded, dynamically created, updated and closed using a handful of fgcommands:

• dialog-new
• dialog-show
• dialog-update
• dialog-apply
• dialog-close

PUI related OpenGL code is particularly infamous for causing rendering artifacts for people on AMD/ATI and Intel hardware (especially in combination with certain fonts/styles and effects/shaders), see ticket #2213.

PUI is also known to affect rendering performance quite significantly (see forum search for anthrax+gui FlightGear Forum), while also preventing FlightGear from using a more recent version of OpenGL.

However, improving the frame-rate and modernised 3D rendering, can’t be worked on until the PUI code, 2D panels and Shiva are removed, but doing so is a frustrating slow path

Besides, while most people seem to agree PUI needs to be replaced, it sounds as if the fallout from doing so would be more painful (cumulatively) than the pain its existence causes.

But given that with many graphics drivers PUI doesn't render correctly when higher shader quality is on, graphics folks are also convinced it needs to be replaced.

In addition, OpenSceneGraph (OSG) can obviously not help optimize any PUI related GL code and PUI widgets are generally considered to be pretty archaic and not easy to extend.

Note  We use the GUI code from PLIB, which doesn't know anything about OSG. See the SGPuDrawable class in $FG_SRC/Viewer/renderer.cxx for the implementation. The one catch is that OSG has a strong notion of separation between the update of a "scene" and its rendering, and that might not play well with arbitrary existing OpenGL code.

As of late 2015, there is heavy activity towards providing alternatives to a PUI-based UI:

• Integrated Web GUI (external, browser-based - fully asynchronous)
• Howto:Processing legacy PUI dialogs using Canvas (internal, using the Canvas system and a simple Nasal parser to deal with existing PUI/XML dialogs)

• 

  Screenshot showing Phi

• 

  Screenshot showing a dialog rendered by the pui2canvas parser

History

See also QtQuick use in FlightGear.

Caution  This is merely kept for future reference, many of the ideas discussed below have become obsolete as of 09/2022, if in doubt, please get in touch via devel-list.

2021

In early 2021, James did a brief evaluation of ImGUI as a possibility - he's also evaluating it in some projects at his day job.

The API is interesting if you’re starting from scratch and only exclusively in C++, but it’s not a great fit for how we define GUIs in FG (via XML/Nasal): there is no back door that he could find, to access the persistent state of the GUI, or build it up in a data-driven way. So making a mapping to keep our existing GUI XML working (and updating correctly) becomes a bit of a chore.

Pretty much all of the FlightGear GUI is defined in XML space, except for a few oddments. Let’s say 95% at this point.

Additionally, even if we were to change that for the core sim (and James' current plan is to keep the XML syntax and just extend it), the installed base of aircraft ship their own XML dialogs (eg., replacing the autopilot dialog with a custom one, or just adding their own aircraft-specific helper controls for doors / lights / cargo / whatever). So even if we found the most sublimely beautiful, intuitive, compact GUI description language in the world, we have to keep the XML syntax working as-is, for the foreseeable future.

Given this, James' intention is to replace the output side PUI but keep the front-end side (XML / properties / bindings) unchanged, and therefore backwards compatible, and simply add some new types / options / widget types going forward. This will mean we still have to deal with the slightly cumbersome XML+Nasal syntax, but, well, it’s not *that* cumbersome.

In mid 2021, James reported that he is working on some ‘PUICompat’ classes which hold/own/represent the GUI state in C++, but which can be used with Nasal CppBind. And of course we could indeed expose those later on.

In July 2021, James suggested not to worry about the existing GUI code, it’s going in the bin.

For the time being, the in-sim GUi can’t do Unicode yet, but James is working on it, hopefully available soon. Also, the in-sim menubar will change with the new GUI, *and* we use a native menubar on macOS. Until James replaces the PUI fonts we can’t use Unicode symbols. So, will have to come back to it once the new GUI is merged.

2020

As of 03/2020, the PLIB replacement was reported to be in the backlog .

With PLIB replacement work likely being re-scheduled for Post FlightGear 2020.2 LTS changes .

In 08/2020, James stated that he was still working (exceptionally slowly!) on replacing PUI (as part of removing legacy OpenGL code (PUI, HUD, 2d panels, porting Canvas away form ShivaVG), so we can switch to Vulkan/VSG at some point (ideally at some point before Apple turn off OpenGL support…)), and that he doesn't think /any/ of us *want* to be working on these features particularly, but projects like these need to be done before anyone can have fun with VSG or Vulkan (personally James would be very enthusiastic about working on VSG support (being on macOS, I also have the most to lose / gain from it)). Sometimes that’s just how it goes. Equally they will all benefit the project ultimately, but the payoff both for the individual and the project is very drawn out.

In 10/2020, the estimation is that PUI should be going away really ‘soon’, the replacement code is already in next / 2020.2. (There’s no plan to actually /use/ the new UI code in 2020.2, but if the bugs in it prove simple, we could actually turn it for specific dialog: it seems very unlikely this makes sense for a stable release however)

The fixed-function removal, is actively in hand (Gaetan is working on the 2D panel part, and James is going to focus on PUI in the rest of 2020). James hopes that means next year he can focus on Vulkan, or at least, using OSG as if /were/ VSG, so the migration in a year or two when VSG is stable, is not so painful.

The PUI UI and replacement can co-exist in the same build (they already do, effectively.

In 11/2020, James suggested not to worry about details such as PUI styling/theming etc: we’re not making such changes for the LTS, and PUI should be gone before the next release. Also, he recommended (to everyone) to stop touching GUI stuff for a few weeks, as he's about to turn it all on its head.

2019

The new GUI has been incubating for a few years, but there's apparently been a lot of progress as of 10/2019.

It is likely the new GUI will be a user opt-in feature - at least initially. The sunset of the PLIB library _could_ happen relatively soon, but people will need to see the new GUI and react to it before we could commit to PLIB removal .

In 02/2019, James stated that the new UI is coming quite soon.

2018

See also QtQuick use in FlightGear.

According to James, we have to accept that changes like testing or making PUI a modular thing or moving the JS code tend to have an immediate pain for some people (because strange stuff gets broken, and takes some time to get fixed), whereas the payoff in terms of improved joystick handling or replacing the UI or testing of all subsystems will take some months or years to be felt.

James said he was in the middle of porting the launcher’s final tab (location) to the new UI scheme, once that’s done he will start sketching out the ‘in sim’ settings UI using the same pieces. He’ll send a request for contributions around about that in the next three-four weeks, once he has enough templates & examples that there is a clear pattern to follow.

James announced that the launcher is now feature complete with the QtQuick UI, he’ll keep making bug-fixes of course as people report them, but my todo list for the core features is now done.

2017

In 10/2017, James said he was getting really close to having the PUI replacement UI suitable for beta-testing.

Originally, James was hoping to land the PUI replacement GUI in the last dev cycle in 2017 (at least as a proof-of-concept, probably not as the default UI), so wouldn’t expend lots of effort on things like collapsible sections which might be a lot of work with the current PUI/Canvas approaches, but are trivial with the new QtQuick based UI scheme.

Once the basic new UI is in place we can experiment with different re-arrangements easily, without being limited by PUI. (James expects PUI to live on as the default / alternate UI while this happens)

Technically, this entails replacing dialog.cxx with some code which builds up some special ‘PUI-Emulation’ QtQuick controls, so that existing dialogs (especially from Aircraft) continue to work. This mode will look somewhat ugly but hopefully no more ugly than PUI! And it helps that the set of widgets we have in PUI is limited, and the ‘tricky’ widgets (map, scrolling list, etc) are in the dialogs will will replace with new ones first.

This PUI emulation isn't going to use any code from PLIB, so what will remain from it in FG afterwards will be the FNT and JS components (reworked).

2016

We need to explore if Qt Quick or widgets is the better way to go, with a preference towards Qt Quick, since it will allow things closer to current PUI (semi-transparent widgets on top of the 3D content). It still allows creating separate windows too (Erik’s use case), but means we get consistent, custom theming on all platforms, since standard desktop look is not so nice for a flight-sim.

James intention is to make a QML-hosting OSG-drawable, and add this as a concept in Qt-enabled builds, then port some PUI dialogs to it, and see how easy / painful the process is. Hell create some QObjects which expose commands and the property tree and Nasal.

This has the advantage of not touching the OSG window at all, QtQuick simply provides some additional OpenGL rendering on top, which we pass events to - pretty much exactly the same as what we already do for PUI, and hopefully can co-exist with it. James got this working in a local OSG tree in a standalone demo, but that’s based on osgQt - he wants to see if he can make it work on a normal, non-Qt OSG window inside FlightGear.

The idea is to host an XML dialog (Currently done with PUI) from Qt also. The question is if that can be done keeping ‘source compatibility’ exactly with the current UI XML and Nasal interface. It probably can be, but many of the PUI widgets have really ugly APIs, especially for combo-boxes, drop-down menus and scrolling lists, where it makes sense to map a Nasal class to a Qt ItemModel, which would be more elegant and easy to work with in both Nasal and integrate on the Qt side. But obviously would mean changing the dialog sources.

Of course, aircraft dialogs tend to be simpler, and not use the more complex PUI widgets, precisely for reasons like this, but that’s a limitation it would be best to remove.

References

References