OSGText Issues
| Note The project "World Scenery 3.0 roadmap" is currently blocked by remaining OSGText Issues: The reason being as of 05/2021, we're now at the stage where the WS30 underlying scenery is close to "complete" - there don't seem to be any missing scenery features - and any issues remaining are ones that we can address at runtime (for example trees on roads). We should set ourselves a goal of delivering a full world build of WS30 via Terrasync for a preview release in the autumn of 2021. However, that of course relies on fixing the osgText issue[1] Fernando has now pushed two commits (SimGear commit 64e4adf0b95f12f2710d40550372b197423b3874 and FGData commit 49fa42faac6e0a1a89321cea4db3670a63662a58) that should hopefully fix the white osgText issue under OSG 3.6. Apparently assigning an Effect to the geode containing the osgText was enough to get it working. He thought it would conflict with OSG internal shaders, but it didn't. This might not be the case with the core profile, but has got a few tricks which should force our shaders to prevail over OSG's. Anyway, Fernando tested it and it looks fine to him. If someone who is more used to using osgText can report if the fix was successful and the text looks exactly the same as before, it would be greatly appreciated.[2] Please report back via the developers mailing list and/or the issue tracker.[3]. |
| This article or section contains out-of-date information
Please help improve this article by updating it. There may be additional information on the talk page. |
| Started in | 11/2020 |
|---|---|
| Description | Make text animations work correctly with the Effects framework |
| Contributor(s) | Icecode[4], Scttgs [5], Stuart[6][7], Zakalawe [8] |
| Status | testing first iteration of patches (blocking WS 3.0 and adoption of OSG 3.6) |
The OSGText animation ($FG_ROOT/Docs/README.README.osgtext) doesn't play nice with shaders.[10]
Another way to describe the bug is ‘make text animations work correctly with the Effects framework’. Because osgtext always uses a shader (now), and shaders cannot be combined, we have to find a smarter way to do that. But the fact that osgtext didn’t work correctly with Effects has always been there. So there is no upstream fix possible: we have to change what we’re doing on our side, but that will risk some aircraft compat issues. The patch we’re applying is to restore things to how they looked before, but it was still broken, just less obviously.[11]
It’s a fairly large number of aircraft: the text animation has been around for a long time, prior to Canvas for example. The problem is not us keeping 3.4.x around : it would be nice to switch to 3.6 everywhere, but we support both today.
Some cockpits have 50 of these (each MCP window, each radio numerical field, etc, etc). There’s at least one CDU impl where each line is two text items. (One for big, one for for small)[12]
The text issue has existed with OSG 3.6 for many years, without anyone stepping up to fix it.[13]
The problem looks to be the known OSGText coloration problem seen with newer versions of OSG... OSG changed the way that it does the colors and the old OpenGL style is no longer used by OSG... FG simply hasn't kept up with all the changes in OSG... this problem has been around since i came to the project 4 or so years back... none of the devs have had the time to refactor the way FG handles OSGText so here we are... someone will get to it one day... for now the main project uses an older version of OSG with the breaking commit reverted... the PPA does not and neither do most of us building our own binaries..[14]
Linux distros provide decently recent version of OSG, which means, flightgear in distro is currently broken by this. Indeed, if even OSG-3.4.1 is breaking the text animation, look how many distros are distributing flightgear already broken (as linked to the distributed OSG)[15]: https://repology.org/project/openscenegraph/versions
If we go to OSG 3.6, they are broken, plain and simple.[16]
So we need to do some Real Work (TM) to support Canvas with osgText 3.6 and upwards, unfortunately.[17]
Currently, osgText doesn't work under the ALS pipeline. That's a big issue for someone wanting to fly an airliner that uses osgText (to the point of being practically unflyable). It's not a breaking change for other developers, but it is for someone wanting to actually use the simulator. That should be the most important criteria when judging if something goes on next.[18]
Some aircraft developers have switched to canvas /because/ of the always white OSGText problem[19]
The OSGText issue is more that the world is moving forwards, and our text-animation element has not. (Shaders required, can’t rely on OpenGL 1.1 built-in lighting)[20]
It’s caused by OSG using a shader to render text now, but this bypasses the built-in emissive/diffuse/ambient material properties, which is what our <text> animation uses to set colours of text.
It should happen ‘everywhere’ that a shader is used to render text - Canvas doesn’t care about it since it sets the text color directly.[21]
The breaking change was introduced with OSG 3.6.5, and was also recently merged into OSG 3.7.[22]
Status
| This section contains out-of-date information
Please help improve this article by updating it. See the note at the top of the page to learn more |
02/2021: The issue with needing an osgText replacement so we can move the windows build to OSG 3.6 has not been forgotten, but neither James nor Stuart are looking at it right now.[23]
For the time being, this has not been addressed: James would be happy to be wrong about that. But hasn't seen any code change which would fix it, and indeed, he does not believe a simple ‘change X to do Y’ fix is actually directly feasible. (Because fundamentally you can’t use two shaders programs together, and osgText now uses shaders)[24]
As to upgrading to OSG 3.6, we'd sooner not do that without a clear way forward about osgText support. It doesn’t need to be perfect but at least some idea of a timeline. If we have a solution we can jump to OSG 3.6.n without hesitation.[25]
Text animations have been updated to use UTF8 internally [26].
We would prefer to keep next in a close-to-release-able state.
So we’d sooner, instead of trying to side-step the issue, we instead actually confront it (and fix it) [...] Basically we all have to pay some price of getting this boring work done, before we can do the ‘fun’ shaders stuff. (Same for HUD and 2D panels)[27].
Background
With OSG 3.6, Aircraft <text> animations will be broken [28] however: that’s the blocking issue stopping us moving to 3.6 everywhere. [29] OSG 3.6 has the text display issue [30], currently osgText doesn't work under the ALS pipeline. [31] one issue that seems to occur in a few places is missing text elements on cockpit devices ... There are issues with osgText not working in OSG > 3.4.2. We need to find a migration solution for osgText that works with Compositor and newer OSG versions [32] ... another option was creating a Canvas for each text animation, but creating a FBO for every small piece of text doesn't sound very performance-friendly. [33] We need to find some way to approximate with the old OSG text did[34]
For the time being, we don’t (often) use OSG 3.6 - looking at the code on master (which became 3.6), there isn’t a single commit we can revert - Robert evidently did a lot of incremental improvements to the text code, adding support for other features and improving the shader support. Then at some point he collected a subset of those changes and back-ported them to 3.4 branch, which is what we revert in the commit on my fork on GitHub.
This means for OSG 3.6 we need to find a solution on our side, which likely means updating aircraft or making some guesses based on the <text> animation element in our XML files, because the set of colouring options supported by the new osgText implementation is more restrictive than what worked before.
(The text animation assumes that setting all of emissive/ambient/diffuse/specular works on an osgText, and it no longer does - for some cases we could map the emissive color to the text color, but this would break some other cases…. and text still won’t receive normal lighting in that case)[35]
There are some ways to fix that, but they might break aircraft compatibility (but that is maybe inevitable for this, to some degree)
The best solution we discussed so far is: render the text to a texture (using the official osgText shaders from 3.6), and then apply that texture as a decal in 3D. This would allow effects to be used, which is what we need to fix the lighting problems. (mostly…) The problem is this means defining a texture quad which is correctly positioned (in terms of Z-offset) above the panel / model the text appears on.
Now, all of the above is /also/ true for osgText in 3.4.0; so the aircraft developers already configured (presumably) a position + offset which gives acceptable tradeoff of ‘appearing coplanar’ and ‘avoiding Z-fighting’. The question is whether we can mimic what old osgText did, close enough that most aircraft work unmodified. And we won’t know that until someone creates the replacement code.[36]
Initial Findings
- this bug seems to be caused by a change added *intentionally* to OSG between 3.4.0 and 3.4.1. The change is intended to fix some *other* issue with text rendering in OSG, but we don’t seem to see those in FG
- it’s not related to platform or driver, and indeed the underlying bug is probably in our code, not OSG
- The FG version is unrelated to this, it just depends which OSG version you’re using it. Some of our releases were built with OSG 3.4.0, some with 3.4.1, but we can build ‘any’ FG release with ‘any’ FG version.
- This doesn’t have anything to do with HiDPi support in PUI, we didn’t change the OSG version requirement so far
- James would much prefer to fix this by finding *our* bug, but would probably need some help doing that - because while we can patch the OSG we use for our binary builds, we have no control over Linux distros shipping OSG 3.4.0 vs 3.4.1
- we definitely shouldn’t make changes to aircraft because of this bug
- the fix is for people to jump into the OSG and rendering code, not bypass this in XML[37]
Investigation
| Note As a workaround, using a canvas for this is actually not that bad - it eats some memory equal to the size of the canvas (so make it as small as possible) but in terms of performance, it will end up about the same. [38] To be clear it’s 100% a bug that can be fixed, just that given the effort to fix it, I’d recommend going with the canvas solution instead.[39]
For the nitty-gritty details, see: https://sourceforge.net/p/flightgear/mailman/message/36240313/ |
At first glance looks as if FG would need to override those shaders with custom ones, to replicate the material animation used in the fixed pipeline.[40]
The osgText is probably (or at least, possibly, there’s a few other options besides) being ignored by the effect system, this is fixable but fairly unpleasant work. [41]
Back in 2017, the issue seemed that we now set a shader program on the font state-set, which is tripping up something (the effects system or otherwise), but we need to investigate further to fix anything. [42]
James tested current OSG (3.7.x) and we still have the same issue to fix on our side.
The issue is using a <material> animation to set the colour of the text no longer works, because osgText no longer uses the OpenGL materials, and hence ignores the material properties in the StateSet. The only control we have over the colour of the text is the osgText::Text::setColor method - which is what Canvas uses of course.[43]
The issue seems many aircraft could be changed to do something simpler (use decal text with no lighting), or use Canvas, but masking a fully generic replacement for what <text> used to do is quite difficult - for correctness it should use the full lighting model / ALS / etc so that painted text on a panel / tail / etc is lit correctly.
That’s completely doable but means hooking an effect into the <text> element, which is certainly some work to get correct, since the shader also needs to interact correctly with the osgText source texture (containing the glyphs and offsets)[44]
Fernando has been investigating this for a while now and it has proven to be a lot less trivial that he initially thought. His first approach was to ust transform the osgTexts into EffectGeodes, but this isn't so elegant as the general rules of effects don't apply to animations (we can't use <inherits-from>, model-default doesn't get applied automatically as it isn't considered a model, etc).
Richard said on Discord that another option was creating a Canvas for each text animation, but creating a FBO for every small piece of text doesn't sound very performance-friendly. This is also worsened by the fact that canvas cameras are scene-graph level cameras and are being rendered multiple times (once per viewer slave camera).[45]
For details, see Post_FlightGear_2020.2_LTS_changes#Canvas.
If someone wants to hack around in this area, you’re going to want to copy the ‘generate the quad’ part of osgText into a new simgear/Scene, (to get the sizing / position right) which internally renders a texture created using an FBO-camera via ‘new’ osgText. The issue is going to be deciding what color to pass to new osgText; where an <emissive> value is supplied that should work directly, the problem case is things like logo/callsign text on exteriors and panels. For that, using the diffuse color will probably work as a starting point, and then apply the standard aircraft model effect. and … show how bad it looks. [46]
The ‘right’ way is to to render osgText into a texture / FBO, and then apply that to the model using an Effect. Then effects will work as normal on text.
This could be hacked using the Canvas but that might be:
- wasteful of resources (for rarely-changing text)
- create a lot of property tree noise
- hard to make it backwards compatible (which is very important)
So we might want a little text > a texture helper (implemented using osgText) which then locates the resulting texture quad at *exactly* the same location, as the quad osgText ‘would have built anyway’. And then applies an effect to it (which previously, was not done, so we need a a default effect close to the old fixed-function)[47]
It would be very good to add the generated text-textures into a texture atlas, if that can be done safely. And it’s also why we don’t want an entry under canvas/ for each one. So we would have a little manager to track in-use areas of the atlases, map an animation to UVs within the atlas, etc. Definitely an optimisation, but probably a friendly one for cockpits with many text elements. (Which I think is the 737, the 747, the 777….)[48]
Ideas
James wondered if we could do a hack in <animation> XML parsing to at least support emissive use - then we have an easy update route for those people, people doing panel legends would need to use canvas text manually?[49]
Mapping to Canvas
For some purposes (pre-canvas HUDs or displays) text animations would have to be emissive, likewise for panel backlighting.
For other purposes - basically panel legends etc. they would be ambient/diffuse.
Since osgText animations also don't play nice with our whole effect framework, might there a way to internally map the animation to an equivalent canvas solution? We might be better off in a number of ways with that.[50]
Of course, emissive for panel legends would be totally wrong.
The problem is supporting emissive would be doable as a few-lines tweak and quite safe. Supporting ambient/diffuse will require the same solution as ‘making the text work with effects’, i.e a custom text shader I thin, and I don’t know if we could do that (or integrate it with the effects framework)
As for mapping to Canvas, James also is not sure how that would work - Canvas Text is also ‘just osgText’ but it seems works for Canvas because the output texture is applied to the cockpit and then effects are applied on top? Doing a safe transformation of osgText-animation to Canvas would mean working out the required canvas extent and somehow mapping it to the correct panel location (and UVs), which he’d be nervous about both the reliability and complexity of. [51]
There have been some recurring discussions to fix/port or rather retarget the osgtext animation so that the core implementation uses a canvas FBO fallback internally. So far, the main concern here has been that allocating one FBO per animation (panel legend) would be rather wasteful. Another idea disussed previously was to use a single Canvas FBO to serve as a cache texture and allocate all strings inside that, while registering an effect/shader that merely gets a handle to the canvas SUB-TEXTURE - which is an approach already used elsewhere (namely the Canvas SymbolCache).[52]
The problem with the text-animations is no one yet did the R&D work to actually decide *what the new approach will even look like*. What we don’t want is aircraft developers hacking up nasty work-arounds in a hurry, which might be very inefficient, but which will then hang around even when a proper solution is developed. At least one proposed work-around is very inefficient, and might cause other failures depending on the user’s GPU and drivers (using one Canvas to replace each text-animation; if we hit an internal limit on FBOs, we will stop arbitrary Canvas updating)
The practical next step which might be feasible, and which we’re waiting for, is for someone to prototype some solutions: for example rendering all <text> to a single internal FBO/texture-atlas and then building textured-quads to hold pieces of that texture in the same place as the old osgText nodes; the quads can then get an Effect applied. There’s variations on that approach, some exploration and R&D is needed for sure. [53]
In mid-2020, Gaétan Allaert reported that he was still working on the replacement of the 2D panel by canvas.[54]
