OSGText Issues: Difference between revisions

From FlightGear wiki
Jump to navigation Jump to search
mNo edit summary
(3 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{Stub}}
{{Stub}}
{{infobox subsystem
|name        = Osgtext  + effects
|started    = 11/2020
|description = Make text animations work correctly with the Effects framework
|status      = discussions
|developers  = {{Usr|Scttgs}} <ref>https://sourceforge.net/p/flightgear/mailman/message/37157550/</ref>
}}




{{Screenshot needed}}


The OSGText animation (README.osgtext) doesn't play nice with shaders.<ref>https://forum.flightgear.org/viewtopic.php?f=87&t=37429&p=376110#p376110</ref>
The OSGText animation (README.osgtext) doesn't play nice with shaders.<ref>https://forum.flightgear.org/viewtopic.php?f=87&t=37429&p=376110#p376110</ref>
Another way to describe the bug is ‘make text animations work correctly with the Effects framework’. Because osgtext always uses a shader (now), and shaders cannot be combined, we have to find a smarter way to do that. But the fact that osgtext didn’t work correctly with Effects has always been there. So there is no upstream fix possible: we have to change what we’re doing on our side, but that will risk some aircraft compat issues. The patch we’re applying is to restore things to how they looked before, but it was still broken, just less obviously.<ref>https://sourceforge.net/p/flightgear/mailman/message/37150030/</ref>


It’s a fairly large number of aircraft: the text animation has been around for a long time, prior to Canvas for example.  
It’s a fairly large number of aircraft: the text animation has been around for a long time, prior to Canvas for example.  
The problem is not us keeping 3.4.x around : it would be nice to switch to 3.6 everywhere, but we support both today.
The problem is not us keeping 3.4.x around : it would be nice to switch to 3.6 everywhere, but we support both today.
Some cockpits have 50 of these (each MCP window, each radio numerical field, etc, etc). There’s at least one CDU impl where each line is two text items. (One for big, one for for small)<ref>https://sourceforge.net/p/flightgear/mailman/message/37157742/</ref>


The text issue has existed with OSG 3.6 for many years, without anyone stepping up to fix it.<ref>https://sourceforge.net/p/flightgear/mailman/message/37149210/</ref>
The text issue has existed with OSG 3.6 for many years, without anyone stepping up to fix it.<ref>https://sourceforge.net/p/flightgear/mailman/message/37149210/</ref>
Line 79: Line 90:


Richard said on Discord that another option was creating a Canvas for each text animation, but creating a FBO for every small piece of text doesn't sound very performance-friendly. This is also worsened by the fact that canvas cameras are scene-graph level cameras and are being rendered multiple times (once per viewer slave camera).<ref>https://sourceforge.net/p/flightgear/mailman/message/37042457/</ref>
Richard said on Discord that another option was creating a Canvas for each text animation, but creating a FBO for every small piece of text doesn't sound very performance-friendly. This is also worsened by the fact that canvas cameras are scene-graph level cameras and are being rendered multiple times (once per viewer slave camera).<ref>https://sourceforge.net/p/flightgear/mailman/message/37042457/</ref>
If someone wants to hack around in this area, you’re going to want to copy the ‘generate the quad’ part of osgText into a new simgear/Scene, (to get the sizing / position right) which internally renders a texture created using an FBO-camera via ‘new’ osgText. The issue is going to be deciding what color to pass to new osgText; where an <emissive> value is supplied that should work directly, the problem case is things like logo/callsign text on exteriors and panels. For that, using the diffuse color will probably work as a starting point, and then apply the standard aircraft model effect. and … show how bad it looks. <ref>https://sourceforge.net/p/flightgear/mailman/message/37149302/</ref>
The ‘right’ way is to to render osgText into a texture / FBO, and then apply that to the model using an Effect. Then effects will work as normal on text.
This could be hacked using the Canvas but I think that will be:
* wasteful of resources (for rarely-changing text)
* create a lot of property tree noise
* hard to make it backwards compatible (which is very important)
So I think we want a little <code>text > a texture</code> helper (implemented using osgText) which then locates the resulting texture quad at *exactly* the same location, as the quad osgText ‘would have built anyway’. And then applies an effect to it (which previously, was not done, so we need a a default effect close to the old fixed-function)<ref>https://sourceforge.net/p/flightgear/mailman/message/37157687/</ref>
It would be very good to add the generated text-textures into an atlas, if that can be done safely. And it’s also why we don’t want an entry under canvas/ for each one. So we would have a little manager to track in-use areas of the atlases, map an animation to UVs within the atlas, etc. Definitely an optimisation, but probably a friendly one for cockpits with many text elements. (Which I think is the 737, the 747, the 777….)<ref>https://sourceforge.net/p/flightgear/mailman/message/37157742/</ref>


== Ideas ==
== Ideas ==

Revision as of 17:38, 20 November 2020

This article is a stub. You can help the wiki by expanding it.
Osgtext + effects
Started in 11/2020
Description Make text animations work correctly with the Effects framework
Contributor(s) Scttgs [1]
Status discussions


The OSGText animation (README.osgtext) doesn't play nice with shaders.[2]

Another way to describe the bug is ‘make text animations work correctly with the Effects framework’. Because osgtext always uses a shader (now), and shaders cannot be combined, we have to find a smarter way to do that. But the fact that osgtext didn’t work correctly with Effects has always been there. So there is no upstream fix possible: we have to change what we’re doing on our side, but that will risk some aircraft compat issues. The patch we’re applying is to restore things to how they looked before, but it was still broken, just less obviously.[3]

It’s a fairly large number of aircraft: the text animation has been around for a long time, prior to Canvas for example. The problem is not us keeping 3.4.x around : it would be nice to switch to 3.6 everywhere, but we support both today.

Some cockpits have 50 of these (each MCP window, each radio numerical field, etc, etc). There’s at least one CDU impl where each line is two text items. (One for big, one for for small)[4]

The text issue has existed with OSG 3.6 for many years, without anyone stepping up to fix it.[5]

The problem looks to be the known OSGText coloration problem seen with newer versions of OSG... OSG changed the way that it does the colors and the old OpenGL style is no longer used by OSG... FG simply hasn't kept up with all the changes in OSG... this problem has been around since i came to the project 4 or so years back... none of the devs have had the time to refactor the way FG handles OSGText so here we are... someone will get to it one day... for now the main project uses an older version of OSG with the breaking commit reverted... the PPA does not and neither do most of us building our own binaries..[6]

Linux distros provide decently recent version of OSG, which means, flightgear in distro is currently broken by this. Indeed, if even OSG-3.4.1 is breaking the text animation, look how many distros are distributing flightgear already broken (as linked to the distributed OSG)[7]: https://repology.org/project/openscenegraph/versions

If we go to OSG 3.6, they are broken, plain and simple.[8]

So we need to do some Real Work (TM) to support Canvas with osgText 3.6 and upwards, unfortunately.[9]

Currently, osgText doesn't work under the ALS pipeline. That's a big issue for someone wanting to fly an airliner that uses osgText (to the point of being practically unflyable). It's not a breaking change for other developers, but it is for someone wanting to actually use the simulator. That should be the most important criteria when judging if something goes on next.[10]


Some aircraft developers have switched to canvas /because/ of the always white OSGText problem[11]

The OSGText issue is more that the world is moving forwards, and our text-animation element has not. (Shaders required, can’t rely on OpenGL 1.1 built-in lighting)[12]

It’s caused by OSG using a shader to render text now, but this bypasses the built-in emissive/diffuse/ambient material properties, which is what our <text> animation uses to set colours of text.

It should happen ‘everywhere’ that a shader is used to render text - Canvas doesn’t care about it since it sets the text color directly.[13]

The breaking change was introduced with OSG 3.6.5, and was also recently merged into OSG 3.7.[14]

Background

With OSG 3.6, Aircraft <text> animations will be broken [15] however: that’s the blocking issue stopping us moving to 3.6 everywhere. [16] OSG 3.6 has the text display issue [17], currently osgText doesn't work under the ALS pipeline. [18] one issue that seems to occur in a few places is missing text elements on cockpit devices ... There are issues with osgText not working in OSG > 3.4.2. We need to find a migration solution for osgText that works with Compositor and newer OSG versions [19] ... another option was creating a Canvas for each text animation, but creating a FBO for every small piece of text doesn't sound very performance-friendly. [20] We need to find some way to approximate with the old OSG text did[21]

For the time being, we don’t (often) use OSG 3.6 - looking at the code on master (which became 3.6), there isn’t a single commit we can revert - Robert evidently did a lot of incremental improvements to the text code, adding support for other features and improving the shader support. Then at some point he collected a subset of those changes and back-ported them to 3.4 branch, which is what we revert in the commit on my fork on GitHub.

This means for OSG 3.6 we need to find a solution on our side, which likely means updating aircraft or making some guesses based on the <text> animation element in our XML files, because the set of colouring options supported by the new osgText implementation is more restrictive than what worked before.

(The text animation assumes that setting all of emissive/ambient/diffuse/specular works on an osgText, and it no longer does - for some cases we could map the emissive color to the text color, but this would break some other cases…. and text still won’t receive normal lighting in that case)[22]

There are some ways to fix that, but they might break aircraft compatibility (but that is maybe inevitable for this, to some degree)

The best solution we discussed so far is: render the text to a texture (using the official osgText shaders from 3.6), and then apply that texture as a decal in 3D. This would allow effects to be used, which is what we need to fix the lighting problems. (mostly…) The problem is this means defining a texture quad which is correctly positioned (in terms of Z-offset) above the panel / model the text appears on.

Now, all of the above is /also/ true for osgText in 3.4.0; so the aircraft developers already configured (presumably) a position + offset which gives acceptable tradeoff of ‘appearing coplanar’ and ‘avoiding Z-fighting’. The question is whether we can mimic what old osgText did, close enough that most aircraft work unmodified. And we won’t know that until someone creates the replacement code.[23]


Initial Findings

  • this bug seems to be caused by a change added *intentionally* to OSG between 3.4.0 and 3.4.1. The change is intended to fix some *other* issue with text rendering in OSG, but we don’t seem to see those in FG
  • it’s not related to platform or driver, and indeed the underlying bug is probably in our code, not OSG
  • The FG version is unrelated to this, it just depends which OSG version you’re using it. Some of our releases were built with OSG 3.4.0, some with 3.4.1, but we can build ‘any’ FG release with ‘any’ FG version.
  • This doesn’t have anything to do with HiDPi support in PUI, we didn’t change the OSG version requirement so far
  • James would much prefer to fix this by finding *our* bug, but would probably need some help doing that - because while we can patch the OSG we use for our binary builds, we have no control over Linux distros shipping OSG 3.4.0 vs 3.4.1
  • we definitely shouldn’t make changes to aircraft because of this bug
  • the fix is for people to jump into the OSG and rendering code, not bypass this in XML[24]

Investigation

Note  As a workaround, using a canvas for this is actually not that bad - it eats some memory equal to the size of the canvas (so make it as small as possible) but in terms of performance, it will end up about the same. [25] To be clear it’s 100% a bug that can be fixed, just that given the effort to fix it, I’d recommend going with the canvas solution instead.[26]

For the nitty-gritty details, see: https://sourceforge.net/p/flightgear/mailman/message/36240313/

At first glance looks as if FG would need to override those shaders with custom ones, to replicate the material animation used in the fixed pipeline.[27]

The osgText is probably (or at least, possibly, there’s a few other options besides) being ignored by the effect system, this is fixable but fairly unpleasant work. [28]

Back in 2017, the issue seemed that we now set a shader program on the font state-set, which is tripping up something (the effects system or otherwise), but we need to investigate further to fix anything. [29]

James tested current OSG (3.7.x) and we still have the same issue to fix on our side.

The issue is using a <material> animation to set the colour of the text no longer works, because osgText no longer uses the OpenGL materials, and hence ignores the material properties in the StateSet. The only control we have over the colour of the text is the osgText::Text::setColor method - which is what Canvas uses of course.[30]

The issue seems many aircraft could be changed to do something simpler (use decal text with no lighting), or use Canvas, but masking a fully generic replacement for what <text> used to do is quite difficult - for correctness it should use the full lighting model / ALS / etc so that painted text on a panel / tail / etc is lit correctly.

That’s completely doable but means hooking an effect into the <text> element, which is certainly some work to get correct, since the shader also needs to interact correctly with the osgText source texture (containing the glyphs and offsets)[31]


Fernando has been investigating this for a while now and it has proven to be a lot less trivial that he initially thought. His first approach was to ust transform the osgTexts into EffectGeodes, but this isn't so elegant as the general rules of effects don't apply to animations (we can't use <inherits-from>, model-default doesn't get applied automatically as it isn't considered a model, etc).

Richard said on Discord that another option was creating a Canvas for each text animation, but creating a FBO for every small piece of text doesn't sound very performance-friendly. This is also worsened by the fact that canvas cameras are scene-graph level cameras and are being rendered multiple times (once per viewer slave camera).[32]

If someone wants to hack around in this area, you’re going to want to copy the ‘generate the quad’ part of osgText into a new simgear/Scene, (to get the sizing / position right) which internally renders a texture created using an FBO-camera via ‘new’ osgText. The issue is going to be deciding what color to pass to new osgText; where an <emissive> value is supplied that should work directly, the problem case is things like logo/callsign text on exteriors and panels. For that, using the diffuse color will probably work as a starting point, and then apply the standard aircraft model effect. and … show how bad it looks. [33]


The ‘right’ way is to to render osgText into a texture / FBO, and then apply that to the model using an Effect. Then effects will work as normal on text.

This could be hacked using the Canvas but I think that will be:

  • wasteful of resources (for rarely-changing text)
  • create a lot of property tree noise
  • hard to make it backwards compatible (which is very important)

So I think we want a little text > a texture helper (implemented using osgText) which then locates the resulting texture quad at *exactly* the same location, as the quad osgText ‘would have built anyway’. And then applies an effect to it (which previously, was not done, so we need a a default effect close to the old fixed-function)[34]

It would be very good to add the generated text-textures into an atlas, if that can be done safely. And it’s also why we don’t want an entry under canvas/ for each one. So we would have a little manager to track in-use areas of the atlases, map an animation to UVs within the atlas, etc. Definitely an optimisation, but probably a friendly one for cockpits with many text elements. (Which I think is the 737, the 747, the 777….)[35]


Ideas

James wondered if we could do a hack in <animation> XML parsing to at least support emissive use - then we have an easy update route for those people, people doing panel legends would need to use canvas text manually?[36]

Mapping to Canvas

For some purposes (pre-canvas HUDs or displays) text animations would have to be emissive, likewise for panel backlighting.

For other purposes - basically panel legends etc. they would be ambient/diffuse.

Since osgText animations also don't play nice with our whole effect framework, might there a way to internally map the animation to an equivalent canvas solution? We might be better off in a number of ways with that.[37]

Of course, emissive for panel legends would be totally wrong.

The problem is supporting emissive would be doable as a few-lines tweak and quite safe. Supporting ambient/diffuse will require the same solution as ‘making the text work with effects’, i.e a custom text shader I thin, and I don’t know if we could do that (or integrate it with the effects framework)

As for mapping to Canvas, James also is not sure how that would work - Canvas Text is also ‘just osgText’ but it seems works for Canvas because the output texture is applied to the cockpit and then effects are applied on top? Doing a safe transformation of osgText-animation to Canvas would mean working out the required canvas extent and somehow mapping it to the correct panel location (and UVs), which he’d be nervous about both the reliability and complexity of. [38]

There have been some recurring discussions to fix/port or rather retarget the osgtext animation so that the core implementation uses a canvas FBO fallback internally. So far, the main concern here has been that allocating one FBO per animation (panel legend) would be rather wasteful. Another idea disussed previously was to use a single Canvas FBO to serve as a cache texture and allocate all strings inside that, while registering an effect/shader that merely gets a handle to the canvas SUB-TEXTURE - which is an approach already used elsewhere (namely the Canvas SymbolCache).[39]

The problem with the text-animations is no one yet did the R&D work to actually decide *what the new approach will even look like*. What we don’t want is aircraft developers hacking up nasty work-arounds in a hurry, which might be very inefficient, but which will then hang around even when a proper solution is developed. At least one proposed work-around is very inefficient, and might cause other failures depending on the user’s GPU and drivers (using one Canvas to replace each text-animation; if we hit an internal limit on FBOs, we will stop arbitrary Canvas updating)

The practical next step which might be feasible, and which we’re waiting for, is for someone to prototype some solutions: for example rendering all <text> to a single internal FBO/texture-atlas and then building textured-quads to hold pieces of that texture in the same place as the old osgText nodes; the quads can then get an Effect applied. There’s variations on that approach, some exploration and R&D is needed for sure. [40]

In mid-2020, Gaétan Allaert reported that he was still working on the replacement of the 2D panel by canvas.[41]

References

References
  1. https://sourceforge.net/p/flightgear/mailman/message/37157550/
  2. https://forum.flightgear.org/viewtopic.php?f=87&t=37429&p=376110#p376110
  3. https://sourceforge.net/p/flightgear/mailman/message/37150030/
  4. https://sourceforge.net/p/flightgear/mailman/message/37157742/
  5. https://sourceforge.net/p/flightgear/mailman/message/37149210/
  6. https://forum.flightgear.org/viewtopic.php?p=353286#p353286
  7. https://sourceforge.net/p/flightgear/mailman/message/37149005/
  8. https://sourceforge.net/p/flightgear/mailman/message/37148949/
  9. https://sourceforge.net/p/flightgear/mailman/message/36730965/
  10. https://sourceforge.net/p/flightgear/mailman/message/37042335/
  11. https://sourceforge.net/p/flightgear/mailman/message/36731025/
  12. https://sourceforge.net/p/flightgear/mailman/message/36731031/
  13. https://sourceforge.net/p/flightgear/mailman/message/36731732/
  14. https://sourceforge.net/p/flightgear/mailman/message/36732917/
  15. https://sourceforge.net/p/flightgear/mailman/message/36240313/
  16. https://sourceforge.net/p/flightgear/mailman/message/37147614/
  17. https://sourceforge.net/p/flightgear/mailman/message/37126793/
  18. https://sourceforge.net/p/flightgear/mailman/message/37042335/
  19. https://sourceforge.net/p/flightgear/mailman/message/37047567/
  20. https://sourceforge.net/p/flightgear/mailman/message/37042457/
  21. https://sourceforge.net/p/flightgear/mailman/message/37042476/
  22. https://sourceforge.net/p/flightgear/mailman/message/36703340/
  23. https://sourceforge.net/p/flightgear/mailman/message/37148949/
  24. https://sourceforge.net/p/flightgear/mailman/message/36240313/
  25. https://sourceforge.net/p/flightgear/mailman/message/36113197/
  26. https://sourceforge.net/p/flightgear/mailman/message/36113508/
  27. https://sourceforge.net/p/flightgear/mailman/message/36116856/
  28. https://sourceforge.net/p/flightgear/mailman/message/36113197/
  29. https://sourceforge.net/p/flightgear/mailman/message/36141064/
  30. https://sourceforge.net/p/flightgear/mailman/message/36624316/
  31. https://sourceforge.net/p/flightgear/mailman/message/36740020/
  32. https://sourceforge.net/p/flightgear/mailman/message/37042457/
  33. https://sourceforge.net/p/flightgear/mailman/message/37149302/
  34. https://sourceforge.net/p/flightgear/mailman/message/37157687/
  35. https://sourceforge.net/p/flightgear/mailman/message/37157742/
  36. https://sourceforge.net/p/flightgear/mailman/message/36627559/
  37. https://sourceforge.net/p/flightgear/mailman/message/36624574/
  38. https://sourceforge.net/p/flightgear/mailman/message/36627559/
  39. https://forum.flightgear.org/viewtopic.php?f=87&t=37429&start=75#p376118
  40. https://sourceforge.net/p/flightgear/mailman/message/37149210/
  41. https://sourceforge.net/p/flightgear/mailman/message/37042351/