Canvas news

From FlightGear wiki
Jump to navigation Jump to search


This page is primarily intended to keep track of Canvas related core changes and discussions, i.e. mainly of interest to people who are not able to follow the forum/devel list regularly, but who may still want to stay up to date regarding future Canvas developments (i.e. not including fgaddon).

Everybody is invited to help contribute to maintain this page and provide up-to-date information. Also, quoting/referincing devel/forum list talks is actively encouraged, too - to help provide the corresponding pointers back to the original discussion threads: Instant-Cquotes.

2018

2017

Canvas & Slave Cameras

First screenshot created by Alant, view from the rear cockpit of TSR2, showing the the custom "camera-display" prototyped by F-JJTH, just to the left of the moving map.Known issue: despite flying more or less straight and level, the image camera point can be seen to jump around between screenshots. [1]
Second screenshot created by Alant, view from the rear cockpit of TSR2, showing the the custom "camera-display" prototyped by F-JJTH, just to the left of the moving map.Known issue: despite flying more or less straight and level, the image camera point can be seen to jump around between screenshots. [2]

Former FlightGear core developer F-JJTH shared a patch [3] implementing a proof-of-concept to render an arbitrary slave view to a FBO/RTT using a new dedicated "camera-display" instrument. This would be the kind of functionality required to implement many interesting features that cannot currently be added easily; Rear view mirrors in fighters could become a reality, as well as camera displays in SAR helicopters, outside views in airliners, backing view displays in cars etc.[4]

The patch seems not to break anything else, and does nothing unless the aircraft side code is added. It is therefore reasonably safe to add the patch to your working copy of FG. No git branch expertise is required.[5]

Once the patch is applied and the binary rebuilt, all that is needed is to add the new camera-display to an existing cockpit panel:

<camera-display>
 <name>cd</name>
 <texture></texture>
 <enabled type="bool">true</enabled>
 <view-number>0</view-number>
</camera-display>


For view configuratoin, it's using the standard view-manager interface, so you primarily need to set the view-number property to change the view (untested, feedback is just based on looking at the patch in question). Specifically, when running the patched fgfs binary, look for: /instrumentation/camera-display and the view-number property underneath[6]

The current state is... well, it "works" ;-) However there are some bugs:

  • The image jumps all over the place when the aircraft is moving.
  • The colours are wrong.[7]


  1. The colours are wrong with Rembrandt. It looks as if the wrong buffer is being used.
  2. Jumping around is the same with and without Rembrandt.[8]

The jumping around effect seems to be in all views. The control tower view jumps back and forward, showing the whole tower at times.[9]


before looking at some of our most advanced use-cases, it's best to tinker with a really simple scenario, to see how well this works, and what needs to be polished.[10]


if anybody is interested in working out what is missing or what can be improved, we should review working features doing similar things, e.g. the screenshot/streaming handlers or the camera setup routines used by CameraGroup.cxx itself.[11]

The patch itself seems overlapping with Zan's original "new-cameras" effort, as mentioned at: Canvas Development#Supporting Cameras If this patch should work well enough, it would probably make sense to integrate it with the Canvas system to come up with a dedicated "camera" element for the Canvas system, that would be configurable using the format/properties used by the view manager: Howto:Configure views in FlightGear At that point, it would actually be a terrific idea to review Zan's original patches and see if any of the features there could/should be integrated with the new Canvas::Camera element - like you say, once this is properly working, "the sky is the limit" (or not even that, think shuttle cams)[12]

Thus, the patch needs some TLC - however, it's definitely a very solid foundation, and very compelling as a prototype, especially given that the patch is pretty lightweight.

Much of the OSG machinery to set up the FBO/RTT (render to texture) is overlapping with Canvas functionality, so we could do away with that by using a dedicated Canvas element (which may fix some issues, e.g. colors being off).

Besides, if/when we actually use the approach used by Torsten's screenshot handler routines (mongoose/Phi), the "jumping" may also be gone (just a guess). At this point, we need to get more people involved who are able to patch/rebuild sg/fg from source and test these patches, so that we can come up with a team of folks to help with testing/developing. Hooray can certainly help with the Canvas side of this, which means that the FBO/RTT setup routines can be simplified, and he can help turn the whole thing into a dedicated Canvas "camera" element.

[13]

G1000 & MapStructure improvements

As of 09/2017, Stuart started making some tentative steps towards a full Garmin G1000 simulation, he started with the excellent xkv1000 as a starting point, particularly for the PFD which very closely mirrors the G1000.

For the MFD he's planning to use the Canvas MapStructure layers (obviously). One area he is going to have to develop is tiled mapping, which the MapStructure in fgdata doesn't currently support as a layer.

Stuart's thoughts are:

  1. Create an OverlayLayer layer with a controller that is analogous to the SymbolLayer, but likely with a simple update method rather than SearchCmd, and no .symbol equivalent
  2. Convert the tiled map example from the wiki, but also other layers such as a compass rose, airport diagram, lat/lon grid.

Stuart is currently undecided on how to handle the projection - whether to have the layer itself have to interpret the centerpoint of the map and scale, and then make a projection itself, or to provide it from the Map. The latter would undoubtably be cleaner, but for something like a tiled map he suspects that the management will need to be quite low down the stack.[14]

Property I/O

As a first step towards developing a G1000, Stuart Buchanan has been looking at Nasal property read/write performance, as he understands that this is a bottleneck for developing complex Canvas applications.[15]

For well-organized code (aka writing only when needed, organizing the displays in suitable groups, staggered updates...), it's going to be fast enough. For a straightforward approach to complicated displays probably not. A thousand properties sounds a lot, but every update in translation is already 2 properties, combined with a visible flag and color information you're down to 166 elements you can update already. If each of them is driven by reading a value from the tree you're losing even more. Generally, if there is a way of making either method faster, it would affect performance in a positive way all over Nasal scripting because currently property I/O is in my view the largest bottleneck (especially of canvas).[16]


low-hanging fruit in the canvas API:

  • add more calls like updateText (updateTranslation, updateRotation,...) which check whether the property to be written has changed at all against a Nasal record before actually doing property I/O
  • make the updateXXX methods store the node reference and use that rather than construct it afresh

If the test suite numbers hold up in reality, this ought to give a factor 2 to <few hundred> (dependent on how often the argument actually has to be written - at least Thorsten was able to write 50 million numbers into a Nasal array much faster than 50.000 numbers into properties).[17]

it’s easy to add native helpers to Canvas to set some properties from code directly, rather than using the canvas.nas wrappers which then use props.nas internally. This could bypass a whole ton of Nasal if that’s the issue. Of course this code will still have to update properties from C++, so we should rather do the analysis first to see where the slowness occurs, since this could either be a very big win, or completely pointless based on that measurement.[18]

Moving map/RNAV discussion

The airspace system is in the process of changing drastically[19]

and there's an interesting discussion that was started by David Megginson regarding implementing G1000-type systems on the mailing list: https://sourceforge.net/p/flightgear/mailman/flightgear-devel/thread/CAKor_TGugT%2Be%3DQhdNBqWq3Pn%2BCEq6iLZsgaB7HmiU%2BBoDK0vxQ%40mail.gmail.com/#msg35924395[20]


When you're flying an IFR approach with a GPS navigator, you can use RNAV for the transitions, but need to switch to the ground ILS transmitter for the actual approach (they both use the same CDI on many planes, so it's easy to miss). The GTN 650 that I'm installing in my plane will switch CDI mode automatically under certain conditions, but it's not guaranteed. In the past, people could practice the hell out of these kinds of details in FlightGear (tuning, twisting, and identifying all the navaids, adding crosswind corrections, etc) so that they were habits when I flew them for real, but we can't practice RNAV procedures like this (or many others).[21]


this is something we need to address, as our GA aircraft will otherwise become more and more old-fashioned.[22]

FlightGear will still be great for people who want to practice the mechanical parts of flying (e.g. crosswind wheel landings in a Cub), but will slip further and further behind for people who want to use it for real IFR practice.[23]


zakh on the forum has been developing a Garmin G1000 clone called thekv1000: https://forum.flightgear.org/viewtopic.php?f=14&t=32056 [24]

the G1000 has multiple pages of data with data entry for flightplans etc. Stuart is sure there is a lot that has been learned by the Shuttle team that can be passed on as "best practise". The Route Manager should be able to handle most of the "magenta line" tasks, but it may be that the more complicated routing such as the RF approach, fly-by vs fly-over requires some new autopilot coding as you describe.[25]

The MFD framework that is used on the Shuttle is more than capable of supporting a most of the requirements; Canvas is capable of rendering the displays. So it's possible to do this[26]


A number of contributors expressed concerns regarding the performance of the Canvas system when it comes to implementing sophisticated avoinics like G1000-style systems. However, the fact that there are people who code canvas like there's no tomorrow and update 10 elements where an update of the group position would do the trick at a fraction of the cost doesn't mean that canvas or Nasal are slow - it means inefficient code is slow, and that's just as true for C++. Nasal as such is fast, and compared with the cost of rendering the terrain, rendering a canvas display is fast on the GPU (you can test by rendering but not updating a canvas display), and unless you're doing something very inefficient, calculating a display but not writing it to the property tree is fast (you can test this by disabling the write commands). It's the property I/O which you need to structure well, and then canvas will run fast. And of course the complexity of your underlying simulation costs - if you run a thermal model underneath to get real temperature reading, it costs more than inventing the numbers. But that has nothing to do with canvas.[27]

Structured in a reasonable way (i.e. minimiing property I/O in update cycles, avoiding canvas pitfalls which trigger high performance needs etc. ) canvas is pretty fast (kind of reminds me of the old 'Nasal is slow' discussions... it's still the same things which are slow, except canvas hides it behind some abstraction - understand property I/O, and it'll run fast)[28]

The G1000 is a fairly significant piece of computer hardware that we're going to emulate. It's not going to be "free" particularly for those on older hardware that's already struggling. However, hopefully we can offload a chunk of the logic (route management, autopilot/flight director) to the core, and do things like offline generation of terrain maps to minimie the impact.[29]


For starters, Stuart is hoping to to optimize the property access and become more familiar with the Canvas C++ code so that he can help bridge the cap between core and Canvas client developers. He thinks James has some thoughts on the Canvas C++, so he'll want to coordinate with him as well.[30]

One of the ideas raised by David is, should there maybe be a separate project to build an embeddable GPS navigator with different faces/UIs (and a default standalone front end)? As others have mentioned, a lot of the backend logic is the same, like doing a gradual transition from RNAV to Localiser on intercept to avoid a sudden jolt in the A/P, or calculating a turn past a fly-by waypoint. Again, that would be a way to join forces with other Sim dev communities. [31]

if we wanted to build a tablet-based emulator ourselves, we could at least join forces with the FSX and X-Plane communities to do it to grow up the pool of talent.[32]

Groups and Colors

Thorsten learned something rather important about canvas: After doing canvas a while, most of us love the concept to suitably group things and then translate or hide the whole group, which is a very performance-friendly way to get the job done compared with moving all elements.

Thorsten had the situation that the Shuttle HUD has several de-clutter levels which are progressively hidden as the de-clutter function is used. So, he introduced dc0 to dc3 as the groups and operate with selective setVisible() on them according to chosen de-clutter level.

Doing brightness adjustment of the HUD by setting color on the group (specifically the alpha channel).

Unlike translations, rotations or visible statements acting on the top level of the group, this one seems to be passed recursively down the child elements, leading to a huge property I/O.

Anyway, if you know what it does, it's a very useful function still - just it can't be used in a performance-critical place, and it should not be confused with performance-friendly group operations.[33]


Canvas GUI

James is planning a bit of work in the Canvas GUI/widgets area but in the medium term (mid 2017), he wouldn't recommend making lots of extensions to the current Canvas GUI toolkit because he wants to give the whole Canvas-GUI architecture a review and sanity-check first. For the time being, he has no idea if this will be compatible or incompatible with the existing GUI controls, or even be successful. [34]

Hhis intuition is the current style system is overkill, but that the layout code will need some enhancements. So he would avoid both those areas for now. Adding missing widget types is probably safe, but if you propose them here we can discuss their API to hopefully keep it stable even if the implementation changes.[35]

Skia talks

James announced he wants to replace the vector drawing routines with another package which was designed with SIMD in mind[36]


Blend2D is quite amazing for a technical point of view: it compiles native CPU code (including SIMD) for every draw operation using a JIT compiler.[37]


The idea is to use Skia (from Chrome), actually, since it has software and hardware backends. This would allow switching based on which gives better performance on a given machine, and easy porting to other devices, such as Android. The remote-canvas code is a proof-of-concept for this, the aim would be to use the same protocol to render the canvases in different threads.[38]

After reviewing Skia (including reading the sources) and it seems like a very promising fit, albeit with the caveat that it is kind of huge. Blend2D looks pretty good but less mature than Skia, and most importantly, lacks a hardware renderer which James thinks is critical to support devices such as the Raspberry Pi and cheap Android tablets, which James expects to be a very good target for the remote canvas (an RPi driving a wide-screen panel via HDMI could do both the PFD and ND of a Boeing/Airbus setup, but that’s 1920 x 1080 pixels which is too much to fill in software at 60H) Anyway, at this point the code is small enough that experimenting with one renderer or another (Cairo also) is probably a weekend’s hacking, if anyone cared to try. James is currently working on hardware acceleration using QtQuick, since we already have Qt as a dependency.[39]

2016

SIMD

1rightarrow.png See SIMD Support for the main article about this subject.

Erik is experimenting with streamlined Canvas rendering using Single Instruction Multiple Data (SIMD) instructions which is able to calculate on 4 floating poit variables simultaneously (e.g 3 or 4 value vectors). For the F-16 map this gives a speedup from 12+ seconds delay rendering at the first select down to 6 seconds delay.[40] [41]


A few users have reported a noticeable increase in framerate on their computers, e.g. on some desktop the framerate has doubled[42].


SSE and NEON work best when considered in the design stage. Adding it afterwards is always suboptimal. Thus, Erik might still update it in a place or two but this work isn't intended to detract from adding Skia.[43]

Porting the legacy HUD

James is planning to port the C++ HUD to use Canvas for its rendering soon (next few months), but he plans to keep XML compatibility (so aircraft don't see any change) and will keep the placement logic relative to the view, for the same reason. [44]

2D Panels

James has done most of the work to port the 2D panels to use Canvas, and will do the C++ HUD next, and hence is creating Canvas elements bypassing the Nasal API.[45]

Skia backend

James is also currently looking at making a Skia backend for the Canvas (to replace Shiva, making a solution using Skia (https://skia.org) as the backend, because this would give good performance via its OpenGL backend, make people who don’t want to use Qt happier, and could also replace the OSG rendering of the Canvas within FlightGear itself, allowing us to ditch ShivaVG (which is unmaintained) and probably improve performance substantially. (Skia can do SSE3/NEON optimised software rendering in a background thread, so Canvas wouldn’t eat into our OSG / triangle budget)[46]), and Skia already has quite intensive SSE/NEON optimisations, in addition to an optimised OpenGL render which he expects to be substantially more performant than the current OSG Node/Drawable hierarchy.[47]

References

References
  1. Alant  (Aug 26th, 2017).  Re: Gear view in cockpit computer .
  2. Alant  (Aug 26th, 2017).  Re: Gear view in cockpit computer .
  3. Hooray  (Aug 26th, 2017).  Re: Gear view in cockpit computer .
  4. Johan G  (Aug 26th, 2017).  Re: Gear view in cockpit computer .
  5. Alant  (Aug 26th, 2017).  Re: Gear view in cockpit computer .
  6. Hooray  (Aug 26th, 2017).  Re: Gear view in cockpit computer .
  7. Alant  (Aug 26th, 2017).  Re: Gear view in cockpit computer .
  8. Alant  (Aug 26th, 2017).  Re: Gear view in cockpit computer .
  9. Alant  (Aug 26th, 2017).  Re: Gear view in cockpit computer .
  10. Hooray  (Aug 26th, 2017).  Re: Gear view in cockpit computer .
  11. Hooray  (Aug 26th, 2017).  Re: Gear view in cockpit computer .
  12. Hooray  (Aug 26th, 2017).  Re: Gear view in cockpit computer .
  13. Hooray  (Aug 26th, 2017).  Re: Gear view in cockpit computer .
  14. stuart  (Aug 21st, 2017).  Canvas G1000 .
  15. Stuart Buchanan  (Jul 31st, 2017).  [Flightgear-devel] Nasal property lookup performance .
  16. Thorsten Renk  (Aug 1st, 2017).  Re: [Flightgear-devel] Nasal property lookup performance .
  17. Thorsten Renk  (Aug 1st, 2017).  Re: [Flightgear-devel] Nasal property lookup performance .
  18. James Turner  (Aug 1st, 2017).  Re: [Flightgear-devel] Nasal property lookup performance .
  19. David Megginson  (Jul 3rd, 2017).  [Flightgear-devel] RFD: FlightGear and the changing state of air navigation .
  20. stuart  (Jul 4th, 2017).  Re: zkv1000 reborn .
  21. David Megginson  (Jul 4th, 2017).  Re: [Flightgear-devel] RFD: FlightGear and the changing state of air navigation .
  22. Stuart Buchanan  (Jul 3rd, 2017).  Re: [Flightgear-devel] RFD: FlightGear and the changing state of air navigation .
  23. David Megginson  (Jul 4th, 2017).  Re: [Flightgear-devel] RFD: FlightGear and the changing state of air navigation .
  24. Stuart Buchanan  (Jul 3rd, 2017).  Re: [Flightgear-devel] RFD: FlightGear and the changing state of air navigation .
  25. Stuart Buchanan  (Jul 3rd, 2017).  Re: [Flightgear-devel] RFD: FlightGear and the changing state of air navigation .
  26. Richard Harrison  (Jul 4th, 2017).  Re: [Flightgear-devel] RFD: FlightGear and the changing state of air navigation .
  27. Thorsten Renk  (Jul 4th, 2017).  Re: [Flightgear-devel] RFD: FlightGear and the changing state of air navigation .
  28. Thorsten Renk  (Jul 3rd, 2017).  Re: [Flightgear-devel] RFD: FlightGear and the changing state of air navigation .
  29. Stuart Buchanan  (Jul 4th, 2017).  Re: [Flightgear-devel] RFD: FlightGear and the changing state of air navigation .
  30. stuart  (Jul 6th, 2017).  Re: zkv1000 reborn .
  31. David Megginson  (Jul 5th, 2017).  Re: [Flightgear-devel] RFD: FlightGear and the changing state of air navigation .
  32. David Megginson  (Jul 4th, 2017).  Re: [Flightgear-devel] RFD: FlightGear and the changing state of air navigation .
  33. Thorsten  (Mar 16th, 2017).  Color and groups - beware! .
  34. James Turner  (Jan 8th, 2017).  Re: [Flightgear-devel] fgplot / currently active canvas developers? .
  35. James Turner  (Jan 17th, 2017).  Re: [Flightgear-devel] fgplot / currently active canvas developers? .
  36. erik  (Jan 3rd, 2017).  Re: Simgear compilation failure .
  37. Erik Hofman  (Jan 28th, 2017).  Re: [Flightgear-devel] Canvas-svg .
  38. James Turner  (Jan 27th, 2017).  [Flightgear-devel] Canvas-svg .
  39. James Turner  (Jan 27th, 2017).  Re: [Flightgear-devel] Canvas-svg .
  40. Erik Hofman  (Dec 16th, 2016).  [Flightgear-devel] Canvas rendering speed .
  41. Erik Hofman  (Dec 16th, 2016).  Re: [Flightgear-devel] Canvas rendering speed .
  42. Alan Teeder  (Dec 16th, 2016).  Re: [Flightgear-devel] Canvas rendering speed .
  43. Erik Hofman  (Dec 17th, 2016).  Re: [Flightgear-devel] Canvas rendering speed .
  44. zakalawe  (Dec 15th, 2016).  Re: Pitfalls of canvas HUDs .
  45. James Turner  (Dec 29th, 2016).  Re: [Flightgear-devel] Observation about the Canvas Z-ordering .
  46. zakalawe  (Dec 10th, 2016).  Remote canvas utility .
  47. James Turner  (Dec 16th, 2016).  Re: [Flightgear-devel] Canvas rendering speed .