Canvas properties

Proposal for a property-driven 2D drawing API for FlightGear

Update: As of 05/2012 this is being worked on as part of merge request 26 and Howto: Create a 2D drawing API for FlightGear.

This proposal is based on a number of related discussions and/or recent developments, for more detail please see:

• FlightGear Glass Cockpits (introduces the need for a 2D drawing API for use in Nasal)
• Plan-zakalawe (suggests a HTML5/canvas based approach for a 2D drawing API in FlightGear)
• Map (a compelling prototype implementation of a navigational display implemented as a custom PUI widget)
• Modularizing, parallelizing and distributing FlightGear (a discussion that focuses on establishing the property tree as the standard and preferred mechanism for doing subsystem communications)

Core developers acknowledge the fact that FlightGear needs a 2D drawing API to implement Head Up Displays (HUD) and Multi Function Displays (MFD) (see [1]):

What we will need is some cairo/glitz like 2d api to draw the HUD and do some 
MFD's. I believe that this is even more apprioriate for such rendering areas 
like issuing raw OpenGL commands.
With such an abstraction we are open to map that to an immediate more 
rendering pass to a texture or if this is used to build up a part of the 
scenegraph in case we do not have RenderTexture available.

Many of the previous discussions focus on a specific use case scenario for introducing a 2D drawing API to FlightGear, namely that of adding 2D drawing support for scripting glass cockpit functionality.

This discussion however focuses solely on the 2D drawing API itself, without any specific emphasis on a particular use case or manifestation.

Background

• "What we will need is some cairo/glitz like 2d api to draw the HUD and do some MFD's. I believe that this is even more apprioriate for such rendering areas like issuing raw OpenGL commands. With such an abstraction we are open to map that to an immediate more rendering pass to a texture or if this is used to build up a part of the scenegraph in case we do not have RenderTexture available."[2]

• "Although the FlightGear design fairly modular it's provided as a single binary.Everyone who wants to create a new I/O module must patch the FlightGear sources and compile the FlightGear binary from scratch. This may discourage those who want to use FlightGear as a tool and extend it in some way. Moreover, it's not always possible to include all functions in a single binary. Some functions may be mutually exclusive"[3]

• "IMHO the one important threading benefit is if we could get all of the rendering off the main simulation loop, meaning that the model runs independent of the presentation."[4]

• "I guess I'm thinking about how FG is pretty monolithic and wondering how much of an over-head there might be in making it more modular. Might also be worth thinking about parallelism aspects."[5]

• "enforcing subsystems to *only* communicate via the property tree [...] it would then become possible to run any 'clean' subsystem on a pool of worker threads (maybe just one, maybe more)."[6]

Proposal

Previous discussions about a 2D drawing API for FlightGear were mostly about providing such an API specifically to be accessed by the Nasal scripting interpreter (e.g. see FlightGear Glass Cockpits), this discussion instead focuses on using the FlightGear property tree as the common 2D drawing interface among FlightGear subsystems, which would make the 2D drawing API accessible to any FlightGear subsystem (including Nasal) or even external programs accessing FlightGear's property tree, while only adding a very small layer of indirection.

• allow textures to be instantiated by issuing writes to a corresponding branch in the property tree, e.g. /canvas-textures/texture[0]/size-x=128 and /canvas-textures/texture[0]/size-y=128 to dynamically create a 128x128 px texture (color depth could be equally configurable)
• allow primitives to be rendered to such a texture by adding corresponding "point", "line", "circle" (and so on) child nodes, along with relevant meta information (position-x,position-y, radius, color, width)
• allow nested hierarchies of "drawable" nodes, to draw complex shapes - each "drawable" may be composed of other drawables

Goals

• provide property tree driven "render to texture" support: create, modify and delete textures using properties
• generic enough to be usable by aircraft panels, GUI dialogs or scenery elements
• provide a 2D rendering API that may be invoked and parametrized by manipulating the property tree
• generic enough to be used for most 2D drawing needs in FlightGear (e.g. instruments, graphs, flight profile view, HUDs, chart views, flight path evaluation, custom UI widgets)
• provide support for basic drawing primitives (i.e. point, line, circle etc)
• make the 2D drawing API network accessible (which it is automatically by implementing it via the property tree)
• provide building blocks for creating new instrument types or developing experimental glass avionics
• with the long term goal of extending functionality to be able to re-implement hardcoded displays (agradar, wxradar, kln89, dclgps, render_area2d)
• allow for retained primitives
• allow for a high degree of configurability and customization (e.g. support styling of individual elements like CSS) by end users

Canvas Properties

"Canvas properties" would be conventional properties, the only difference being that a certain structure and hierarchy is internally expected to make them valid, so that they can be used to dynamically render to textures.

Creating and drawing to textures would be made possible by setting properties to a branch with registered listeners, that internally manage and implement the rendering aspect, transparently to other subsystems and the end user.

So that for example dynamically drawing a circle to a texture would be as simple as doing something along the lines of:

# set up a texture for the canvas
setprop("/canvas-textures/texture[0]/size-x",128);
setprop("/canvas-textures/texture[0]/size-y",128);
setprop("/canvas-textures/texture[0]/bpp",32);

# add a drawable child node
setprop("/canvas-textures/texture[0]/drawable[0]/type","circle");  # set the type
setprop("/canvas-textures/texture[0]/drawable[0]/position-x",0.5); # add type specific arguments
setprop("/canvas-textures/texture[0]/drawable[0]/position-y",0.5); # add type specific arguments
setprop("/canvas-textures/texture[0]/drawable[0]/radius",30);      # set radius for this type

This is pretty intuitive to work with and it would be straightforward to add a simple Nasal wrapper on top of it.

Internally, the managing subsystem could for example convert each drawable to an OpenGL display list or vertex array to improve efficiency. Once a drawable (or any of its child nodes) is modified, the drawable would be marked as "dirty" to trigger an update.

Approach

Using property listeners, provide support for drawing basic primitives (i.e. points, lines, rectangles, circles) to dynamically created and updated textures by setting properties in a dedicated branch of the property tree. This proposal integrates well with previously discussed plans to make FlightGear more modular by leveraging its property tree to handle IPC among all subsystems and related external programs.

Advantages

While this adds a thin layer of indirection, it has many advantages:

• textures may be dynamically created and modified by arbitrary other FlightGear subsystems, simply by writing to a corresponding node in the property tree, so that new textures can be easily instantiated, and existing ones can be easily modified just by accessing child properties of a "canvas"
• a property driven 2D drawing API has the added benefit of being fully accessible to and usable from other programs accessing the FlightGear property tree, for example using the telnet interface
• this means that it would be perfectly possible to easily provide the infrastructure to allow users to recreate the functionality provided by projects like OpenGC or fggc, because instantiating textures in FlightGear and drawing to these could be easily controlled from external programs
• this approach follows the MVC pattern, that is already in use in many other parts of FlightGear (e.g. FDM)
• this practice of using the property tree as an abstraction layer to generalize and channel access to a subsystem is already in place in other parts of FlightGear (i.e. FDM, GUI, Autopilot)
• all IPC across subsystems would be implicitly handled by writing to properties with registered listeners, no need to explicitly call other subsystems (the AI traffic system is an excellent example for a subsystem that can be largely controlled just by writing to properties)
• so following this approach has the advantage that more complex systems could be easily run outside the FlightGear process space, while still rendering to an OpenGL context shown in FlightGear
• there is a general consensus among FlightGear developers that having subsystems that exclusively communicate with other FlightGear systems using the property tree, will help make FlightGear more modular and more easily parallelizable (see Modularizing, parallelizing and distributing FlightGear)
• by following this route, it will be very straightforward to run components in different threads or processes
• this would make it for example possible to do standalone rendering of certain panel instruments in a dedicated fgfs instance, driven by another one (dedicated instrument viewer)
• so a property driven 2D rendering API provides the infrastructure to facilitate the creation of such systems
• adding a thin layer of Nasal code to provide a wrapper would be very straightforward (similar to how the GUI system is currently wrapped by a corresponding Nasal module, which also just instantiates GUI dialogs by writing to the property tree)
• by using properties with registered listeners, there will be a high degree of configurability and customizability
• symbols would not need to be hardcoded, instead live in the base package and be contributed and maintained there
• serialization: automatically supported by storing everything in property tree XML format
• reusable drawables won't need to be hardcoded, instead they could be saved as XML file in the base package
• parametrization of drawables would make it possible to "derive" from a drawable (i.e. a VOR symbol) and customize it (e.g. by changing the color or size)

Primitives

• Point (x,y)
• Line (x1,y1 - x2,y2)
• Circle (x,y,r)
• Rectangle
• Texture (image)
• Text (string)

See also

• Howto: Create a 2D drawing API for FlightGear