Hi fellow wiki editors!

To help newly registered users get more familiar with the wiki (and maybe older users too) there is now a {{Welcome to the wiki}} template. Have a look at it and feel free to add it to new users discussion pages (and perhaps your own).

I have tried to keep the template short, but meaningful. /Johan G

Difference between revisions of "Recommended Property Tree Enhancements"

From FlightGear wiki
Jump to: navigation, search
m (Intro)
m (Suggested new attributes in PropertyList XML files)
Line 69: Line 69:
 
== Suggested new attributes in PropertyList XML files==
 
== Suggested new attributes in PropertyList XML files==
  
 +
* "scope" or "lifetime" to formally describe the scope of a node as discussed in [[FlightGear Sessions]]
 
* "live" (or similar) - to indicate dynamically updated (at runtime) node data, i.e. automatically register a corresponding listener for such properties, rather than just using the static value obtained during parse time and treat it as const then, this is likely to be useful for a number of scenarios: right now, each individual FG component requiring this feature, needs to implement it separately, i.e. the autopilot code now features support for dynamically modifiable properties, as well [http://sourceforge.net/mailarchive/message.php?msg_id=47F0027F.6080304%40telus.net], the same applies to the gui/dialog code, which may also make use of "live" properties [http://mail.flightgear.org/pipermail/flightgear-cvslogs/2004-May/007871.html], whose values may change at runtime. Increasingly, it is getting obvious that other FlightGear components may also benefit from such a feature [http://sourceforge.net/mailarchive/forum.php?thread_name=483C054B.5010305%40g2ms.com&forum_name=flightgear-devel]. Thus, it would seem appropriate to generalize the concept of "live properties", whose values may be changed and ought to take effect accordingly. This attribute might from another attribute to indicate the corresponding listener-bound property, i.e.:
 
* "live" (or similar) - to indicate dynamically updated (at runtime) node data, i.e. automatically register a corresponding listener for such properties, rather than just using the static value obtained during parse time and treat it as const then, this is likely to be useful for a number of scenarios: right now, each individual FG component requiring this feature, needs to implement it separately, i.e. the autopilot code now features support for dynamically modifiable properties, as well [http://sourceforge.net/mailarchive/message.php?msg_id=47F0027F.6080304%40telus.net], the same applies to the gui/dialog code, which may also make use of "live" properties [http://mail.flightgear.org/pipermail/flightgear-cvslogs/2004-May/007871.html], whose values may change at runtime. Increasingly, it is getting obvious that other FlightGear components may also benefit from such a feature [http://sourceforge.net/mailarchive/forum.php?thread_name=483C054B.5010305%40g2ms.com&forum_name=flightgear-devel]. Thus, it would seem appropriate to generalize the concept of "live properties", whose values may be changed and ought to take effect accordingly. This attribute might from another attribute to indicate the corresponding listener-bound property, i.e.:
 
  <PropertyList>
 
  <PropertyList>

Revision as of 16:54, 1 November 2008

Intro

While FlightGear's powerful PropertyTree implementation can be considered to be the core of FlightGear's dynamic, flexible and open nature (and thus also FlightGear's success in the first place) it is more and more often getting obvious that the PropertyTree itself could benefit from an -at least optionally- more structural approach to processing (reading/writing) certain data (property tree variables), whose integrity may be directly critical for the simulator's integrity itself. This applies in particular in an increasingly parallel, multi-threaded and possibly distributed environment.

Today's Practice

Simply put: property tree use in many places in FlightGear today is often following a very simple pattern, where the property tree is simply used as a (very) powerful and easy-to-use "dump space" for all sorts of data and runtime state stored in property tree variables, most of which often being completely typeless or at least very losely-typed, so that access to this state is at most also very losely-organized due to the lack of integrity-enforcing and access-limiting mechanisms.

While this flexibility can be tremendously appreciated from a user's point of view (because it makes it so much more intuitive and easy to use the property tree) this very flexibility (that is, the lack of typing-requirements and data-integrity enforcement mechanisms) can be considered to be a troublesome factor when it comes to ensuring (and optionally also enforcing) proper internal sim state to guarantee valid state for critical variables that may seriously impact simulator performance.

Thus, data flow dependencies need to be formalized and made explicit [1] in order to help FlightGear become more structured in its housekeeping department. Implementing these suggestions will automatically also help in other areas that are currently limited by FlightGear's current model of internal state management, such as for example by making it easier to implement support for dynamically switchable aircraft (for details, see FlightGear Sessions).

Problems

No access control taking place

Currently, it is possible -and common practice- for all properties to be easily accessed (read) and written to from arbitrary FlightGear subsystems and components. In fact, it is even possible for aircraft configuration files and scripts to affect/overwrite crucial internal state. Encapsulation is basically non-existent for many properties.

No concept of (exclusive) property ownership

In fact, it is unfortunately even possible to easily register multiple writing property listeners for the very same property (for example, it would be possible -and has been the case in the past- that registered listeners are exercising write access to properties exclusively owned (at least conceptually) by other components, such as the FDM, whose state would then be automatically invalidated due to its registered listeners each time a property is updated).

So that, one change (write access) to such a property may result in multiple subsequent write accesses to the same property by its registered listeners.

This is problematic in that property tree state may be mutated and invalidated by its own registered listeners [2]. And for the majority of FlightGear components relying on tied property listeners, this would indeed result in undefined behavior and can be considered troublesome, because conceptually components such as an FDM engine will normally require exclusive write access to certain output properties.

Differentiating between "active" and "passive" listeners

While there are thinkable scenarios where such a behavior is desired, i.e. to implement software filters, the approach of coming up with new state by overwriting the underlying original state can be generally considered bad practice as it could be much more reliably, powerfully and easily implemented by adding a corresponding layer of indirection, where new output doesn't automatically invalidate original input state, which then wouldn't be available any more.

In fact, at least informally on the FlightGear Devel mailing list, this problem has been repeatedly discussed, so that it is now generally considered bad practice to register multiple updating/writing listeners for the same property.

However, on the other hand there is currently no clear distinction taking place between active and passive property listeners.

Basically, for more controlled access to properties, there needs to be a formal difference between listeners that require write access to a property and non-writing listeners that must not affect the state of the listened-to property. By providing the corresponding wrappers around the SGPropertyListener interface (i.e. SGPassivePropertyListener(const&) & SGActivePropertyListener) it should be possible to establish a more formal approach to accessing properties via listeners within the FlightGear core code.

No formal validation taking place

Today, in FlightGear's core code there are numerous places where initial key-state is -for the sake of flexibility- retrieved dynamically from the property tree. Which is a good thing.

However, only rarely proper and full validation of the obtained data is done afterwards, rather unvalidated data is often directly used in statements that may at some point either affect the simulator's performance adversely, or even completely shut down the simulator.

Examples for possible problems:

  • values of wrong type being written to/read from a node
  • values of wrong unit being written to/read from a node
  • values outside of valid range being written to/read from a node
  • "0" values read from nodes that must not be 0
  • crucial runtime constants being changed at runtime
  • mis-spelled property names [3] - this could be easily addressed by extending the current fgSet*/fgGet* API functions to also take an optional parameter indicating whether a node MUST_EXIST, NOT_NULL etc.

...

Using automatic validation

While the lack of proper validation is probably mainly because of the tedious process of manually validating data every time something is read from the property tree, this "necessary evil" could be greatly simplified by enriching the property tree itself with attributive meta information for critical nodes/variables, so that the property tree itself could -given the availability of said meta information- handle taking care of ensuring data integrity by doing optional background validation (i.e. possibly SGPropertyListener-based) when writing new state to such critical nodes. Likewise, variables whose state would be monitored and managed this way, would automatically guarantee valid state, so that manual validation would become obsolete.

Introducing "Managed Properties"

The idea is to introduce so called "managed properties" for crucial runtime state, in order to ensure, enforce and maintain integrity of such state variables by using additional meta information to apply data-integrity requirements, as well as access privileges to prevent important runtime state from being mutated by non-authorized components or invalid data.

Recommended new 'types' for nodes

Currently, there is no clear distinction taking place between arbitrary strings, filenames, paths, property paths or property nodes - this makes it increasingly complicated to keep track of what sort of data is represented by a particular node/string in the tree and makes it basically impossible to provide validation routines, or even come up with XML schemas that might help validating PropertyList-encoded XML files.

Thus, it is suggested to start differentiating clearly between these different types of strings:

File handling related

  • type="string:path" - value to be specified is a path
  • type="string:filename" - value to be specified is a filename, this would help tremendously in tracking file dependencies among FlightGear base package XML files (see [4]).

Property Tree related

  • type="string:property-path" - value to be specified is a property path
  • type="string:property-node" - value to be specified is a property node

Suggested new attributes in PropertyList XML files

  • "scope" or "lifetime" to formally describe the scope of a node as discussed in FlightGear Sessions
  • "live" (or similar) - to indicate dynamically updated (at runtime) node data, i.e. automatically register a corresponding listener for such properties, rather than just using the static value obtained during parse time and treat it as const then, this is likely to be useful for a number of scenarios: right now, each individual FG component requiring this feature, needs to implement it separately, i.e. the autopilot code now features support for dynamically modifiable properties, as well [5], the same applies to the gui/dialog code, which may also make use of "live" properties [6], whose values may change at runtime. Increasingly, it is getting obvious that other FlightGear components may also benefit from such a feature [7]. Thus, it would seem appropriate to generalize the concept of "live properties", whose values may be changed and ought to take effect accordingly. This attribute might from another attribute to indicate the corresponding listener-bound property, i.e.:
<PropertyList>
 <foo type="double" live="true" property="/foo/myfoo/value"/>
</PropertyList>
  • "default-value" - to enable contributors to provide sensible default values in XML files
  • "value" attribute - to directly provide node values as attributes more easily in the form of "<node type="string" value="foo"/> instead of the currently required form "<node type="string">foo</node>"
  • "description" attribute - to directly document the purpose of a node, so that documentation can be automatically created by walking the property tree

Validation related

(validation optional: runtime-configurable)

  • "unit" to directly keep track of the unit of a numeric value and optionally ensure that only values of same unit are written to a variable (currently, units are merely by convention tracked using a corresponding suffix)
  • "min"/"max" attributes to define valid ranges for numeric values
  • "stepping" attribute to define valid stepping for numeric values
  • "enum" for textual multi-state variables, to ensure that only valid state can be written to such a variable
  • "enum-values" - store valid values for enum type
  • "flip-around/mod" - to specify whether a value overflow means to flip back to the 1st valid state
  • "notnil" - value may never be empty/unset
  • tie restrictions ? [8]
  • listener restrictions (max/none)?
  • max simultaneously registered active (writing) listeners

Mutability related

While many variables in FlightGear can currently be easily changed at runtime, their changes do not necessarily take effect immediatley, i.e. because there are no listeners bound to them, or because the corresponding subsystems do not yet support re-initialization to work with an updated value. In fact, there are several variables that cannot take effect without resetting individual subsystems, resetting the sim or even completely restarting FlightGear.

It should be considered paramount to document the mutability (or lack thereof) of such variables to ensure that the core's runtime system may eventually become able to track down (and warn about) situations where users are trying to write new state to such variables who may not be aware of these restrictions.

This could probably be achieved by providing attributes for the corresponding nodes that explicitly highlight restrictions such as:

  • changes may not take effect until subsystem reset
  • changes may not take effect until session reset
  • changes may not take effect until simulator reset
  • changes may not take effect until complete simulator restart

Additional information may need to be provided in order to provide further information about which subsystem/s need to be reset, this could be based on global subsystem inode IDs in the form of 2/4/8/16/32... to enable arbitrary combinations of subsystem dependencies.

Permission handling related

  • "readonly/constant" for constant values that may not change during runtime (or currently: whose changes won't take effect until restart?)

File handling related

  • "filetype:" to explicitly highlight type of file being expected/provided (i.e. "texture/rgb"
  • "default-working-dir" - default cwd if only filename specified without path info

Introducing Variables in PropertyList XML Files

  • introducing the concept of "base/frame pointers" for property nodes (and possibly attributes) as described in [9] would surely simplify PropertyList XML files tremendously and make them much more intuitive to work with (this is in fact conceptually pretty much identical to another suggestion, to provide basically the same facility for (non-relative, file system) paths and filenames provided in aircraft/PropertyList XML files, as posted in response[10] to a request by C. Olson to enable aircraft to become more easily relocatable [11] [12]). This could for example be achieved by generalizing the current concept of aliases or by introducing a possibility to set file-specific (or possibly just global) variables in XML files that can later on be referenced and resolved in attributes or nodes in the XML file, i.e. along the lines of (the details might still need to be discussed, though):
  <PropertyList>
    <!-- setting up some variables to be used in attributes or nodes -->
    <xml-variable name="ROOT_TREE"></xml-variable>
    <xml-variable name="FG_ROOT"></xml-variable>

    <!-- preferably, the underlying code would feature support for recursion to allow for use such as: -->
    <xml-variable name="SIM">ROOT_TREE/sim<xml-variable>
    <xml-variable name="FG_AIRCRAFT">FG_ROOT/Aircraft</xml-variable>

    <!-- making use of "base pointers" to directly refer to the proper location instead of using lots of nested relative path specs -->
    <aircraft-filename type="string">%FG_ROOT%/myAircraft/myAc-set.xml</aircraft-filename>
    <some-property type="string">%ROOT_TREE%/gui/
  </PropertyList>

Implementation-wise, it would be possible to add support for this by either extending the current XMLVisitor code in simgear, or by sub-classing it, so that a static std::map<std::string,std::string> could be used to store variable/value pairs, and automatically parse each encountered attribute or node value for a corresponding variable identifier token that should normally not show up in common use scenarios (such as i.e. '%'). That way, all such occurrences could be automatically looked up in the std::map and resolved to their corresponding equivalents at runtime. So that users would then be able to easily refer to "short hands" for well-defined locations (it might make sense to differentiate between local and global xml variables, the latter of which could automatically take effect for all parsed XML files, while the former would only use a lookup map for the root XML file or any files it references using the include directive).