Recommended Property Tree Enhancements: Difference between revisions

Jump to navigation Jump to search
Switch to {{flightgear url}} and {{simgear url}} to fix the broken Gitorious links.
(Switch to {{flightgear url}} and {{simgear url}} to fix the broken Gitorious links.)
 
(17 intermediate revisions by 4 users not shown)
Line 1: Line 1:
{{Out of date}}
{{Template:RFC Navigation}}
== Intro ==
== Intro ==
While FlightGear's powerful PropertyTree implementation can be considered to be the core of FlightGear's dynamic, flexible and open nature (and thus also FlightGear's success in the first place) it is more and more often getting obvious that the PropertyTree itself could benefit from an -at least optionally- more structural approach to processing (reading/writing) certain data (property tree variables), whose integrity may be directly critical for the simulator's integrity itself. This applies in particular in an increasingly parallel, multi-threaded and possibly distributed environment. Where data flow dependencies and relationships need to be made as explicit and obvious as possible.
While FlightGear's powerful PropertyTree implementation can be considered to be the core of FlightGear's dynamic, flexible and open nature (and thus also FlightGear's success in the first place) it is more and more often getting obvious that the PropertyTree itself could benefit from an -at least optionally- more structural approach to processing (reading/writing) certain data (property tree variables), whose integrity may be directly critical for the simulator's integrity itself. This applies in particular in an increasingly parallel, multi-threaded and possibly distributed environment. Where data flow dependencies and relationships need to be made as explicit and obvious as possible.
Line 7: Line 10:
While this flexibility can be tremendously appreciated from a user's point of view (because it makes it so much more intuitive and easy to use the property tree) this very flexibility (that is, the lack of typing-requirements and data-integrity enforcement mechanisms) can be considered to be a troublesome factor when it comes to ensuring (and optionally also enforcing) proper internal sim state to guarantee valid state for critical variables that may seriously impact simulator performance.  
While this flexibility can be tremendously appreciated from a user's point of view (because it makes it so much more intuitive and easy to use the property tree) this very flexibility (that is, the lack of typing-requirements and data-integrity enforcement mechanisms) can be considered to be a troublesome factor when it comes to ensuring (and optionally also enforcing) proper internal sim state to guarantee valid state for critical variables that may seriously impact simulator performance.  


Thus, data flow dependencies need to be formalized and made '''explicit''' [http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg18066.html] in order to help FlightGear become more structured in its housekeeping department. Implementing these suggestions will automatically also help in other areas that are currently limited by FlightGear's current model of internal state management, such as for example by making it easier to implement support for dynamically switchable aircraft (for details, see [[FlightGear Sessions]]).
Thus, data flow dependencies need to be formalized and made '''explicit''' [http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg18066.html] in order to help FlightGear become more structured in its housekeeping department. Implementing these suggestions will automatically also help in other areas that are currently limited by FlightGear's current model of internal state management, such as for example by making it easier to implement support for dynamically switchable aircraft (for details, see [[FlightGear Sessions]] or [[Reset & re-init]]).
 
In addition, this very meta information will also help make FlightGear more easily parallelizable:
 
"In general, FG has quite a few data dependencies internally which make multi-threading challenging
right now - there's groundwork to make the data-dependencies more explicit (i.e, via the property
tree) that has to happen before pieces can easily move to other threads."[http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg18066.html]


== Problems ==
== Problems ==
Line 18: Line 27:
In addition, such losely structured and organized write access to properties also raises the question of housekeeping and cleanup responsibility, once data needs to be cleaned up for example, i.e. when re-initializing a specific subsystem or possibly the whole simulator.
In addition, such losely structured and organized write access to properties also raises the question of housekeeping and cleanup responsibility, once data needs to be cleaned up for example, i.e. when re-initializing a specific subsystem or possibly the whole simulator.


As of 05/2009 this is an issue that has also been discussed on the jsbsim developers mailing list, because there is currently no clear policy whether distinct components (such as an FDM) should generally only mutate internal/private state (i.e. state that is at least conceptually 'owned' by the component) or whether components should also be allowed to mutate outside state outside their own branch of the private property tree (see [[FDM_engine_feature_standardization#Constraining_Property_Tree_Access|Constraining Property Tree Access for FDMs]] for details).
As of 05/2009 this is an issue that has also been discussed on the jsbsim developers mailing list, because there is currently no clear policy whether distinct components (such as an FDM) should generally only mutate internal/private state (i.e. state that is at least conceptually 'owned' by the component) or whether components should also be allowed to mutate outside state outside their own branch of the private property tree (see [[FDM engine feature standardization#Constraining Property Tree Access|Constraining Property Tree Access for FDMs]] for details).


=== No concept of (exclusive) property ownership ===
=== No concept of (exclusive) property ownership ===
Line 28: Line 37:


=== Concerning the Concept of Property Ownership ===
=== Concerning the Concept of Property Ownership ===
It's pretty obvious that this might probably also be very useful to help prepare FlightGear for a more modular and parallelized (multithreaded) future (as discussed and proposed in [http://wiki.flightgear.org/flightgear_wiki/images/1/1e/New_FG_architecture.pdf]), where it would be of paramount importance to provide very finely-grained access to data that may possibly require locking in a threaded environment, so that FG components would no longer directly write to properties (that are conceptually owned by other subsystems) but rather write their requests [=new values] to a blocking, component-specific queue, which would in turn be processed (validated, checked for integrity) by the owning component in a sequential fashion to ensure valid internal state.
It's pretty obvious that this might probably also be very useful to help prepare FlightGear for a more modular and parallelized (multithreaded) future (as discussed and proposed in [http://wiki.flightgear.org/flightgear_wiki/images/1/1e/New_FG_architecture.pdf]), where it would be of paramount importance to provide very finely-grained access to data that may possibly require locking in a threaded environment, so that FG components would no longer directly write to properties (that are conceptually owned by other subsystems) but rather write their requests [=new values] to a blocking, component-specific queue, which would in turn be processed (validated, checked for integrity) by the owning component in a sequential fashion to ensure valid internal state.


Basically, resembling/emulating queue-based [http://en.wikipedia.org/wiki/Software_transactional_memory Software Transactional Memory]. FlightGear components would then publish a public getter method which could be accessed by all other components to read arbitrary component-specific state, unlike updates (write access) to properties, which would only take place using a well-defined component-specific interface.
Basically, resembling/emulating queue-based [http://en.wikipedia.org/wiki/Software_transactional_memory Software Transactional Memory]. FlightGear components would then publish a public getter method which could be accessed by all other components to read arbitrary component-specific state, unlike updates (write access) to properties, which would only take place using a well-defined component-specific interface.
Line 72: Line 81:


== Introducing "Managed Properties" ==
== Introducing "Managed Properties" ==
{{FGCquote
|1= Making the property tree thread-safe is an interesting but a very daunting task since properties are (ab)used in so many ways.. :) I have a soft spot for using multi-buffering to support concurrent readers with concurrent writers, but for something as unstructured (in terms of who writes where) as this I'm at a loss to see how or if it could be applied.
|2= {{cite web
  | url    = http://sourceforge.net/p/flightgear/mailman/message/34797127/
  | title  = <nowiki>Re: [Flightgear-devel] Designing a thread-safe property tree API
(was Re: A FGPythonSys implementation: ...)</nowiki>
  | author = <nowiki>Anders Gidenstam</nowiki>
  | date  = Jan 26th, 2016
  | added  = Jan 26th, 2016
  | script_version = 0.23
  }}
}}


The idea is to introduce so called "managed properties" for crucial runtime state, in order to ensure, enforce and maintain integrity of such state variables by using additional meta information to apply data-integrity requirements, as well as access privileges to prevent important runtime state from being mutated by non-authorized components or invalid data.
{{FGCquote
|1= the property tree as it is currently is in need of some rework because of the ownship (single desktop aircraft) approach. This is easier than it sounds - basically most of the property tree becomes part of the aircraft and only a few items are shared. This will also allow the switching of aircraft. The reason to consider this now, and maybe not implement it, is to ensure that the design will support this when it is time to implement it.
|2= {{cite web
  | url    = http://sourceforge.net/p/flightgear/mailman/message/34631875/
  | title  = <nowiki>Re: [Flightgear-devel] HLA developments</nowiki>
  | author = <nowiki>Richard Harrison</nowiki>
  | date  = Nov 19th, 2015
  | added  = Nov 19th, 2015
  | script_version = 0.23
  }}
}}


== Recommended new 'types' for nodes ==
The idea is to introduce so called "managed properties" for crucial runtime state, in order to optionally ensure, enforce and maintain integrity of such state variables by using additional meta information to apply data-integrity requirements, as well as access privileges to prevent important runtime state from being mutated by non-authorized components or invalid data.
 
== Recommended new 'types' for nodes ==
Currently, there is no clear distinction taking place between arbitrary strings, filenames, paths, property paths or property nodes - this makes it increasingly complicated to keep track of what sort of data is represented by a particular node/string in the tree and makes it basically impossible to provide validation routines, or even come up with XML schemas [http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg21404.html] that might help validating PropertyList-encoded XML files.
Currently, there is no clear distinction taking place between arbitrary strings, filenames, paths, property paths or property nodes - this makes it increasingly complicated to keep track of what sort of data is represented by a particular node/string in the tree and makes it basically impossible to provide validation routines, or even come up with XML schemas [http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg21404.html] that might help validating PropertyList-encoded XML files.


Line 88: Line 121:
* type="string:property-node" - value to be specified is a property node
* type="string:property-node" - value to be specified is a property node


== Suggested new attributes in PropertyList XML files==
== Suggested new attributes in PropertyList XML files ==


* "scope" or "lifetime" to formally describe the scope of a node as discussed in [[FlightGear Sessions]]
* "scope" or "lifetime" to formally describe the scope of a node as discussed in [[FlightGear Sessions]]
* "live" (or similar) - to indicate dynamically updated (at runtime) node data, i.e. automatically register a corresponding listener for such properties, rather than just using the static value obtained during parse time and treat it as const then, this is likely to be useful for a number of scenarios: right now, each individual FG component requiring this feature, needs to implement it separately, i.e. the autopilot code now features support for dynamically modifiable properties, as well [http://sourceforge.net/mailarchive/message.php?msg_id=47F0027F.6080304%40telus.net], the same applies to the gui/dialog code, which may also make use of "live" properties [http://mail.flightgear.org/pipermail/flightgear-cvslogs/2004-May/007871.html], whose values may change at runtime. Increasingly, it is getting obvious that other FlightGear components may also benefit from such a feature [http://sourceforge.net/mailarchive/forum.php?thread_name=483C054B.5010305%40g2ms.com&forum_name=flightgear-devel] or [http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg22403.html]. Thus, it would seem appropriate to generalize the concept of "live properties", whose values may be changed and ought to take effect accordingly. This attribute might from another attribute to indicate the corresponding listener-bound property, i.e.:
* "live" (or similar, jsbsim these properties are called "interface properties" [http://sourceforge.net/mailarchive/forum.php?thread_name=004001c9e8b2%248bb8da20%24a32a8e60%24%40net&forum_name=jsbsim-devel]) - to indicate dynamically updated (at runtime) node data, i.e. automatically register a corresponding listener for such properties, rather than just using the static value obtained during parse time and treat it as const then, this is likely to be useful for a number of scenarios: right now, each individual FG component requiring this feature, needs to implement it separately, i.e. the autopilot code now features support for dynamically modifiable properties, as well [http://sourceforge.net/mailarchive/message.php?msg_id=47F0027F.6080304%40telus.net], the same applies to the gui/dialog code, which may also make use of "live" properties [http://mail.flightgear.org/pipermail/flightgear-cvslogs/2004-May/007871.html], whose values may change at runtime. Increasingly, it is getting obvious that other FlightGear components may also benefit from such a feature [http://sourceforge.net/mailarchive/forum.php?thread_name=483C054B.5010305%40g2ms.com&forum_name=flightgear-devel] or [http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg22403.html], [http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg10688.html]. Thus, it would seem appropriate to generalize the concept of "live properties", whose values may be changed and ought to take effect accordingly. This attribute might come from another attribute to indicate the corresponding listener-bound property, i.e.:
  <PropertyList>
  <PropertyList>
   <foo type="double" live="true" property="/foo/myfoo/value"/>
   <foo type="double" live="true" property="/foo/myfoo/value"/>
  </PropertyList>
  </PropertyList>


* "default-value" - to enable contributors to provide sensible default values in XML files (jsbsim has this already in the form of an initial value [http://sourceforge.net/tracker/?func=detail&aid=1796992&group_id=19399&atid=369399])
* "default-value" - to enable contributors to provide sensible default values in XML files (jsbsim has this already in the form of an initial value [http://sourceforge.net/tracker/?func=detail&aid=1796992&group_id=19399&atid=369399], [http://sourceforge.net/mailarchive/forum.php?thread_name=004001c9e8b2%248bb8da20%24a32a8e60%24%40net&forum_name=jsbsim-devel])
* "value" attribute - to directly provide node values as attributes more easily in the form of "<node type="string" value="foo"/> instead of the currently required form "<node type="string">foo</node>"
* "value" attribute - to directly provide node values as attributes more easily in the form of "<node type="string" value="foo"/> instead of the currently required form "<node type="string">foo</node>"
* "description" attribute - to directly document the purpose of a node, so that documentation can be automatically created by walking the property tree
* "description" attribute - to directly document the purpose of a node, so that documentation can be automatically created by walking the property tree
Line 105: Line 138:


* "unit" to directly keep track of the unit of a numeric value and optionally ensure that only values of same unit are written to a variable (currently, units are merely by convention tracked using a corresponding suffix) (also see: [http://sourceforge.net/tracker/?func=detail&aid=1241948&group_id=19399&atid=369399])
* "unit" to directly keep track of the unit of a numeric value and optionally ensure that only values of same unit are written to a variable (currently, units are merely by convention tracked using a corresponding suffix) (also see: [http://sourceforge.net/tracker/?func=detail&aid=1241948&group_id=19399&atid=369399])
* "min"/"max" attributes to define valid ranges for numeric values
* "min"/"max" attributes to define valid ranges for numeric values [http://www.mail-archive.com/flightgear-devel@flightgear.org/msg10993.html]
* "stepping" attribute to define valid stepping for numeric values
* "stepping" attribute to define valid stepping for numeric values
* "enum" for textual multi-state variables, to ensure that only valid state can be written to such a variable
* "enum" for textual multi-state variables, to ensure that only valid state can be written to such a variable
* "enum-values" - store valid values for enum type
* "enum-values" - store valid values for enum type
* "flip-around/mod" - to specify whether a value overflow means to flip back to the 1st valid state
* "wrap/mod" - to specify whether a value overflow means to flip back to the 1st valid state [http://www.mail-archive.com/flightgear-devel@flightgear.org/msg10993.html]
* "notnil" - value may never be empty/unset
* "notnil" - value may never be empty/unset
* tie restrictions ? [http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg00717.html]
* tie restrictions ? [http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg00717.html]
* listener restrictions (max/none)?
* listener restrictions (max/none)?
* max simultaneously registered active (writing) listeners
* max simultaneously registered active (writing) listeners
Line 130: Line 163:


=== Permission handling related ===
=== Permission handling related ===
* "readonly/constant" for constant values that may not change during runtime (or currently: whose changes won't take effect until restart?) [http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg19312.html]
* <del>"readonly/constant" for constant values that may not change during runtime (or currently: whose changes won't take effect until restart?)</del> [http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg19312.html] {{Done}} (by TheTom [{{simgear url|commit=0b197501e1cc611606b9c3728613530714539717}}] [{{flightgear url|commit=771586854b84f6824112f10f854488979e960094}}])


===File handling related ===  
=== File handling related ===
* "filetype:" to explicitly highlight type of file being expected/provided (i.e. "texture/rgb"
* "filetype:" to explicitly highlight type of file being expected/provided (i.e. "texture/rgb") - this should probably make use of standard MIME types
* "default-working-dir" - default cwd if only filename specified without path info
* "default-working-dir" - default cwd if only filename specified without path info


Line 150: Line 183:


     <!-- making use of "base pointers" to directly refer to the proper location instead of using lots of nested relative path specs -->
     <!-- making use of "base pointers" to directly refer to the proper location instead of using lots of nested relative path specs -->
     <aircraft-filename type="string">$FG_ROOT/myAircraft/myAc-set.xml</aircraft-filename>
     <aircraft-filename type="string">[[$FG_ROOT]]/myAircraft/myAc-set.xml</aircraft-filename>
     <some-property type="string">$ROOT_TREE/gui/
     <some-property type="string">$ROOT_TREE/gui/
   </PropertyList>
   </PropertyList>
Line 158: Line 191:


[[Category:RFC]]
[[Category:RFC]]
[[Category:Property Tree]]

Navigation menu