Aircraft rating system
Formalizing Aircraft Status
Based on various mailing list discussions since 11/2007, it is getting clear that the current method of classifying aircraft development status is not satisfactory to adequately and fully describe an aircraft's development progress. Which can be pretty frustrating for developers, contributors and users, because of different expectations about an aircraft's "status".
This can be even witnessed in more recent discussions (03/2009), where users explicitly inquire about recommendations for feature-complete aircraft , or users pointing out weaknesses in FDM configurations .
This is mainly because of the fact that different people may have different requirements when it comes to aircraft status and are thus viewing the status from different angles, possibly neglecting progress in other areas.
In addition, users are frequently facing issues when it comes to determining compatibility requirements between aircraft tarballs and fgfs binary releases (see for example ). This very issue is further complicated by the fact, that FlightGear in general doesn't expose any detailed version information that could be queried by arbitrary FlightGear resources (such as aircraft). Furthermore, it is questionable whether simply starting to expose version information in the property tree would be sufficient, i.e. for scenarios where individual features may be removed/added or modified in CVS/HEAD, which however wouldn't necessarily result in a modified version number, so that such modifications wouldn't be easily communicated to FlightGear resources that depend on such features and information (for a more in depth discussion of this particular issue, please see ).
This page is dedicated to providing a place to start collecting suggestions about how to describe aircraft development status more formally and reliably, to point out the strengths and weaknesses of individual aircraft implementations more comprehensively.
Given that people seem to be expecting different things when they read about an aircraft's development status being "alpha", "beta" or "production", it should be possible to address this issue, by starting to classify each aircraft's individual components (i.e. fdm, cockpit panel, 3d model etc) directly, rather than the overall aircraft itself which would result in less-developed parts of an aircraft (which can usually be assumed to be most of it) affecting an aircraft's development status negatively, even if it should feature some very well-developed parts.
This classification process should preferably take place by following certain conventions, so that it is likely that the majority of aircraft types, as well as their various features, are covered thoroughly.
This could for example be achieved by starting to evaluate aircraft based on abstract factors such as the following:
- status/realism of FDM configs (performance characteristics)
- status/realism of exterior 3D models
- status/realism of cockpit panels (2D/3D)
- status/realism of autopilot implementation
- status/realism of aircraft-specific instruments
- status/realism of procedure modeling (i.e. startup)
- status/realism of systems modeling
- status/realism of failure modeling
- status/realism of integrated checklists/procedures
- status/realism of integrated help (i.e. in the form of help dialogs)
- status/realism of integrated scripted tutorials
- status/realism of available documentation
- status/realism of overall usability
Furthermore, it might also be worthwhile to consider evaluating aircraft based on additional criteria such as:
- available support for multiplayer
- available support for AI traffic (i.e. 3D models featuring LOD support may be reused by the AIT system)
- available support for runtime customization (i.e. liveries)
- available support for eye candy/effects
- available support for saving/loading and continuing flights
- available support for saving/loading aircraft configurations
In addition, it is important to keep in mind that there are different requirements for aircraft to appear "realistic" - depending on whether they are mainly used in VFR settings or IFR settings, where the former application scenario puts an increased demand on visual effects, the latter requires IFR-specific features to be in place, for example proper IFR instrumentation, as well as correct navigation lights.
Therefore, it might be a good idea to not only evaluate individual components of an aircraft, but also its readiness for certain usage scenarios, such as for example VFR/IFR flight.
It can be expected that following this or similarly factually objective approaches to classifying aircraft development status, should not only help people understand more easily and more completely an individual aircraft's development status, but also highlight the strengths and weaknesses of an aircraft, so that aircraft authors are motivated because of the work they have actually done, rather than being discouraged because of the work they could not (yet) do (or might possibly never be able to do because of it being outside their usual scope of contributions).
Additionally, this would also help aircraft becoming more self-documented, so that new contributors could more easily see where and how their help could be used.
Implementing support for more thorough status-tracking should be fairly straight forward, as the current approach of keeping aircraft status information by using a property tree node in an aircraft's *-set.xml file, could be easily extended to provide additional meta information, of the aforementioned nature - using the previously suggested type of information, this could end up looking like the following:
<PropertyList> <aircraft-status> <fdm>beta</fdm> <exterior>early-production</exterior> <cockpit>alpha</cockpit> <instrumentation>alpha</instrumentation> <procedures>alpha</procedures> <system-modeling>beta</system-modeling> <failure-modeling>alpha<failure-modeling> <checklist-support>alpha</checklist-support> <tutorial-support>alpha</tutorial-support> <documentation>alpha</documentation> <usability>beta</usability> <eye-candy>alpha</eye-candy> </aircraft-status> </PropertyList>
On the other hand, some of this information should probably not directly go into the toplevel *-set.xml file, but rather into files of the various components (i.e. the cockpit/panel status would be a good candidate to have its status being tracked directly in the panel file).
Extending status information
Given that not all conceivable status for the above mentioned criteria can be properly represented using the existing notion of "status" information being restricted to "alpha, beta, early-production...", it should be considered to support additional states, such as for example "none/unavailable", "draft" or "design".
While it would of course be much easier to simply provide a full-length description space of the status information for each individual entry, where each contributor could leave some information, this would make it more complicated to later on provide web-based frontends to facilitate searching/picking aircraft based on certain attributes.
On the other hand, it is not unlikely that even this level of more finely-grained status information should turn out insufficient in some cases, to describe the status properly, thus combining the two previously mentioned approaches of 1) providing formal status information and 2) providing an optional textual description of the progress would probably be the most powerful long-term solution.
This could for example be accomplished by allowing contributors to optionally specify a corresponding attribute to point to a plaintext (or possibly even HTML) file (residing in the aircraft's folder) that may contain further details, such an XML attribute, named "info-file" is shown in the following example:
<PropertyList> <aircraft-status> <fdm info-file="fdm-status.txt">beta</fdm> <exterior info-file="ext-status.txt">early-production</exterior> <cockpit info-file="cockpit-status.txt">alpha</cockpit> [...] <eye-candy info-file="effects-status.txt">alpha</eye-candy> </aircraft-status> </PropertyList>
These attributes could not only be easily parsed by an XML parser to provide web-based search facilities, they could even be used within FlightGear at runtime, to provide enriched status information by reading in the corresponding files and showing their contents in a dialog.
In the long run, it might even be an interesting option to provide optional support for three different attributes specifying filenames for these status-tags:
That way, it would be possible for contributors to easily provide very detailed information about the development status of individual components or features, simply by dropping a corresponding file into the aircraft tarball, possibly using a standard-location/folder such as "meta-info" within the aircraft's folder.
Likewise, users noticing new issues could easily augment said files and send them to the maintainer to have them committed to HEAD, so that future users can first check out this information before possibly reporting a duplicate issue/feature request.
While it may seem unnecessary to provide this sort of information-collecting infrastructure to aircraft authors and their contributors, it would probably help aircraft development status become more self-documenting and aircraft-development itself become more self-contained, given that key information would be collected in one central place (the repository) and aircraft-specific issues/ideas could be easily reported and checked for by new contributors. In the long run
A Flexible Approach
It is obvious that regardless of how many sensible rating-categories are now identified and used in FlightGear, people will sooner or later want to use modified or even completey new categories to rate the strengths and weaknesses of their aircraft.
Because of this, but also because hardcoding such tag names (i.e. property paths) would be unnecessarily tedious, given that FlightGear itself does not yet actively use this sort of information at the moment to provide aircraft picker/sorting facilities, one could simply by convention have ALL tags below one specific node (i.e. PropertyList/sim/aircraft-status/) -regardless- of their name be honored by potential search frontends.
Because of the easy and flexibility of directly using XML/PropertyList files together with the FlightGear Property Tree, this approach to supporting more finely-grained status information tracking is extremely powerful and is promising enough to even support highly detailed status information for complex aircraft that may feature a number of distinct components, for example to appropriately describe various high performance aircraft or airliners, child nodes could be used:
<systems-modeling> <electrical></electrical> <hydraulic></hydraulic> <pneumatic></pneumatic> ... </systems-modeling>
Fully searchable Aircraft - by Type
Furthermore, many users have frequently requested a possibility to search and pick aircraft based on certain aircraft-specific characteristics, such as for example the type of aircraft (i.e. single engine vs. multi engine, piston vs. jet etc.)
This could also be easily accomplished by having aircraft authors and contributors provide such meta information in the same way, for example:
<PropertyList> <type-of-aircraft include ="aircraft-type.xml"/> </PropertyList>
NOTE: for the sake of illustration purposes, false values have also been specified, even though they would be obsolete if all all values are by default assumed to be false, which would significantly reduce unnecessary overhead.
<PropertyList> <is-fixed-wing type="bool">true</is-fixed-wing> <is-helicopter type="bool">false</is-helicopter> <is-sailplane type="bool">false</is-sailplane> <is-amphibian type="bool">false</is-amphibian>
<is-military type="bool">false</is-military> <is-civilian type="bool">true</is-civilian> <is-airliner type="bool">true</is-airliner> <is-multi-cew type="bool">true</is-multi-cew>
<is-single-engine type="bool">false<is-single-engine> <is-multi-engine type="bool">true<is-multi-engine>
<has-piston-propulsion type="bool">false</has-piston-propulsion> <has-jet-propulsion type="bool">true</has-jet-propulsion> ... </PropertyList>
From a programming point of view, this doesn't require any work as it only requires contributors to provide said information-however from a usability point of view, it adds the possibility for GUI/WEB frontends to easily offer a way to search aircraft using this information. By restricting this information to be of Boolean type, it is very straightforward to either directly search the base package or to simply create a small database for very powerful queries-by just having users check/uncheck a corresponding checkbox, no matter whether this is on a webpage or directly in a FlightGear GUI launchr such as fgrun.
For some components (like FDM configs, 2D panels, 3D cockpits, exterior models or liveries) it may be possible to specify multiple separately referenced files in one single *-set.xml file, in these cases it would seem more appropriate to also track their development status separately in these files, rather than in the top-level aircraft file, this would make sure that the development status is really tracked on a per-component basis. So, that sharing aircraft resources still works properly while retaining component status information.
Ease of Deployment
In order to simplify the introduction of the aforementioned concept to store aircraft status information, it might make sense to consider requesting aircraft authors/maintainers by convention to specify said status information externally, in a separate PropertyList-encoded XML file, that is then only referenced in a corresponding include attribute (i.e. <aircraft-status include="ac-status.xml"/>.
While this doesn't make any conceptual difference, it helps to to keep this structured information in a separate place.
Enforcing this convention could simplify the required parsing process in certain situations and would thus not only help creating web-based search/sort frontends using such status information attributes, but would also enable the wealth of existing standalone FlightGear launchers (most of which do not link to SimGear, and may thus not easily process PropertyList-XML files) to be able to easily deal with this information without having to parse the whole *-set.xml file, rather it would become possible to just concentrate on parsing said "aircraft-status.xml" file for certain purposes.
If an overall aircraft rating is desired, it should be possible to easily determine it, as a product of factors, based on the status information of its individual components, taking into account that the weight of said information varies, depending on the type of aircraft, and its requirements.
Related Community Discussions
- Aircraft frustration (04/2010)
- Aircraft Metadata (06/2010)
- fewer aircraft (08/2010)
- Naming conventions (08/2010)
- Discussing aircraft status (10/2010)