Simplifying Aircraft Deployment

From FlightGear wiki
Jump to navigation Jump to search
This article or section contains out-of-date information

Please help improve this article by updating it. There may be additional information on the talk page.

Package management

Background

Implementation

Front Ends

Related efforts

Request for Comments:

This page is dedicated to collecting and discussing possible steps to help simplifying aircraft deployment in FlightGear. In particular, shortcomings of the current approach of deploying aircraft are meant to be identified and addressed.

Status (06/2014)

Note  This article is purely kept for reference here, it should be considered deprecated, the feature itself is currently being worked on as part of Catalog metadata and Aircraft Center.
Cquote1.png Well, if we make a manual catalog.xml from the current fgdata/Aircraft (which I have parts of a script to do), and we treat the current aircraft zip URLs as canonical, that is possible with current code right now - since it can do the zip download + extraction to a directory of our choice. We could throw an (ugly) PUI UI around that, and then if someone with more talent than me wants to Canvas-GUI-ify it, that will be great.

I will attempt to get a prototype version of this working tonight / tomorrow. I don’t promise to produce anything usable but as you say, anything would be an improvement.
— James Turner (2014-05-30). Re: [Flightgear-devel] select/download aircraft.
Cquote2.png
Cquote1.png Thomas Geymayer and I are working on it, I am not sure where’s he go to with the UI, the backend pieces are mostly there but will need some intensive testing. There is a #define flag you can toggle (in HTTPClient.cxx) to enable the code including the Nasal API; it will download / refresh a catalog, which is generated by the scripts I committed to fgmeta a week ago. Then you can access the package system from pkg.root in the Nasal console, using the API defined at the bottom of HTTPClient.cxx.

If you are crazy enough to enable this code, and run it, be aware it downloads zips, unpacks them, makes calls to unlink files, renames directories, and so on. I would recommend some caution, and especially, don’t run it as root - while developing it I had it extract a few zips to ‘/‘ or worse due to screwed up path logic.

Equally, I would welcome a code review of the code in simgear/package/Install.cxx to check any security issues or dangerous behaviour. The code /tries/ to be ‘safe’ - extract zip to a temporary folder, and uses rename/unlink to atomically update if the zip extraction succeeds. But it’s only had one pair of eyes on it so far.
— James Turner (2014-06-10). Re: [Flightgear-devel] select/download aircraft.
Cquote2.png

Current limitations

  • Aircraft need to be manually downloaded from FlightGear webpage.
  • Aircraft need to be manually checked for version compatibility
  • Aircraft authors need to provide separate *-set.xml files for their aircraft to account for different versions of FlightGear (i.e. see $FG_ROOT/Aircraft/bluebird)
  • Aircraft dependencies need to be determined and satisfied manually
  • Aircraft need to be manually installed by extracting archive to base package
  • Most distributions of Linux will install the FG Root as read only to standard users.
    This means that the user is unable to extract a new aircraft to the same directory tree. See subsection

Desirable long-term goals

  • Allow aircraft to be fully searchable based on aircraft-type/development status (i.e. provide meta info to be used by GUI & web frontends) Formalizing Aircraft Status
  • Allow aircraft to be downloaded from within FlightGear on demand
  • Allow aircraft to be installed from within FlightGear
  • Allow aircraft to be uninstalled/removed (managed) from within FlightGear
  • Allow aircraft to be updated from within FlightGear
  • Allow aircraft to be copied/duplicated within base package (to facilitate working on multiple versions)
  • Allow aircraft to be relocated/moved (renamed) within base package
  • Allow aircraft to be used without requiring an installation
  • Allow aircraft to be installed in and used from custom directories (i.e. non-base package, as in ~/.fgfs/Aircraft or support something like --user-aircraft=/folder/with/non-base-package-aircraft) [1] [2] [3]
  • Allow aircraft to be run from tarballs/archives to allow users to test aircraft easily (Libraries to transparently access archives likes directories: ZZIPLIB [4])
  • Provide support for proper dependency tracking between aircraft and common resources [5]

Possible steps

There are three actions that can be carried out on aircraft:-

  1. Install
  2. Update
  3. Delete (Uninstall)

Installation involved checking if the aircraft (versioning!) exists (better to pull the list of available aircraft to allow user selection).

Installing

We should check for version compatibility of the installed FlightGear binary If an aircraft is not compatible with the current version of FlightGear, do not allow the installation of it.

Version tracking/Compatibility checking

Regarding compatibility between aircraft and fgfs binaries, it is important to keep in mind that so far FlightGear doesn't provide any reliable way for aircraft authors and other contributors to determine what version of FlightGear they're running, or which features are supported by a binary. In fact, currently the only way for aircraft authors to account for different versions of FlightGear (and thus different feature sets) is to provide separate *-set.xml files, so that users themselves may explicitly chose the right ones (or fail to do so, for that matter).

Basically, so far only the base package provides very basic version information, which is however only marginally representative of FlightGear's overall version (i.e. think about FlightGear pulled from CVS/HEAD, where each day features and resources may added, re-implemented or even removed, without such changes being communicated to possible dependencies (aircraft).

This would also be an important factor because of FlightGear's irregular release cycles.

Feature-based version tracking

Thus, the most promising approach to address overall versioning requirements in the context of FlightGear resources might be to tackle this issue from a feature-centric point of view, where the presence (or absence) of features is simply communicated to FlightGear resources by tracking corresponding properties within the property tree.

This could for example take place by including a "features.xml" file from "preferences.xml", which would contain contents similar to: 

 <PropertyList>
   <feature-support>
     <core>
       <has-osg-support type="bool">true</has-osg-support>
       <has-precipitation-support type="bool">true</has-precitipation-support>
       <has-shader-tree-support type="bool">true</has-shader-tree-support>
     </core>
     <fgcommands>
       <has-toggle-command type="bool">true</has-toggle-command>
     </fgcommands>
     <animations>
       <has-pick-animation type="bool">true</has-pick-animation>
     </animations>
   </feature-support>
 </PropertyList>

So that relevant feature set modifications would simply be "communicated" by having contributors use fgSet("/sim/feature-support/...",true/false) and additionally edit the corresponding XML files. This way, it would be fairly easy for aircraft authors to sort of "query" the binary for its capabilities and feature support, i.e. using either a standard nasal script or even an aircraft-specific XML file that would be used at aircraft startup to automatically determine whether a set of given conditions is met or not, such an XML-file could make use of the already existing support for XML-configurable conditions via (SGCondition).

Updating

Updating an aircraft involves checks if the aircraft is installed and then comparing the installed version with the latest version on a server. if the versions are the same (and the local copy is unmodified), alert the user that the versions are the same and no update will occur. Note: While explicit versioning (in XML files) may be one way to tackle the compatibility issue, another one might be to to read in both *-set.xml files and directly compare the resulting SGPropertyNode* structure, this could tell if the XML contents are identical. To do the same for resources included via XML files (i.e. textures, sounds, 3D models etc), one could calculate a simple CRC or hash of each included file.

Deletion/uninstallation

Deleting an aircraft is even easier as we may not need to talk to the server. Instead we check if the aircraft is installed (and unmodified) and should check to ensure that there are no dependencies for the aircraft. An example of which is an aircraft that makes use of another aircraft's instruments and is not centrally installed. Preferably, aircraft would by default only be deleted/uninstalled if they match the following requirements:

  • Aircraft folders would reside in a user-specific location (i.e. $FG_HOMEAircraft): to avoid possibly corrupting system-level FG_ROOT
  • Aircraft folder must be in pristine state/untouched:
    • No removed files
    • No edited/changed files
    • No added folders

While deletion of modified aircraft folders seems possible, it would make sense to only offer this in some sort of "--force" mode, so that users are definitively aware of the fact that an aircraft has been modified, and possible changes would be lost.

By default, an uninstallation routine would only delete those files and folders that are explicitly referenced in a corresponding XML file if they match a given CRC or hash, this would help avoid cases where modified/new files may be accidently deleted by simply doing something like "rm -f $FOLDER".

Installation: Extracting new aircraft

As mentioned above, installing to the distribution approved location such as /usr/local/share/FlightGear/ (for Ubuntu/Debian like distros) would require using something like sudo and prompting for the users password. However, this could be done in helper programs such as fgrun [6]. Despite implementing this it still may not be successful as most distributions do not allow general users to use sudo.

One way around this is to allow the setting of a second location to store aircraft.

Process

  1. User selects an aircraft which is not available
  2. Sub process is created to deal with the request (rather than holding up the entire launcher)
  3. Process will commence downloading the selected aircraft
  4. Extract the aircraft to a user writable Aircraft directory.
  5. Optionally report back to the launcher that the sub process is done.
  6. Sub process exits.

Requirements

Regardless of whether an aircraft needs to be installed, updated or deleted, there's a common set of requirements that need to be satisfied in order to implement support for these actions:

Preferably, each aircraft *-set.xml file would reference a separate XML file detailing its own structure (folders and files, in the sense of "META-INF"), such a list of files would enable future aircraft maintenance routines and arbitrary frontends to easily determine an aircraft's internal dependencies. So that pristine state can be ensured, and possibly critical actions are only performed under valid circumstances (i.e. unaltered state).

In addition, it might make sense to consider using checksums or hashes of each aircraft-specific file to track possible modifications (and thus avoid possible accidental deletion of modified files).

Such an XML file would contain sufficient detail to enable installing, updating and uninstalling aircraft. Due to the possibly dangerous nature of such actions like updating and uninstalling (deleting), users would preferably never directly conduct such operations on the base package, but rather a user-specific aircraft directory, possibly within ~/.fgfs/User-Aircraft.

Relevant information would among others comprise:

  • Filename
  • Location of file
  • Hash of file contents
  • Folders in aircraft directory (while it would see redundant, it seems easier from a dev POV to also explicitly list directories in such XML files)

In XML, this information might be represented in the following form: 

<PropertyList>
 <Aircraft-Files>
  <file name="aircraft-set.xml" location="$AIRCRAFT_FOLDER" original-hash="......"/>
  <file name="foo.rgb" location="$AIRCRAFT_FOLDER/Textures" original-hash="......"/>
  <file name="sound.wav" location="$AIRCRAFT_FOLDER/Sounds" original-hash="......"/>
 </Aircraft-Files>
</PropertyList>

Such files could be created and maintained automatically for each aircraft by using a corresponding perl/python script that would allow contributors to easily create, edit/update and remove entries in such files, so that hashes are automatically updated, too.

Having such a file available, the actions of installing, updating and removing aircraft in a custom location would boil down to:

  • Install
    • Integrity check: iterate over folder contents to ensure pristine state, i.e. all referenced files must exist and their hashes must match, if they don't: cancel
    • Ask for customization options (i.e. new location)
    • Check if location exists and is writable
    • If ok: foreach referenced file, create corresponding folders and copy files, validate results (compare)
  • Update
    • Integrity check, as above
    • Ask for custo
  • Uninstall (Removal/Deletion)
    • Ask for customization options (i.e. folder location & aircraft)
    • Check if location is writable
    • Integrity check
    • If untouched: iterate over referenced entries and unlink
    • If not untouched: warn, show differences and offer "force" mode to remove those files known and referenced

Related content

Wiki articles

Mailing list discussions

Feature requests

Retrieved from "https://wiki.flightgear.org/w/index.php?title=Simplifying_Aircraft_Deployment&oldid=117832"