TerraSync

Not to be confused with TerraGear, a toolset to generate scenery.

If you have a steady and reasonably fast internet connection, you can use the built-in TerraSync instead of manually installing sceneries. TerraSync is a utility that automatically downloads the newest version of the needed FlightGear scenery while the simulator is running. It runs in the background (optionally as a separate process), monitors your position, and downloads (or updates) the latest scenery from the master scenery server "just in time". For some time now TerraSync has been integrated into the core FlightGear process, so there is no need to deal with TerraSync for the typical user.

The master repository for TerraSync, i.e. the online resource from which TerraSync downloads its files, is synchronized with the FlightGear Scenery Database once a day. So when using TerraSync, you will always have

1. the latest .stg-files, which tell FlightGear where to place an object
2. the latest static models for objects. (Static models define unique objects that exist in one place only, such as famous buildings or landmarks.)
3. the latest shared models for objects. (Generic models used more than once in different places, each can represent many different objects, like generic houses or ships)

Enabling TerraSync

In the launcher

The scenery download options in the settings panel.

In the launcher, click Settings > Download scenery automatically. If you meet errors related to DNS after starting FG, you may click the "Show more" button on the right and choose a DNS server.

Inside FlightGear

In the FlightGear menu, click File > Scenery Download and then check the "Enable automatic scenery download" option.

Command line

By default, when enabled with --enable-terrasync, TerraSync now downloads to download_dir/TerraSync, where download_dir is FlightGear's download directory.

FlightGear's download directory may be explicitly set using FlightGear's --download-dir command line option, otherwise it defaults to:

• $FG_HOME on non-Windows systems;
• %USERPROFILE%\FlightGear\Downloads on Windows (which may be located under C:\Users\username\Documents, however there are likely several possibilities depending on the Windows version and configuration).

If you neither pass --terrasync-dir nor --download-dir to FlightGear, then the default TerraSync directory is:

• $FG_HOME/TerraSync on non-Windows systems;
• %USERPROFILE%\FlightGear\Downloads\TerraSync on Windows (which is possibly under C:\Users\username\Documents but probably depends on the Windows version and configuration).

POSIX compliant command line shell

Start TerraSync:

% nice terrasync -p 5500 -S -d "$HOME/fgfsScenery"

The -S option tells terrasync to use the SVN protocol to fetch data. If you omit it terrasync will use the rsync program instead (which has to be installed on your system).

Start FlightGear:

% fgfs --atlas=socket,out,1,localhost,5500,udp --fg-scenery="$FG_ROOT/Scenery/:$HOME/fgfsScenery"

The full documentation and source for TerraSync is located in the FlightGear source distribution (in utils/TerraSync/).

Troubleshooting

Network or DNS error

If network/DNS errors are encountered, enter --prop:/sim/terrasync/http-server= your server in the additional settings in launcher (e.g., --prop:/sim/terrasync/http-server=http://flightgear.sourceforge.net/scenery), or click the "Show more" button on the right and select a DNS server.

There is a forum post: Network errors occurred

Corrupted and unreadable files or directories

If you get an error similar to the following in the command line console (black dialog):

Airports/L ... failed:
Can't move 'C:\FlightGear\terrasync\Airports\L\E\.svn\tmp\entries' to 'C:\FlightGear\terrasync\Airports\L\E\.svn\entries': The file or directory is corrupted and unreadable.

and possibly the following popup appears:

Solution

You can probably fix the error by upgrading to new Windows versions.

Locked airport directories

You get an error indicating locked airport directories while TerraSync is running.

Working copy 'D:\Program Files\FlightGear 2.4.0\terrasync\Airports\K' locked

While those directories often actually are updated, the error is annoying.

Solution

Search the TerraSync directory for files named lock and delete them. They are supposed to be removed automatically when a TerraSync update is completed, but sometimes that fails.

Failure to remove file

You get an error indicating that some files can not be removed.

file remove failed: (./.terrasync_cache)  reason: Permission denied

TerraSync cache files can not be deleted. You will see errors in console like above. There is not much to do other then delete the .terrasync_cache files manually and then run FlightGear again.

Resolution

Caution  Not recommended. Understand the consequences

Forum Post w/ PowerShell command is here

Internals

The SHA1 hashes are what’s in the .dirindex files, both on the server and in the local tree. We do not synchronise time-stamps with the server copies because it complicates (makes much less efficient) mirroring on the server side. I am fairly confident hash clashes are not an issue, if the Git folks don’t think they are. We explicitly do not rely on any custom or particular HTTP headers, to maximise the chance of ‘just working’ with different HTTP providers and CDN options in the future. The structure of the .dirindex files is text based and hopefully trivial to anyone who cares to look at them; they are generated on the server side by some scripts Torsten wrote.

The code uses basically identical logic to Git to avoid recalculating SHA hashes for the entire repository; if any of various stat() fields, most especially sie or mod-time, change, we re-compute the hash for that file on disk. Whenever a file’s hash on disk differs from the what the .dirindex file says it should have, we download the file. That’s pretty much a complete description of the synchronisation model.

Hence, it’s safe to copy files into the tree using any tool, or modify then - this will cause the stat() data to change, which will cause the code to recompute the SHA hash, which will trigger a re-download to the server-side copy if the SHA does not match. Note there is no provision to have modified (dirty) files inside the tree; they will always be over-written when next checked.

terrasync.py

First of all, the script is now part of the fgmeta-python repository (this was necessary to share .dirindex processing code with other infrastructure tools, in particular for producing the base package). Installation instructions can be found below the list of files, or more directly in README.md.

The script can provide a complete mirror of the scenery data, also known as TerraSync scenery or just TerraScenery.

Caution  If you fully run this script for the first time, it will download more than 90 GB of data and more than 1,800,000 files in 40,000 folders to your hard disk. It will run for hours, probably days (depending on your Internet connection). Since someone has to pay for the bandwidth, please don't let it download large amounts of data unless you have a good reason to do so (like, you maintain a public scenery mirror); otherwise, the service could be discontinued. That being said, if you just want to see the script in action, you can run it for a few seconds and stop it with Ctrl+C.

If you already have at least a partial set of scenery data from the FlightGear built-in TerraSync or even a full TerraScenery, existing and up-to-date files will be reused and won't be downloaded a second time.

How it works

Note  The following is not completely up-to-date. In particular, current terrasync.py has a check mode in addition to the default sync mode. Run terrasync.py --help for up-to-date usage information.

The HTTP TerraScenery server holds all the well-know files required for scenery, organized in the also well-known directory structure (/Models, /Objects, /Terrain and /Airports) with all their subdirectories. The content of the /Models folder for example is here: http://flightgear.sourceforge.net/scenery/Models/ Each folder contains a (hidden) file called .dirindex (e.g. http://flightgear.sourceforge.net/scenery/Models/.dirindex) that describes the content of that folder by listing its files, subdirectories and checksums of the files and their subdirectories' .dirindex files. The terrasync.py script does the following (simplified):

• a) download the file at the root of the scenery-url (http://flightgear.sourceforge.net/scenery/.dirindex)
• b) walks into any subdirectory listed within the .dirindex
• c) compares the sha1sum of listed files with the local copy and downloads them on mismatch
• d) for listed subdirectories, calls recursively step b)

Even if all scenery files exist on the server, there are initially still approx. 40,000 .dirindex files missing on your drive and downloading will take hours. To speed up the procedure once you have all .dirindex files locally, try running terrasync.py with the --quick option. This will tell terrasync.py to compare the computed sha1sum of every .dirindex file on your disk with that published in the entry in the parent directory's .dirindex. If you have an up-to-date mirror, this will be very quick as only the root .dirindex needs to be downloaded. Subsequent updates with the --quick option will also be very fast as only the updated files will be downloaded and also the .dirindex files of the parent folders up to the root directory.

So far, we have just added files to your disk, what about files that have been removed on the server? The script usually does not touch them unless you add --remove-orphan to the command-line. By doing so, you ask terrasync.py to remove all files (not directories) that exist in a subdirectory but are not listed in the corresponding .dirindex file.

If you don't want to download everything into your current working directory, add --target=/some/folder to your command line. This describes pretty much everything terrasync.py does. Here are some examples how to use it:

# fetch everything from the master repository server into /home/joe/fg/scenery, keep orphan files, (re-)download every .dirindex file 
terrasync.py --target=/home/joe/fg/scenery 

# just pull in changes, quick and lean for subsequent updates 
terrasync.py --target=/home/joe/fg/scenery --quick 

# run a mirror, only keep those files listed on the server 
terrasync.py --target=/home/joe/fg/scenery --quick --remove-orphan 

# use another server to pull the data, not the sourceforge master 
terrasync.py --target=/home/joe/fg/scenery --url=http://someserver.org/otherscenery 

# restrict the area (must be integers, not decimals)
terrasync.py --target=/home/joe/fg/scenery --top 50 --left 18 --bottom 49 --right 19

Enjoy, feedback welcome.

Related content

• Howto: Install scenery
• TerraMaster
• TerraGear