20,741
edits
TerraSync: Difference between revisions
Jump to navigation
Jump to search
m
→News: https://sourceforge.net/p/flightgear/mailman/message/35081518/
m (→News) |
m (→News: https://sourceforge.net/p/flightgear/mailman/message/35081518/) |
||
| Line 108: | Line 108: | ||
< | {{Appendix}} | ||
== terrasync.py == | |||
First of all: you will now find it at its new home here: https://sourceforge.net/p/flightgear/flightgear/ci/next/tree/scripts/python/terrasync.py | |||
The script now is able to provide a complete mirror of the scenery data also known as terrasync scenery or just terrascenery. | |||
{{Caution|For the impatient, before you try it out: If you run this script for the first time, it will download more than 80GB of data and more than 1,000,000 files in 40,000 folders to your hard disk. It will run for hours, probably days (depending on your internet connection).}} | |||
If you already have at least a partial set of scenery data from flightgear's built-in terrasync or even a full SVN checkout, existing and up-to-date files will be reused and are not being downloaded a second time. | |||
=== How does it work === | |||
The HTTP terrascenery server holds all the well-know files required for scenery, organied in the also well-know directory structure (/Models, /Objects, /Terrain and /Airports) with all their subdirectories. The content of the /Models folder for example is here: http://flightgear.sourceforge.net/scenery/Models/ Each folder contains a (hidden) file called .dirindex (e.g. http://flightgear.sourceforge.net/scenery/Models/.dirindex) that describes the content of that folder by listing it's files, subdirectories and checksums of the files and their subdirectories' .dirindex files. The terrasync.py script does the following (simplified): | |||
* a) download the file at the root of the scenery-url ( http://flightgear.sourceforge.net/scenery/.dirindex) | |||
* b) walks into any subdirectory listed within the .dirindex | |||
* c) compares the sha1sum of listed files with the local copy and downloads them on mismatch | |||
* d) for listed subdirectories, calls recursively step b) | |||
Even if all scenery files exist on the server, there are initially still approx. 40,000 .dirindex files missing on your drive and downloading will take hours. To speed up the procedure once you have all .dirindex files locally, try running terrasync.py with the --quick option. This will tell terrasync.py to compare the computed sha1sum of every .dirindex file on your disk with that published in the entry in the parent''s directories .dirindex. If you have an up-to-date mirror, this will be _very_ quick as only the root .dirindex needs to be downloaded. Subsequent updates with the --quick option will also be very fast as only the updated files will be downloaded and also the .dirindex files of the parent folders up to the root directory. | |||
So far, we have just added files to your disk, what about files that have been removed on the server? The script usually does not touch them unless you add --remove-orphan to the command-line. By doing so, you ask terrasync.py to remove all files (not directories) that exist in a subdirectory but are not listed in the corresponding .dirindex file. | |||
If you don't want to download everything into your current working directory, add --target=/some/folder to your commandline. This describes pretty much everything terrasync.py does. Here are some examples how to use it: | |||
<syntaxhighlight lang="bash"> | |||
# fetch everything from the master repository server into /home/joe/fg/scenery, keep orphan files, (re-)download every .dirindex file | |||
terrasync.py --target=/home/joe/fg/scenery | |||
# just pull in changes, quick and lean for subsequent updates | |||
terrasync.py --target=/home/joe/fg/scenery --quick | |||
# run a mirror, only keep those files listed on the server | |||
terrasync.py --target=/home/joe/fg/scenery --quick --remove-orphan | |||
# use another server to pull the data, not the sourceforge master | |||
terrasync.py --target=/home/joe/fg/scenery --url= http://someserver.org/otherscenery | |||
</syntaxhighlight> | |||
Enjoy, feedback welcome.<ref>{{cite web | |||
|url = https://sourceforge.net/p/flightgear/mailman/message/35081518/ | |||
|title = <nowiki> [Flightgear-devel] Clone the HTTP scenery data with terrasync.py </nowiki> | |||
|author = <nowiki> Torsten Dreyer </nowiki> | |||
|date = May 11th, 2016 | |||
|added = May 11th, 2016 | |||
|script_version = 0.40 | |||
}}</ref> | |||
== Enabling TerraSync == | == Enabling TerraSync == | ||