TerraSync: Difference between revisions

Jump to navigation Jump to search
Line 21: Line 21:
</ref>
</ref>


as some of you have probably noticed, some coding is currently happening in the terrasync corner, mostly by James and some small parts by Torsten. The main intention is to get rid of the SVN protocol to distribute scenery but use easier to implement protocols. The idea is to use plain HTTP as the access method for files required by terrasync. This should make it much easier for hosts to set up a mirror, probably enables us to use a CDN for load balancing and also makes the client code slim. We already have three mirrors of the complete scenery online, one at sourceforge and two at privately owned servers. The sourceforge mirror receives daily updates from the main scenery export from http://scenery.flightgear.org/ and all other mirrors pull their data from the sourceforge server. We decided to drop the previous method of telling the client about it's nearest mirror by querying a web service ( http://scenery.flightgear.org/svn-server) in favour of using a more stable service provider of the internet: the domain name service (DNS) system. The fgfs client will query a fancy resource record (RR) type of NAPTR (network authority pointer, https://en.wikipedia.org/wiki/NAPTR_record) for the dns name terrasync.flightgear.org and analye the entries to find the best terrasync mirror. Those entries are already configured and can be examined here: https://toolbox.googleapps.com/apps/dig/#ANY/terrasync.flightgear.org This technique should enable us to provide different sets of scenery, eventually even for only parts of the world in a much easier fashion that it would have been the case with SVN. It also spares many gigabytes on the scenery mirror as they don't have to keep all the 50,000+ revisions we already carry around. I'll start adding the required code over the next couple of days to have it running hopefully for the 2016.2 release. I'll post a message to this list when I flip the final switch and enable the new terrasync. In the mean time, you might see some build failures on Jenkins or other hickups of the system. I promise to fix those as quick as possible. BTW: If you use terrafs, you already receive your data from the new http mirrors, although not use the DNS service resolver.<ref>{{cite web
as some of you have probably noticed, some coding is currently happening in the terrasync corner, mostly by James and some parts by Torsten. The main intention is to get rid of the SVN protocol to distribute scenery but use easier to implement protocols. The idea is to use plain HTTP as the access method for files required by terrasync. This should make it much easier for hosts to set up a mirror, probably enables us to use a CDN for load balancing and also makes the client code slim.  
 
The intention is to migrate to a pure HTTP solution for TerraSync (instead of SVN) after 2016.1 is released, since this will give us many more mirroring and hosting options. The cache files are tiny and reflect the current revision of each file in the local copy; they’re analogous to the data a real SVN client stores in its ‘.svn’ directory in the root of the checkout. I expect the pure HTTP solution will need some very similar files to record the HTTP cache stamps of files it downloads - either one file at the root of the tree, or per-directory. (There’s advantages and disadvantages to both).<ref>{{cite web
  |url    = https://sourceforge.net/p/flightgear/mailman/message/34767003/
  |title  = <nowiki>Re: [Flightgear-devel] Launcher changes for 3.8</nowiki>
  |author = <nowiki>James Turner</nowiki>
  |date  = Jan 14th, 2016
  |added  = May 4th, 2016
  |script_version = 0.31
  }}
</ref>
 
We already have three mirrors of the complete scenery online, one at sourceforge and two at privately owned servers. The sourceforge mirror receives daily updates from the main scenery export from http://scenery.flightgear.org/ and all other mirrors pull their data from the sourceforge server. We decided to drop the previous method of telling the client about it's nearest mirror by querying a web service ( http://scenery.flightgear.org/svn-server) in favour of using a more stable service provider of the internet: the domain name service (DNS) system. The fgfs client will query a fancy resource record (RR) type of NAPTR (network authority pointer, https://en.wikipedia.org/wiki/NAPTR_record) for the dns name terrasync.flightgear.org and analye the entries to find the best terrasync mirror. Those entries are already configured and can be examined here: https://toolbox.googleapps.com/apps/dig/#ANY/terrasync.flightgear.org This technique should enable us to provide different sets of scenery, eventually even for only parts of the world in a much easier fashion that it would have been the case with SVN. It also spares many gigabytes on the scenery mirror as they don't have to keep all the 50,000+ revisions we already carry around. I'll start adding the required code over the next couple of days to have it running hopefully for the 2016.2 release. I'll post a message to this list when I flip the final switch and enable the new terrasync. In the mean time, you might see some build failures on Jenkins or other hickups of the system. I promise to fix those as quick as possible. BTW: If you use terrafs, you already receive your data from the new http mirrors, although not use the DNS service resolver.<ref>{{cite web
   |url    = http://sourceforge.net/p/flightgear/mailman/message/35057670/
   |url    = http://sourceforge.net/p/flightgear/mailman/message/35057670/
   |title  = <nowiki>[Flightgear-devel] The future of terrasync</nowiki>
   |title  = <nowiki>[Flightgear-devel] The future of terrasync</nowiki>

Navigation menu