Mapserver: Difference between revisions

From FlightGear wiki
Jump to navigation Jump to search
No edit summary
No edit summary
 
(16 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{Stub}}
{{Stub}}


[[File:Mapserver.jpeg|thumb|mapserver screenshot]]


First announced in April 2016 <ref> {{cite web
First announced in April 2006 <ref> {{cite web
   | url    = http://sourceforge.net/p/flightgear/mailman/message/13175363/
   | url    = http://sourceforge.net/p/flightgear/mailman/message/13175363/
   | title  = <nowiki>[Flightgear-devel] Mapserver interface</nowiki>
   | title  = <nowiki>[Flightgear-devel] Mapserver interface</nowiki>
Line 10: Line 11:
   | script_version = 0.25
   | script_version = 0.25
   }}
   }}
</ref>, the Landcover DB (database) was a web service/server that used to provide {{Abbr|GIS| geographical information system}} data (mainly shapefiles and elevation data) via a PostgreSQL/PostGIS database behind the MapServer and Scenemodels websites <ref>{{cite web
</ref>, the '''Landcover DB''' (database) was a web service/server that used to provide {{Abbr|GIS| geographical information system}} data (mainly shapefiles and elevation data) via a PostgreSQL/PostGIS database behind the '''MapServer''' and '''Scenemodels''' websites <ref>{{cite web
   | url    = http://sourceforge.net/p/flightgear/mailman/message/28592333/
   | url    = http://sourceforge.net/p/flightgear/mailman/message/28592333/
   | title  = <nowiki>[Flightgear-devel] PostGIS update @ Landcover-DB</nowiki>
   | title  = <nowiki>[Flightgear-devel] PostGIS update @ Landcover-DB</nowiki>
Line 18: Line 19:
   | script_version = 0.25
   | script_version = 0.25
   }}
   }}
</ref> for use by FlightGear-related tools like [[TerraGear]] and [[TerraGear GUI]] to help create scenery tiles for FlightGear.  
</ref> for use by FlightGear-related tools like [[TerraGear]] and [[TerraGear GUI]] to help provide data for creating scenery tiles for FlightGear.  
 
The primary intention of the Scenemodels repository is and has ever been from it's initial start to have a common repository which is collaboratively filled with the best scenery models we have for the entire world and which are 'compatible' with the GPL. <ref> {{cite web
  | url    = http://sourceforge.net/p/flightgear/mailman/message/28839631/
  | title  = <nowiki>Re: [Flightgear-devel] Re : Re : Scenery web and scene models</nowiki>
  | author = <nowiki>Martin Spott</nowiki>
  | date  = Feb 15th, 2012
  | added  = Feb 15th, 2012
  | script_version = 0.25
  }}
</ref>
 
Before Martin Spott started the "'''Landcover-DB'''"-project we were having an ongoing dispute about wether the underlying land cover data or some "tweaking the data"-techniques in the TerraGear toolchain had been responsible for certain "unexpected effects" in FlightGear's Scenery.
 
Finally, to resolve this issue, Martin stuffed all our land cover data into a PostGIS database, put a {{wikipedia|MapServer}} as a frontend so everyone could have a look at what's been the actual input to our Scenery processing
<ref> {{cite web
  | url    = http://sourceforge.net/p/flightgear/mailman/message/24018550/
  | title  = <nowiki>Re: [Flightgear-devel] Cliffs</nowiki>
  | author = <nowiki>Martin Spott</nowiki>
  | date  = Nov 20th, 2009
  | added  = Nov 20th, 2009
  | script_version = 0.25
  }}
</ref>


For the time being, the service has been discontinued <ref>{{cite web
For the time being, the service has been discontinued <ref>{{cite web
Line 26: Line 50:
   | date  = Jul 3rd, 2007
   | date  = Jul 3rd, 2007
   | added  = Jul 3rd, 2007
   | added  = Jul 3rd, 2007
  | script_version = 0.25
  }} </ref> <ref>{{cite web
  | url    = http://sourceforge.net/p/flightgear/mailman/message/28880489/
  | title  = <nowiki>Re: [Flightgear-devel] Looking at a nice project from outside</nowiki>
  | author = <nowiki>Martin Spott</nowiki>
  | date  = Feb 23rd, 2012
  | added  = Feb 23rd, 2012
   | script_version = 0.25
   | script_version = 0.25
   }}
   }}
Line 40: Line 71:
   | title  = <nowiki>Re: Is the Scenery Workflow broken?</nowiki>
   | title  = <nowiki>Re: Is the Scenery Workflow broken?</nowiki>
   | author = <nowiki>psadro_gm</nowiki>
   | author = <nowiki>psadro_gm</nowiki>
  | date  = Mar 23rd, 2016
  | added  = Mar 23rd, 2016
  | script_version = 0.25
  }}
</ref> <ref>{{cite web
  | url    = http://forum.flightgear.org/viewtopic.php?p=263629#p263629
  | title  = <nowiki> and terrasync are going </nowiki>
  | author = <nowiki>Torsten</nowiki>
  | date  = Nov 9th, 2015
  | added  = Nov 9th, 2015
  | script_version = 0.25
  }}
</ref>, and the underlying data are unavailable <ref>{{cite web
  | url    = http://forum.flightgear.org/viewtopic.php?p=280237#p280237
  | title  = <nowiki>Re: Is the Scenery Workflow broken?</nowiki>
  | author = <nowiki>Torsten</nowiki>
   | date  = Mar 23rd, 2016
   | date  = Mar 23rd, 2016
   | added  = Mar 23rd, 2016
   | added  = Mar 23rd, 2016
Line 45: Line 92:
   }}
   }}
</ref>)
</ref>)
The data now missing represents man-months of research and tests in GIS land, building and testing tools, building and maintaining the database, shaping all the data into its current form and loading it into the DB, researching and testing how to build detailed airport layouts with TerraGear, the same with OSM roads, ensuring a certain quality level for the land cover data as well as for the Scenemodels repository <ref> {{cite web
  | url    = http://sourceforge.net/p/flightgear/mailman/message/28882536/
  | title  = <nowiki>Re: [Flightgear-devel] Looking at a nice project from outside</nowiki>
  | author = <nowiki>Martin Spott</nowiki>
  | date  = Feb 24th, 2012
  | added  = Feb 24th, 2012
  | script_version = 0.25
  }}
</ref>


Various more or less complete attempts at re-creating this infrastructure have been discussed/attempted meanwhile <ref> {{cite web
Various more or less complete attempts at re-creating this infrastructure have been discussed/attempted meanwhile <ref> {{cite web
Line 82: Line 139:
   | script_version = 0.25
   | script_version = 0.25
   }}
   }}
</ref> - as well as creating surrounding infrastructure/tools like [[TerraFS]] <ref>{{cite web
</ref> (see also elgaton's ongoing status updates in the archives <ref>{{cite web
  |url    =  https://sourceforge.net/p/flightgear/mailman/message/35273151/
  |title  =  <nowiki> Re: [Flightgear-devel] Ready for the 2016.3.1 release? </nowiki>
  |author =  <nowiki> Alessandro Menti </nowiki>
  |date  =  Aug 11th, 2016
  |added  =  Aug 11th, 2016
  |script_version = 0.40
  }}</ref> <ref>{{cite web
  |url    =  https://sourceforge.net/p/flightgear/mailman/message/35135139/
  |title  =  <nowiki> Re: [Flightgear-devel] SIMGEAR build error </nowiki>
  |author =  <nowiki> Alessandro Menti </nowiki>
  |date  =  Jun 3rd, 2016
  |added  =  Jun 3rd, 2016
  |script_version = 0.40
  }}</ref> <ref>{{cite web
  |url    =  https://sourceforge.net/p/flightgear/mailman/message/35110565/
  |title  =  <nowiki> Re: [Flightgear-devel] SIMGEAR build error </nowiki>
  |author =  <nowiki> Alessandro Menti </nowiki>
  |date  =  May 24th, 2016
  |added  =  May 24th, 2016
  |script_version = 0.40
  }}</ref>)- as well as creating surrounding infrastructure/tools like [[TerraFS]] <ref>{{cite web
   | url    = http://sourceforge.net/p/flightgear/mailman/message/34970123/
   | url    = http://sourceforge.net/p/flightgear/mailman/message/34970123/
   | title  = <nowiki>[Flightgear-devel] TerraSync, the lazy way: terrafs</nowiki>
   | title  = <nowiki>[Flightgear-devel] TerraSync, the lazy way: terrafs</nowiki>
Line 91: Line 169:
   }}
   }}
</ref>.
</ref>.
<references/>
<references/>
== Status ==
(last updated in 04/2016)
Because of Torsten's work, we will be able to keep up the current service of maintaining a scenemodels database and regular updates to the distributed scenery.
We will freeze the terrain at its current state to improve the way we handle it today.<ref> {{cite web
  | url    = http://sourceforge.net/p/flightgear/mailman/message/34627039/
  | title  = <nowiki>[Flightgear-devel] Some thoughts regarding scenemodels and
terrascenery</nowiki>
  | author = <nowiki>Torsten Dreyer</nowiki>
  | date  = Nov 17th, 2015
  | added  = Nov 17th, 2015
  | script_version = 0.25
  }}
</ref>
Currently almost all of our scenery is maintained in a database and the resulting files are automatically generated. We try hard to avoid incompatible chunks of scenery that don't match at their corners. Martin Spott has alway tried to encourage people to maintain data upstream and we would like follow this path.<ref>{{cite web
  | url    = http://sourceforge.net/p/flightgear/mailman/message/34921577/
  | title  = <nowiki>Re: [Flightgear-devel] Barcelona Scenery Pack</nowiki>
  | author = <nowiki>Torsten Dreyer</nowiki>
  | date  = Mar 9th, 2016
  | added  = Mar 9th, 2016
  | script_version = 0.25
  }}
</ref>
The vision is having a solid tool chain generating a world scenery from a detailed and common database.
Lets avoid gaps, elevation steps, overlapping and incompatible scenery tiles and work together on a one-fits-all scenery.
Until we have that, the suggestion is this:
Somebody provides self-contained and tested chunks of Terrain to be merged into Terrasync. That "somebody" is responsible for the usability of that terrain chunk. By "provide" I mean:
* make it available by rsync, tarball or anything else to be fetched, unpacked into terrasync without human interaction.
* Work out a method to discover new or outdated chunks
* Work out a method how to tell scenemodels-db which objects need new elevations
* Somehow "sign" the chunk so we have somebody who gurantees GPL compatibility
* Work out a method how to avoid mutual exclusive changes (one person updating EDFE, another improving EDDF [both share a tile])
Not everything need to be ready in detail but I'd like at least have an idea how this should work.
What will not be accepted is:
Follow some dropbox / gg-drive links, unpack an archive, guess what it might be, remove all side effects, do some hours of test flying, resetting object elevations by guessing the area etc. etc.
'''And always remember - at one day we will have Martin's landcover db back and hopefully a working toolchain. Whenever that happens, we build a new "world scenery". What happens to all those manual updates? I fear they are lost.'''<ref> {{cite web
  | url    = http://sourceforge.net/p/flightgear/mailman/message/34924793/
  | title  = <nowiki>[Flightgear-devel] New Scenery: How? Was: Barcelona Scenery Pack</nowiki>
  | author = <nowiki>Torsten Dreyer</nowiki>
  | date  = Mar 10th, 2016
  | added  = Mar 10th, 2016
  | script_version = 0.25
  }}
</ref>
<references/>
== History ==
== History ==


Line 123: Line 257:


<references/>
<references/>
{{TerraGear}}

Latest revision as of 19:16, 25 August 2016

This article is a stub. You can help the wiki by expanding it.
mapserver screenshot

First announced in April 2006 [1], the Landcover DB (database) was a web service/server that used to provide GIS data (mainly shapefiles and elevation data) via a PostgreSQL/PostGIS database behind the MapServer and Scenemodels websites [2] for use by FlightGear-related tools like TerraGear and TerraGear GUI to help provide data for creating scenery tiles for FlightGear.

The primary intention of the Scenemodels repository is and has ever been from it's initial start to have a common repository which is collaboratively filled with the best scenery models we have for the entire world and which are 'compatible' with the GPL. [3]

Before Martin Spott started the "Landcover-DB"-project we were having an ongoing dispute about wether the underlying land cover data or some "tweaking the data"-techniques in the TerraGear toolchain had been responsible for certain "unexpected effects" in FlightGear's Scenery.

Finally, to resolve this issue, Martin stuffed all our land cover data into a PostGIS database, put a MapServer This is a link to a Wikipedia article as a frontend so everyone could have a look at what's been the actual input to our Scenery processing [4]

For the time being, the service has been discontinued [5] [6] (i.e. is no longer operational [7] [8] [9], and the underlying data are unavailable [10])

The data now missing represents man-months of research and tests in GIS land, building and testing tools, building and maintaining the database, shaping all the data into its current form and loading it into the DB, researching and testing how to build detailed airport layouts with TerraGear, the same with OSM roads, ensuring a certain quality level for the land cover data as well as for the Scenemodels repository [11]

Various more or less complete attempts at re-creating this infrastructure have been discussed/attempted meanwhile [12]

In addition, several contributors have discussed providing osgEarth-based scenery at least as a temporary workaround while the disturbances caused by the mapserver's disappearance are sorted out.[13]

FlightGear core developer Torsten has since begun exploring alternatives, i.e. starting over with a fresh landcover database [14], including a BOINC-based approach [15] (see also elgaton's ongoing status updates in the archives [16] [17] [18])- as well as creating surrounding infrastructure/tools like TerraFS [19].

  1. Martin Spott (Apr 8th, 2006). [Flightgear-devel] Mapserver interface.
  2. Martin Spott (Dec 26th, 2011). [Flightgear-devel] PostGIS update @ Landcover-DB.
  3. Martin Spott (Feb 15th, 2012). Re: [Flightgear-devel] Re : Re : Scenery web and scene models.
  4. Martin Spott (Nov 20th, 2009). Re: [Flightgear-devel] Cliffs.
  5. Martin Spott (Jul 3rd, 2007). [Flightgear-devel] Looking at a nice project from outside.
  6. Martin Spott (Feb 23rd, 2012). Re: [Flightgear-devel] Looking at a nice project from outside.
  7. Torsten Dreyer (Nov 17th, 2015). [Flightgear-devel] Some thoughts regarding scenemodels and terrascenery.
  8. psadro_gm (Mar 23rd, 2016). Re: Is the Scenery Workflow broken?.
  9. Torsten (Nov 9th, 2015).  and terrasync are going .
  10. Torsten (Mar 23rd, 2016). Re: Is the Scenery Workflow broken?.
  11. Martin Spott (Feb 24th, 2012). Re: [Flightgear-devel] Looking at a nice project from outside.
  12. HB-GRAL (Aug 24th, 2011). [Flightgear-devel] New experimental mapserver.
  13. Hooray (Apr 10th, 2016). Re: OSGearth as default scenery?.
  14. Torsten (Mar 23rd, 2016). Re: Is the .
  15. Torsten Dreyer (Dec 3rd, 2015). [Flightgear-devel] Continous, Distributed Generation of Terrain.
  16. Alessandro Menti  (Aug 11th, 2016).  Re: [Flightgear-devel] Ready for the 2016.3.1 release? .
  17. Alessandro Menti  (Jun 3rd, 2016).  Re: [Flightgear-devel] SIMGEAR build error .
  18. Alessandro Menti  (May 24th, 2016).  Re: [Flightgear-devel] SIMGEAR build error .
  19. Torsten Dreyer (Mar 27th, 2016). [Flightgear-devel] TerraSync, the lazy way: terrafs.

Status

(last updated in 04/2016)

Because of Torsten's work, we will be able to keep up the current service of maintaining a scenemodels database and regular updates to the distributed scenery. We will freeze the terrain at its current state to improve the way we handle it today.[1]

Currently almost all of our scenery is maintained in a database and the resulting files are automatically generated. We try hard to avoid incompatible chunks of scenery that don't match at their corners. Martin Spott has alway tried to encourage people to maintain data upstream and we would like follow this path.[2]

The vision is having a solid tool chain generating a world scenery from a detailed and common database.

Lets avoid gaps, elevation steps, overlapping and incompatible scenery tiles and work together on a one-fits-all scenery.

Until we have that, the suggestion is this: Somebody provides self-contained and tested chunks of Terrain to be merged into Terrasync. That "somebody" is responsible for the usability of that terrain chunk. By "provide" I mean:

  • make it available by rsync, tarball or anything else to be fetched, unpacked into terrasync without human interaction.
  • Work out a method to discover new or outdated chunks
  • Work out a method how to tell scenemodels-db which objects need new elevations
  • Somehow "sign" the chunk so we have somebody who gurantees GPL compatibility
  • Work out a method how to avoid mutual exclusive changes (one person updating EDFE, another improving EDDF [both share a tile])

Not everything need to be ready in detail but I'd like at least have an idea how this should work.

What will not be accepted is: Follow some dropbox / gg-drive links, unpack an archive, guess what it might be, remove all side effects, do some hours of test flying, resetting object elevations by guessing the area etc. etc.

And always remember - at one day we will have Martin's landcover db back and hopefully a working toolchain. Whenever that happens, we build a new "world scenery". What happens to all those manual updates? I fear they are lost.[3]

History

Originally, Martin Spott and Ralf Gerlich's work was concentrated on making more and different datasources accessible for scenery creation.

Martin was doing a pretty good job in aggregating a huge set of datasources in the common landcover database (see http://mapserver.flightgear.org/ for a quick look at the data already available) in cooperation with the OSGeo foundation.

They also wanted to make use of freely available raster data, such as Landsat ETM+, for automatic classification of landcover and conversion to vector data which can then be used to replace the low-accuracy VMAP0 data at least partially both in terms of space and content (e.g. we won't be able to extract streets from 28.5m- resp. 14.25m-resolution Landsat bands).

You can still see some results of their original experiments in the landsat_* layers at the above-mentioned mapserver URL and a bit more detailed look at the process at http://custom-scenery.org/Satellite-Image.304.0.html[1]


Whereas CORINE and quite likely all the other datasets, which have mostly been aquired from remote sensing, just contain the coverage you can see from above, VMap is significantly different in it's structure. CORINE shows just the vegetation, let's say evergreen forest _or_ the bare soil, where _no_ vegetation is present. VMap instead may feature the forest _and_ the soil, on which the forest grows, in the same place. Even in really big areas, not just in small, accidential snippets. Thus, in order to transform VMap0 into a topology of the CORINE style, you have to cut feature A out of B and store the remains of B as M. Then add A plus M as, let's say, X, cut X out of C and store the remains of C as N, add X and N into Y, .... and so on. That's rather time consuming, even in GRASS. Unfortunately there's no reliable progress meter. Well, there is one, but in certain cases it rises up to 98 % in just two hours and remains there for an undetermined period. Thus you never know wether your job is going to take just a few days, a few weeks or a few months. I don't like jobs running several months. I had a few of these but the most promising ones were killed when the machine had to be rebooted due to a planned power outage - or an unplanned power outage - or some other reason. Therefore I've recently been trying to re-run the VMap0 cleaning using a slightly different approach. After VMap0 has proven to be really clean, the next step would be to either cut the CLC06 coverage out of CLC00 and then cut the result out of VMap0, or first cut CLC00 out of VMap0 and later replace CLC00 by CLC06 where available.[2]