TerraGear CORINE

See CORINE for the main article about this subject.

Building Europe scenery with SRTM v4, Corine landclass and OpenStreetMap data

This is intended to complete and add up-to-date information on the Using TerraGear tutorial.

Important note: In case you want to contribute to official FlightGear Scenery: Some data sources mentioned in this tutorial are not compatible with FlightGear GPL licence. Scenery created with SRTM 4 elevation data from CGIAR (no commercial use) can’t be released under GPL terms and therefore unfortunately can not come to official scenery releases.

I will assume you are using the Linux build of Terragear. For instructions on how to get it compiled and installed, see Building terragear-cs in Ubuntu 64 Replace all paths where appropriate.

First step after you have finished compiling terragear is to go to the file BuildTiles/Main/main.cxx and make sure CPU rlimit is a large number. I suggest 600000, this will let fgfs-construct run for almost a week. If you don't perform this check fgfs-construct will exit with error after a short time. Make sure you edit the right values, there are two sets one below another. The first ones are not used.

Acquiring data

I. Elevation

Note: SRTM version 4 is an updated and free available elevation dataset by CGIAR. Unfortunately the licence is not compatible with FlightGear GPL licence, when you’re going to use this data for scenery creation it can not come to official scenery releases. When you want to use GPL compliant data please download elevation data from USGS (SRTM 3, version 2_1).

SRTM 4 Download: this file and open it in Google Earth. In Temporary Places select elevation and choose the 5x5 degrees tiles or the 1x1 degrees subtiles. Make sure you download GEOTIFF files. I suggest doing 10x10 degrees areas, this means 4 files. It takes longer to compile scenery but on the whole the process is more efficient. Later you will see why.

Better suggestion for up-to-date data would be to use the heightmaps (1" and 3") from viewfinderpanoramas.org FlightGear is not listed in the list of individuals and sites that have contributed to, acknowledged use of, or discussed products on this site so it might be best to credit him invidually for the moment if you plan on using that site.

II. Landmass

Get VMAP0 landmass from here (v0_landmass) for the 10 by 10 degrees terrain area you selected. Alternatively use other sources for landmass areas like OpenStreetMap Data Land Polygons Download the biggest file and if you are not trying to build the world in one go, use ogr2ogr to make yourself a suitable slice:

./ogr2ogr -clipsrc 21.2695 57.4568 28.3887 60.0539 land_polygons_slice.shp land_polygons.shp

III. Corine landclass

Get the full Corine set from [1]

IV. Road, railroad and coastline data

Download shapefiles only for countries inside the 10x10 tile from here. This will give you roads and railroads. For coastline, go to this site. Download the same files for your country but you will only use coastline for landmass adjustments.

Other datasources can be found here.

Preparing data

I. Elevation

Assuming you are following the directory structure from Using TerraGear , run srtmchop on each TIF file from data/srtm.

There is a catch here: work/SRTM-3 has to be named exactly like that. Otherwise genapts won't find the elevation required for airport surfaces. This name is hardcoded inside the program.

I suggest to run srtmchop manually for each file to check for errors. Ignore warnings generated, some header data cannot be decoded correctly but it won't affect the result.

On certain TIF files, srtmchop will terminate with a memory free() error. We have not figured yet what causes this error only on some files. Until a fix is commited to GIT, you can run srtmchop through valgrind:

valgrind --leak-check=yes ./srtmchop <file> <output dir> 

This will stop the program from exiting and will get the job done, albeit 10 times slower.

After you are finished with generating arr files, you can run terrafit.py:

./terrafit.py -m 50 -x 4000 -e 2 work/SRTM-3

II. Airports

This is a tedious step. genapts has to be run for each airport inside your 10x10 degree area. Copy apt.dat.gz from fgdata/Airports to data/airports. Extract it. For a quick look of which airports are in your area, open this file in Google Earth and remove all other markers, also check view -> grid in menu.

It is best to make a list and iterate through each of the elements. Make sure you have all .arr and .fit files inside work/SRTM-3.

#!/bin/sh
for i in LROP LRBS LRSB
do
   ./genapts --input=./data/airports/apt.dat --work=./work --airport=$i
done

III. Landcover

First open the SRTM files as raster layers in QGis. Edit project properties to set on the fly CRS transformation.

SRTM will serve as a cropping aid, so you can crop the other layers to the size needed.

Edit properties for each SRTM layer and set transparency well above 50% so you can see other layers. After that open each Corine shapefile and make sure the layer really has elements in your area. Not all layers may have. Use the select tool to select a rectangle around SRTM borders. Right click on layer and choose Save Selection As.

After cropping all needed layers, open them again and reproject on WGS84. Perform this operation from the menu Vector -> Data management tools-> Export to new projection. This step may not be necessary as I could gather from the source code, but it's easier to work with all data on the same projection.

Next you will need to move all files for a layer to its own directory inside data/shapefiles. You can perform mapping between Corine classes this way too. For instance I have put all shapefiles for "Green urban areas" in "DeciduousBroadCover" so urban green areas are filled with trees. You can put several shapefiles inside a data/shapefile/*** directory and they will be processed as the same material. You can get creative with roads too, put Corine polygons for roads, railroads etc. inside GrassCover or DeciduousForest, so OSM roads will have a buffer around them made up of trees or grass.

One problem I have faced with Corine is that "Continuous urban fabric" which is mapped to Urban is simply not present in my area. To avoid having only Towns, you can create a new layer in QGis and open the "Discontinuous urban fabric" layer (112) at the same time. Select edit mode for both layers, go to options and set the program to only show vectors from selected area. Then, select cities you would like to be mapped to Urban by holding down CTRL. After you are done, cut the selected polys and paste them into the new layer. Now you have both Town and Urban.

You can then open roads and railroads from OSM and make sure they are ok and have WSG84 as CRS.

One big issue is that fgfs-construct can't handle big street networks. To eliminate them and reduce scene complexity, open the attribute table for the shapefile. Search for features of type "residential", "track", "path", "pedestrian", "footway", "service", "steps" and other unimportant features. Once selected, delete these lines to simplify the shapefile. The process of deleting these features could take a long type if the number is large. During this time QGis may appear to be frozen. If you need something quickier (expecially with large shape files) you can use the utility ogr2ogr to select only the shape you want to keep:

./ogr2ogr -where "type in ('motorway','motorway_link','primary','primary_link','secondary','secondary_link','tertiary','tertiary_link','trunk','trunk_link','road','rest_area')" roads_new.shp roads.shp

Try to simplify the Road shapefile as much as you can, eliminating dense areas of streets. This way you will also decrease your build time. The roads will be simulated by the urban shader anyway. If you have to correct the coastline, open v0_landmass in QGis and the coastline shapefile from OSM. In edit mode you can modify landmass shape to conform to OSM data.

When all shapefiles are arranged properly inside the directory structure, you can run ogr-decode and set materials. But first you may have to edit default_priorities.txt and usgsmap.txt in share/TerraGear to add missing landclasses. Check with materials.xml and CORINE to materials mapping

On normal polygon shapefiles ogr-decode takes the usual parameters:

./ogr-decode --max-segment 500 --area-type Default work/Landmass data/shapefiles/v0_landmass

For roads and railroads, add a third option: --line-width 10 Streams will be covered by Corine polys so don't worry about that.

When building roads and/or railroads, ogr-decode may terminate with some error because elements do not have the correct coordinates. If it does, check the work directory, chances are most of the polys in your area have compiled fine and you don't need to worry about it. To prevent bogus data from killing ogr-decode, you may patch the following file in two places: terragear/src/Lib/terragear/tg_chopper.cxx

Line 85 replace with:

if ( (dx > 2880) || (dy > 1440) )
        //throw sg_exception("something is really wrong in split_polygon()!!!!");
	return;

Bogus data will be skipped, ogr-decode will continue. In order to use OSM road data at maximum potential, you can also use SQL-like queries on the dataset using the --where option. Eg:

 ./ogr-decode --max-segment 500 --line-width 12 --area-type Freeway --where "TYPE=trunk" work/Freeway data/shapefiles/road

./ogr-decode --max-segment 500 --line-width 14 --area-type Freeway --where "TYPE=motorway" work/Freeway data/shapefiles/road

./ogr-decode --max-segment 500 --line-width 10 --area-type Road --where "TYPE=primary" work/Road data/shapefiles/road

./ogr-decode --max-segment 500 --line-width 7 --area-type Road --where "TYPE!=primary AND TYPE!=motorway AND TYPE!=trunk" work/Road data/shapefiles/road

These are suggested road and freeway widths for SE Europe. Your local mileage may vary. At this time, Road and Freeway are mapped as the same material in materials.xml but that may change in the future so build your scenery with that in mind.

According to which lines have been tagged in OSM data, you could even extract bridge segments and asign them a specific landclass which can then have the physical model of a bridge (this may require some hardcoding to ensure proper orientation).

In order to proceed with your own custom landclass mappings, you should also edit the following files to insert your own: install/terragear-cs/share/TerraGear/default_priorities.txt and install/terragear-cs/share/TerraGear/usgsmap.txt

(local paths may vary according to your own installation)

Build process

If all data has been prepared without errors, check inside each directory in work/ and delete folders which do not fit your 10x10 degree tiles. Time to run fgfs-construct and generate the scenery. This may take a very long time depending on your machine, so let it run over the night.

./fgfs-construct --work-dir=./../work --output-dir=./../output --lon=XX --lat=XX --xdist=5 --ydist=5 \
AirportArea AirportObj BarrenCover BuiltUpCover CropGrassCover \
CropWoodCover DeciduousBroadCover DryCropPastureCover EvergreenBroadCover \
GolfCourse GrassCover HerbTundraCover IrrCropPastureCover Lake Landmass Marsh MixedCropPastureCover\
MixedForestCover OpenMining  \
Road Railroad Rock Sand ShrubCover SRTM-3 Stream Town Urban Freeway

The whole process can take 2-3 full days to complete. Still, it is possible to have the whole Europe in one or two months.

General comments from forum discussion

f-ojac, you are right, the parameters used in scenery 2.0 were "-e 5 -x 20000" according to the wiki. I don't know why I had the impression these parameters were not public. In any case, it does not matter because using the same parameters will create closer results, but they are not guaranteed to be the same. The parameters used to create scenery 2,0 seem to be: -m ??: the minimum number of vertices in a tile. In FG, any number bellow 100 (and probably, any number below 1000) will do. I don't think there is any surface on the world perfectly flat for several kilometers. The default value is 50 and I'm sure is ok for any normal use. -e 5: the max allowed error for elevation, in meters. That is: if terrafit calculates a simplification of the terrain where all points are at most this distance from the real elevation, no more vertices are created. The default value is 40 meters: a point may have an elevation error up to 40m (~100ft) High values for this parameter create less detailed mountains and smaller (in disk size) and more efficient (in FPS) terrain. -x 20000: the max number of vertices in a tile. If this number of vertices is reached, terrafit stop regardless the max error of the vertices. The default value is 1000 Keep in mind you can set values that do not make sense: "-e 1 -x 100" does not make sense because it is going to be impossible to calculate errors less than 1 meters using only 100 vertices. The max number of vertices will be reached always and the max error will be probably ignored. "-e 300 -x 20000" does not make sense, tiles are going to use for sure much less vertices than 20,000 because you are allowing huge elevation errors. If you want an efficient scenery (less vertices), use the default values "-e 40 -x 1000". If you want more accurate elevation at the cost of disk space and FPS, use values similar to scenery 2.0 ("-e 5 -x 20000") Anything in the middle will produce performance and disk use in the middle. — ludomotico