Howto:Create WS3.0 terrain
| World Scenery 3.0 |
|---|
| General |
This article provides instructions on how to generate basic WS3.0 terrain.
WS3.0 terrain consists of two parts:
- A terrain mesh consisting of a landclass texture draped over an elevation model.
- Line features such as roads, railways, rivers and coastline.
The terrain is generated by a set of tools that are packaged in a docker image for convenience.
Prerequisites
Set up a Workspace
Create a directory with the following sub-directories:
dataoutput/vpboutput/Terrain
Docker
- Install Docker on your platform.
- Pull the docker image by running the following command
docker pull flightgear/ws30-vpb-generator:latest
Optionally, if you are hitting rate limits:
- Create an account on https://hub.docker.com/. (Note that you will need to click on an email verification link before you can log in for the first time)
- Run
docker loginbefore the docker pull command above
Getting the base data
You need two pieces of data for the area of scenery you are generating:
- An elevation model (aka DEM). This indicates what altitude each point of the surface is.
- Landclass data showing what type of terrain is at each point of the surface. This is often either a Raster (effectively a texture), or vector data.
Elevation Model
Download the NASADEM elevation model for the area of scenery you wish to generate. This is available in 1x1 degree blocks from here, and with an interactive browser here.
Unzip the files into the data directory.
Landclass Raster
Download an landclass raster for the area of scenery you wish to generate.
- For Europe, use of CORINE is recommended.
- For the USA NLCD is recommended
- Sentinel-2 data is available for the entire world via ESRI, but has limited set of landclasses.
Put these into the data directory.
More detailed terrain can be created by modifying the landclass raster, and/or generating a new raster from vector data. These processes are discussed below.
Generating Terrain
To generate terrain you need to run the tools within the docker container we installed above. The docker image is like a small, independent virtual computing environment running within your system. This particular docker image has all the scenery generation tools already installed.
Running the docker container
Firstly, get the container running from the directory containing your data and output directories:
docker run --rm --mount "type=bind,source=`pwd`/data,target=/home/flightgear/data,readonly" --mount "type=bind,source=`pwd`/output,target=/home/flightgear/output" -it flightgear/ws30-vpb-generator:latest /bin/bash
You should now find yourself in a bash shell within your container. You should see data and output directories which are linked to the directories you created earlier:
flightgear@ddcac77f7d5e:~$ ls data output
Building the terrain
To build the terrain mesh, use the genVPB.py tool from inside the docker container.
Usage: genVPB.py <min-lon> <min-lat> <max-lon> <max-lat> <input-raster> [--reclass <reclass>] [--coastline <coastline>] [--hgt-dir <hgt-dir>] [--output-dir <output-dir>] <min-lon> <min-lat> <max-lon> <max-lat> - bounding box of scenery to generated <input-raster> - Input landclass raster [coastline] - Optional coastline polygon data (.osm) to clip against. E.g. from https://osmdata.openstreetmap.de/data/land-polygons.html [reclass] - Optional file containing a set of rules for reclassifying the raster similar to r.reclass. See fgmeta/ws30/mappings/ [hgt-dir] - Optional directory containing unzipped NASADEM HGT files. Defaults to '/home/flightgear/data'. [output-dir] - Optional directory for output scenery. Defaults to '/home/flightgear/output/vpb'
For example, to generate a piece of terrain around Edinburgh (latitude 55.5, longitude 3 degrees West)
./scripts/genVPB.py -3 55 -2 56 ./data/uk_wgs84_10m_N54.tif
If you are using anything other than a CORINE raster you will need to reclassify the data to match the landclasses used by FlightGear. Those classes are defined in Materials/base/landclass-mapping.xml. You can reclassify them using the files in the scripts/mappings directory. E.g. to reclassify NLCD2019 data you can use --reclassify ./scripts/mappings/nlcd2019.txt.
genVPB.py will output the data to the output/vpb directory, in which you should find a series of files and directories.
Adding roads, railways and rivers
The terrain mesh does not have any line features - things like roads. These are generated separately by the genRoads.py command by downloading data from OpenStreetMap. To generate line features simply run the genroads.py command:
Usage: ./scripts/genroads.py <scenery_dir> <lon1> <lat1> <lon2> <lat2> <scenery_dir> Scenery directory to write to <lon1> <lat1> Bottom left lon/lat of bounding box <lon2> <lat2> Top right lon/lat of bounding box
The scenery directory should be your output/Terrain directory created above. For example, to generate roads for the area around Edinburgh:
./scripts/genroads.py ./output/Terrain -4 55 -3 56
Inside the ./output/Terrain directory there should be a set of directories and, .STG files text files.
Running FlightGear with the new WS3.0 Terrain
To test the new terrain, simply include the output directory in your scenery path and run FlightGear with the --prop:/scenery/use-vpb=true to enable WS3.0.
Advanced Techniques
The following sections describe more complex techniques to generate higher quality WS3.0 terrain. Almost all of them involve using different data sources to generate a more detailed landclass raster before running the final scenery generation processes described above. Generating a highly detailed landclass raster is where the magic happens.
Most techniques use gdal or grass to modify the raster/vector data, typically using the QGIS program.
Using a different elevation model
If you are using another elevation model other than NASAEM, then you may need to re-project it using QGIS/gdalwarp to the WGS84 CRS (aka EPSG:4326).
Landclass Data Requirements
For any landclass data we need to ensure the data is in the correct format. That means:
- Is a Raster (geotiff) rather than Vector data. This raster will become the texture on the terrain that the terrain shaders do their magic on.
- Uses the WGS84 Coordinate Reference System. The ensures that the terrain generation step is efficient.
- Has the correct landclass values for each terrain type. We use a set of values based on the CORINE raster set, defined in Materials/base/landclass-mapping.xml.
Below is a quick table showing what steps you need to take for common landclass data sources.
| Landclass Data | Warp to WGS84 required? | Landclass re-classification Required? | Raster Simplification Required? | Conversion to Raster Required? |
|---|---|---|---|---|
| CORINE Raster | Yes | No | No | No |
| CORINE Vector | Yes | Yes | No | Yes |
| NLCD | No | Yes | Yes | No |
| Sentinel-2 | Yes | Yes | No | No |
Conversion to Raster must be done manually. Converting to WGS84 and the correct landclasses can be done by the genVPB.py script, but slows down scenery generation. Therefore if you are planning to generate scenery multiple times it is best to pre-process the files yourself.
The easiest way to do these operations is using QGIS, which is available for most platforms. If you are scripting a toolchain, the QGIS tools include command-line equivalents for all commands.
When using QGIS, set the Project CRS to WGS84 (aka EPSG:4326). You can then add layers of Raster or Vector data from files from the Layer->Add Layer menu. When performing any operations, always write out the data to a real file so you can go back to it later. Disk space is cheap :).
Warping Raster Layers to WGS84
genVPB.py will automatically convert to WGS84 if it detects a landclass raster with a different Coordinate System. However, you may wish to do so manually instead so all your data is on the same coordinate system.
In QGIS you can warp a raster layer to a different CRS using the Raster->Projections-Warp (Reproject) tool.
Select the following options in the dialog:
- Input Layer - Check you have selected the correct layer. The CRS is shown at the right.
- Target CRS - set to
EPSG:4326 - WGS84, which should be your project CRS. - Resampling Method to Use - Nearest Neighbour. (Landclass data is not like normal images. You don't want to interpolate between values.)
- Nodata value for output bands - 0.0 (This means that any data at the edges will be Ocean, usually a reasonable default)
- Advanced Parameters - No Compression
- Output data type - Byte
- Reprojected - Choose a target filename
Alternatively you can do this step from the commandline.
gdalwarp -t_srs EPSG:4326 -dstnodata 0.0 -r near -ot Byte -of GTiff -co COMPRESS=NONE -co BIGTIFF=IF_NEEDED /home/stuart/FlightGear/VPB/data/CORINE/u2018_clc2018_v2020_20u1_raster100m/DATA/U2018_CLC2018_V2020_20u1.tif /home/stuart/FlightGear/VPB/data/scratch/corine_WGS84.tif
Warping Vector Layers to WGS84
genVPB.py will automatically convert to WGS84 if it detects a landclass raster with a different Coordinate System. However, you may wish to do so manually instead so all your data is on the same coordinate system.
In QGIS you can warp a vector layer using Vector->Data Management Tools->Reproject Layer.
Set the following options in the dialog:
- Input Layer - Check you have selected the correct layer. The CRS is shown at the right.
- Target CRS - set to
EPSG:4326 - WGS84, which should be your project CRS. - Reprojected - Choose a target filename
Reclassifying Vector Layers
For CORINE vector data in particular, the attributes used in the vector data are not the same as those used by the CORINE Raster data. So we need to create a new attribute on the data.
To do this
- Select
Layer->Open Attribute Table. You should see a table with multiple columns. Each row represents a feature in the data. - Click on
Toggle Editing Mode(Ctrl + E), on the top left of the dialog - Click on
New Field(Ctrl+W) and create a new field called Landclass of type "Whole Number (Integer)". This will create a new column which we will populate with the correct landclass data - Click on the
Open Field Calculatorbutton (Ctrl + I). (If you get an error about only being able to create Virtual fields, go back to the Layer menu, export it and open the exported file). - Select the following options:
- Update Existing Field
- Select the Landclass field you just created.
- Copy the contents of https://sourceforge.net/p/flightgear/fgmeta/ci/next/tree/ws30/mappings/corine_vector.txt into the Expression box (without the comment lines starting with
#). This is just some simple code to set the attribute correctly. The code should be correct for CORINE vector data. If your data is from other sources you will need to work out how you want to map your source data landclasses to the CORINE ones. Materials/base/landclass-mapping.xml can be used as a guide.
- Select
OK. You should see that your landclass column is now populated with the landclass data. - Select
Layer->Save Layer Editsto save you changes
Creating a Raster from a Vector Layer
To create a Raster from a Vector Layer select Raster->Conversion->Rasterize (Vector to Raster).
.png)
Select the following options:
- Input Layer - correct layer, check CRS
- Field to use for burn-in value - select the Landclass column you created above.
- Output raster size units. This is going to set the resolution of your raster. You can work out the resolution in two different ways:
- Select "Georeferenced units" and determine how many degrees each pixel is in latitude and longitude.
- Select "Pixels" and determine the size of raster you want in pixels. This is a good calculator to help.
- Width/Horizontal Resolution. Enter the values you've calculated for the horizontal resolution (longitudinal), or the width of the raster
- Height/Vertical Resolution. Enter the values you've calculated for the vertical resolution (latitude or the height of the raster
- Output extent - Select an option from the box on the right. You can edit the text afterwards. Best practise is to create long thin strips of 1 degree latitude in height, as this makes subsequent processing much easier.
- Assign a specific nodata value to output bands - Select 0.0 for Ocean. CORINE vector data in particular has a lot of nodata for Oceans
- Advanced Parameters - No Compression
- Output data type - Byte
- Rasterized - Select a new filename
Simplifying a Raster Layer
Some Raster Landclass data (NLCD included) has too much noise - in particular large US highway systems are identified as Urban areas.
To smooth it out we can use the GRASS n.neighbors function from the Processing Toolbox in QGIS.
Select the following options:
- Input Layer - correct layer, check CRS
- Neighborhood operation - median. (This is not a normal image, so using an average will result in weird values)
- Neighborhood size - 5.
- Neighbors - Select a new filename.
Reclassifying a Raster Layer
WS3.0 uses CORINE landclass values. If using data from other sources it needs to be reclassified to the correct values. genVPB.py has an option to do this, but you may wish to do so manually.
To do this select GRASS->Raster->r.reclass from the Processing Toolbox.
Select the following options:
- Input Raster Layer - correct layer, check CRS
- Reclass rules text - copy in the contents of https://sourceforge.net/p/flightgear/fgmeta/ci/next/tree/ws30/mappings/nlcd2019.txt. Or an appropriate mapping from your landclass data to CORINE. Note that you can also reference a file using the "File containing reclass rules" option. Note a mapping of 22 24 = 1 is the same as 22 and 24 = 1. For a range of 22 to 24 use 22 23 24 = 1.
- Reclassified - Select a new filename.
(If this doesn't work a similar function is available in the Processing Toolbox under Raster analysis->Reclassify by table. However this doesn't save your table once you close the dialog, and entries have to be manually entered individually which takes a lot of effort)
Step By Step Procedure for Processing NLCD for the USA using the Raster Calculator, Up sampling and GRASS r.neighbors
We will use a predetermined file naming convention throughout this procedure for simplicity. You can use a naming convention of your choice.
Obtain your NLCD_2019_Land_Cover from source of choice, https://www.mrlc.gov/viewer/
NLCD_2019_Land_Cover data = NLCD_2019_Land_Cover_California-Southern.tiff
Obtain your NLCD_2016_Tree_Canopy from source of choice, https://www.mrlc.gov/viewer/
NLCD_2016_Tree_Canopy data = NLCD_2016_Tree_Canopy_California-Southern.tiff
Make Deciduous Layer 255 to 0 EPSG:5070 - NAD83 - Raster Calculator
("NLCD_2016_Tree_Canopy_California-Southern@1" = 255) * 0 + ("NLCD_2016_Tree_Canopy_California-Southern@1" != 255) * "NLCD_2016_Tree_Canopy_California-Southern@1"
Output CRS = EPSG:5070 - NAD83
Output Layer = California-Southern_deciduous-coast-clipped.tiff
OK
Make Tree Canopy Layer EPSG:5070 - NAD83 - Raster Calculator
Use the prevailing tree type, for Deciduous use * 41, for Evergreen use * 42, for MixedForest use 43.
Later we add this Tree Canopy layer to the raster where there is currently no trees in the NLCD
If ocean bordering with index 255 -
("California-Southern_deciduous-coast-clipped@1" != 0) * 42
Else
("NLCD_2016_Tree_Canopy_California-Southern@1" != 0) * 42
Output CRS = EPSG:5070 - NAD83
Output Layer = California-Southern_deciduous.tiff
OK
Make Clipped Ocean Frontage EPSG:5070 - NAD83 - GIMP
Source = NLCD_2019_Land_Cover_California-Southern.tiff
Flood fill index 11 (blue) with 0 (black)
Export tiff = California-Southern_coast-clipped.tiff
Warp NLCD Deciduous and Base NLCD from EPSG:5070 - NAD83 / Conus Albers NLCD to Project CRS: EPSG:4326 - WGS 84 - WARP
If ocean bordering with index 255
Warp California-Southern_coast-clipped.tiff
Else
Warp NLCD_2016_Tree_Canopy_California-Southern.tiff
and
Warp Deciduous NLCD from EPSG:5070 - NAD83 / Conus Albers NLCD to Project CRS: EPSG:4326 - WGS 84
Warp California-Southern_deciduous.tiff
Raster -> Projection - > Warp
Target CRS = EPSG:4326 - WGS 84
Resampling method to use = Nearest Neighbour
Output data type = byte
Reprojected = California-Southern_4326-84.tiff and California-Southern_deciduous_4326-84.tiff
Run
Combine "California-Southern_deciduous_4326-84.@1" and "California-Southern_4326-84@1" - Raster Calculator
This step adds the Tree Canopy layer to the existing NLCD raster only where there is currently no existing tree land cover.
(("California-Southern_4326-84@1" > 0 AND "California-Southern_4326-84@1" != 41 AND "California-Southern_4326-84@1" != 42 AND "California-Southern_4326-84@1" != 43) AND "California-Southern_deciduous_4326-84@1" > 0) * "California-Southern_deciduous_4326-84@1" + ("California-Southern_4326-84@1" = 41 OR "California-Southern_4326-84@1" = 42 OR "California-Southern_4326-84@1" = 43 OR "California-Southern_deciduous_4326-84@1" <= 0) * "California-Southern_4326-84@1"
Output CRS = EPSG:4326 - WGS 84
Output Layer = California-Southern_adjusted.tiff
OK
Re-class urban 21, 22, 23, 24 = grass 26 - Raster Calculator
This step is being used to erase all the man made clutter on the raster with grassland. Instead of grassland you can replace it with any type of prevailing groundcover in the area your processing.
For example this places a road easement of grass on all the road clutter that was originally in the raster. If your processing a desert area you might want to use dirt or sand instead of grassland. For Arctic areas maybe use tundra.
Part of the clutter gets replaced later in the process.
("California-Southern_adjusted@1" = 11) * 41 + ("California-Southern_adjusted@1" = 12) * 34 + ("California-Southern_adjusted@1" = 21) * 26 + ("California-Southern_adjusted@1" = 22) * 26 + ("California-Southern_adjusted@1" = 23) * 26 + ("California-Southern_adjusted@1" = 24) * 26 + ("California-Southern_adjusted@1" = 31) * 27 + ("California-Southern_adjusted@1" = 41) * 26 + ("California-Southern_adjusted@1" = 42) * 24 + ("California-Southern_adjusted@1" = 43) * 25 + ("California-Southern_adjusted@1" = 51) * 30 + ("California-Southern_adjusted@1" = 52) * 29 + ("California-Southern_adjusted@1" = 71) * 26 + ("California-Southern_adjusted@1" = 72) * 32 + ("California-Southern_adjusted@1" = 73) * 31 + ("California-Southern_adjusted@1" = 74) * 31 + ("California-Southern_adjusted@1" = 75) * 32 + ("California-Southern_adjusted@1" = 81) * 18 + ("California-Southern_adjusted@1" = 82) * 19 + ("California-Southern_adjusted@1" = 90) * 25 + ("California-Southern_adjusted@1" = 95) * 35
Output CRS = EPSG:4326 - WGS 84
Output Layer = California-Southern_reclass-grass.tiff
OK
Make Urban layer
("California-Southern_adjusted@1" = 21) * 10 + ("California-Southern_adjusted@1" = 22) * 1 + ("California-Southern_adjusted@1" = 23) * 1 + ("California-Southern_adjusted@1" = 24) * 2
Output CRS = EPSG:4326 - WGS 84
Output Layer = California-Southern_urban.tiff
OK
Remove the road clutter with r.neighbor
GRASS/Raster/r.neighbor
Neighborhood operation = median
Neighborhood size = 7
Neighbors = California-Southern_urban-only.tiff
Run
Combine "California-Southern_reclass-urban@1" and "California-Southern_reclass-grass@1"
This step replaces the "cleaned up" urban areas back into the raster.
("California-Southern_urban-only@1" < 1) * "California-Southern_reclass-grass@1" + ("California-Southern_urban-only@1" != 0) * "California-Southern_urban-only@1"
Output CRS = EPSG:4326 - WGS 84
Output Layer = California-Southern_adjusted-combined.tiff
OK
Up sample California-Southern_adjusted-combined.tiff 2x, 4x or 8x resolution 0.00008309125
Active Layer = California-Southern_adjusted-combined.tiff
Layer/Save As
2x Horizontal = 0.00008309125
2x Vertical = 0.00008309125
4x Horizontal = 0.00010637625
4x Vertical = 0.00010637625
8x Horizontal = 0.00005318812
8x Vertical = 0.00005318812
File Name = California-Southern_final-prep-4x.tiff
OK
Simplify California-Southern_4326-84-hd.tiff with r.neighbor
GRASS/Raster/r.neighbor
Neighborhood operation = median
Neighborhood size = 7
Neighbors = California-Southern_urban-only.tiff
Run
Reclass index 0 to 44, leave the rest the same
("California-Southern_4326-84-hd@1" = 0) * 44 + ("California-Southern_4326-84-hd@1" != 0) * "California-Southern_4326-84-hd@1"
Output CRS = EPSG:4326 - WGS 84
Output Layer = California-Southern_4326-84-hd-corrected.tiff
OK
Optional HD Fresh Water Option
- Step 1 - Obtain and load hi resolution vector layer
Make sure that the vector layer and the raster layer you will eventually merge to have the same projection. ** Extent can be different if you use the option below.
- Use Top Menu: "Raster" -> "Conversion" -> "Rasterize (vector to raster)" or Processing Toolbox: GDAL -> "Vector conversion" -> "Rasterize (vector to raster)"
- Input layer = Southern-California_water_4326-84
- Fixed value to burn = 41 (water)
- Output raster size units = "Georeferenced units"
- Width/Horizontal resolution = 0.00008309125 (4x the base NLCD resolution )
- Height/Vertical resolution = 0.00008309125 (4x the base NLCD resolution)
- ** Output extent = Select the raster layer you will eventually merge with as the "Output extent".
- Under "Rasterized" choose a new filename to save the new raster to. We'll use Southern-California_4326-84-hd-water.tiff.
Click "Run" to generate the hi resolution raster layer.
- Step 2 - Reclass hi res, smoothed water to grass
- Open Raster Calculator
- Under "Output layer" choose a new filename to save the new raster to. We'll use Southern-California_4326-84-hd-nowater.tiff.
- Copy the following into the "Raster Calculator Expression" box. Where Southern-California_4326-84-hd is the name of your hi resolution, smoothed NLCD that includes the water data.
"Southern-California_4326-84-hd@1" * ("Southern-California_4326-84-hd@1" != 41) + 26 * ("Southern-California_4326-84-hd@1 = 41")
Click on "OK". When finished you will have a new raster with the water layer changed to grassland. Another choice would be to change it to sand.
- Step 3 - Combine the hi resolution no water raster and the hi resolution water raster.
- Use Top Menu: "Raster" -> "Miscellaneous" -> "Merge" or Processing Toolbax: GDAL -> Raster Miscellaneous -> Merge
- Input layers = Select "Southern-California_4326-84-hd-nowater@1" and Southern-California_4326-84-hd-water.tiff
- Output data type = "byte".
- Under "Merged" choose a new filename to save the new raster to. We'll use Southern-California_4326-84-hdwater.tiff.
Click "Run" to generate the new merged hi resolution raster layer.
Generating the Terrain using osgdem
Instead of using genVPB.py, you may wish to run osgdem directly.
In the Windows/Docker platform you can send the generate tile command directly to osgdem.exe, one tile at a time.
Using the NLCD raster processing convention from above, following is the the final step after creating the raster and entering bash shell with the windows version of "docker run..."
osgdem --TERRAIN --image-ext png --RGBA --no-interpolate-imagery --disable-error-diffusion --geocentric --no-mip-mapping -t ./data/California-Southern_4326-84-hd-corrected.tiff -d ./SRTM-3/N32W115.hgt -b -115 32 -114 33 --PagedLOD -l 7 --radius-to-max-visible-distance-ratio 3 -o ./output/vpb/w120n30/w115n32/ws_w115n32.osgb
Note: the --image-ext png --RGBA flags are critical to successfully building correctly placed landclasses in the final VPB generated scenery.
If you prefer to run the scenery generation manually, running the VPB osgdem process is described in more detail here: Virtual Planet Builder#Running VPB.
After doing this you should have an output directory containing files of the form output/vpb/w010n50/w004n50/ws_w004n50.osgb, plus a host of sub-directories. Each one of these is a 1x1 tile of terrain.
to leave the container simply type exit.
Packaging the Scenery
Once you have the terrain and line features they should be packaged in a scenery directory in vpb and Terrain sub-directories respectively. E.g.
MyCoolScenery/Terrain MyCoolScenery/vpb
It is good practise to document the data sources used in scenery generation. Some source licenses require attribution of the original data source for anything derived, published or distributed.
To assist in fulfilling these license obligations, you can create a source.xml file in the scenery directory which includes attribution information. This will then be available from within the simulator under Help->Scenery Sources, and may fulfil the attribution requirements of your license. Note that you are responsible for fulfilling any license requirements from the data, not FlightGear.
The format of the file is straightforward:
<?xml version="1.0"?> <PropertyList> <source> <name>Corine Land Cover (CLC) 2018, Version 2020_20u1</name> <link>http://web.archive.org/web/20221112175615/https://land.copernicus.eu/pan-european/corine-land-cover/clc2018?tab=metadata%2A</link> <license>GMES Open License</license> </source> <source> <name>NASADEM Merged DEM Global 1 arc second V001</name> <link>https://www.earthdata.nasa.gov/</link> <license>Public Domain</license> </source> <source> <name>OpenStreetMap</name> <link>https://www.openstreetmap.org/copyright</link> <license>Open Data Commons Open Database License</license> </source> </PropertyList>