Howto:Create WS3.0 terrain

From FlightGear wiki
Jump to navigation Jump to search
World Scenery 3.0
General

This article provides instructions on how to generate base WS3.0 terrain.

WS3.0 terrain consists of two parts:

  1. A landclass texture draped over an elevation model. This is the terrain mesh.
  2. Line features such as roads, railways and rivers. This is draped over the terrain mesh at runtime.

Getting the base data

First, you need two pieces of data for the scenery area you are generating:

  1. An elevation model (aka DEM). Use of the "NASADEM" with 30m resolution is recommended, and is available from here, and with an interactive browser here.
  2. Landclass data showing what type of terrain is at each point of the surface. This is often either a Raster Geotiff (effectively a texture), or vector data. For Europe, use of CORINE is recommended. For the USA NLCD is recommended

Put these files into a data directory and create an output directory next to it. This is where the scenery will be generated..

Pre-processing the Elevation Model

If you are using NASADEM, then the elevation model is already using WGS84 and is ready to use as-is.

If you are using another elevation model, then you may need to re-project it using QGIS/gdalwarp to the WGS84 CRS (aka EPSG:4326).

Pre-processing the Landclass Data

For any landclass data we need to ensure the data is in the correct format. That means:

  1. Uses the WGS84 Coordinate Reference System. The ensures that the terrain generation step is efficient.
  2. Is a Raster (geotiff) rather than Vector data. This raster will become the texture on the terrain that the terrain shaders do their magic on.
  3. Has the correct landclass values for each terrain type. Different landclass systems use different values. At present we use the CORINE values, defined in Materials/base/landclass-mapping.xml.

Below is a quick table showing what steps you need to take for common landclass data sources.

Landclass Data Warp to WGS84 required? Landclass re-classification Required? Raster Simplification Required? Conversion to Raster Required?
CORINE Raster Yes No No No
CORINE Vector Yes Yes No Yes
NLCD No Yes Yes No

The easiest way to do these operations is using QGIS, which is available for most platforms. If you are scripting a toolchain, the QGIS tools include command-line equivalents for all commands.

When using QGIS, set the Project CRS to WGS84 (aka EPSG:4326). You can then add layers of Raster or Vector data from files from the Layer->Add Layer menu. When performing any operations, always write out the data to a real file so you can go back to it later. Disk space is cheap :).

Warping Raster Layers to WGS84

This is applicable to CORINE Raster data

You can warp a raster layer to a different CRS using the Raster->Projections-Warp (Reproject) tool.

Select the following options in the dialog:

  • Input Layer - Check you have selected the correct layer. The CRS is shown at the right.
  • Target CRS - set to EPSG:4326 - WGS84, which should be your project CRS.
  • Resampling Method to Use - Nearest Neighbour. (Landclass data is not like normal images. You don't want to interpolate between values.)
  • Nodata value for output bands - 0.0 (This means that any data at the edges will be Ocean, usually a reasonable default)
  • Advanced Parameters - No Compression
  • Output data type - Byte
  • Reprojected - Choose a target filename


Alternatively you can do this step from the commandline. 

gdalwarp -t_srs EPSG:4326 -dstnodata 0.0 -r near -ot Byte -of GTiff -co COMPRESS=NONE -co BIGTIFF=IF_NEEDED /home/stuart/FlightGear/VPB/data/CORINE/u2018_clc2018_v2020_20u1_raster100m/DATA/U2018_CLC2018_V2020_20u1.tif /home/stuart/FlightGear/VPB/data/scratch/corine_WGS84.tif

Warping Vector Layers to WGS84

You can warp a vector layer using Vector->Data Management Tools->Reproject Layer.

Set the following options in the dialog:

  • Input Layer - Check you have selected the correct layer. The CRS is shown at the right.
  • Target CRS - set to EPSG:4326 - WGS84, which should be your project CRS.
  • Reprojected - Choose a target filename

Reclassifying Vector Layers

For CORINE vector data in particular, the attributes used in the vector data are not the same as those used by the CORINE Raster data. So we need to create a new attribute on the data.

QGIS Field Calculator

To do this

  • Select Layer->Open Attribute Table. You should see a table with multiple columns. Each row represents a feature in the data.
  • Click on New Field (Ctrl+W) and create a new field called Landclass of type "Whole Number (Integer)". This will create a new column which we will populate with the correct landclass data
  • Click on the Open Field Calculator button (Ctrl + I). (If you get an error about only being able to create Virtual fields, go back to the Layer menu, export it and open the exported file).
  • Select the following options:
  • Select OK. You should see that your landclass column is now populated with the landclass data.
  • Select Layer->Save Layer Edits to save you changes

Creating a Raster from a Vector Layer

To create a Raster from a Vector Layer select Raster->Conversion->Rasterize (Vector to Raster).

Creating a Raster from a Vector Layer - QGIS Rasterize

Select the following options:

  • Input Layer - correct layer, check CRS
  • Field to use for burn-in value - select the Landclass column you created above.
  • Output raster size units. This is going to set the resolution of your raster. You can work out the resolution in two different ways:
    • Select "Georeferenced units" and determine how many degrees each pixel is in latitude and longitude.
    • Select "Pixels" and determine the size of raster you want in pixels. This is a good calculator to help.
  • Width/Horizontal Resolution. Enter the values you've calculated for the horizontal resolution (longitudinal), or the width of the raster
  • Height/Vertical Resolution. Enter the values you've calculated for the vertical resolution (latitude or the height of the raster
  • Output extent - Select an option from the box on the right. You can edit the text afterwards.
  • Assign a specific nodata value to output bands - Select 0.0 for Ocean. CORINE vector data in particular has a lot of nodata for Oceans
  • Advanced Parameters - No Compression
  • Output data type - Byte
  • Rasterized - Select a new filename

Simplifying a Raster Layer

Some Raster Landclass data (NLCD included) has too much noise - in particular large US highway systems are identified as Urban areas.

To smooth it out we can use the GRASS n.neighbors function from the Processing Toolbox in QGIS.

Select the following options:

  • Input Layer - correct layer, check CRS
  • Neighborhood operation - median. (This is not a normal image, so using an average will result in weird values)
  • Neighborhood size - 5.
  • Neighbors - Select a new filename.

Reclassifying a Raster Layer

WS3.0 uses CORINE landclass values. If using data from other sources it needs to be reclassified to the correct values.

To do this select GRASS->Raster->r.reclass from the Processing Toolbox.

Select the following options:

  • Input Raster Layer - correct layer, check CRS
  • Reclass rules text - copy in the contents of https://sourceforge.net/p/flightgear/fgmeta/ci/next/tree/ws30/mappings/nlcd2019.txt. Or an appropriate mapping from your landclass data to CORINE. Note that you can also reference a file using the "File containing reclass rules" option. Note a mapping of 22 24 = 1 is the same as 22 and 24 = 1. For a range of 22 to 24 use 22 23 24 = 1.
  • Reclassified - Select a new filename.

(If this doesn't work a similar function is available in the Processing Toolbox under Raster analysis->Reclassify by table. However this doesn't save your table once you close the dialog, and entries have to be manually entered individually which takes a lot of effort)

Step By Step Procedure for Processing NLCD for the USA using the Raster Calculator, Up sampling and GRASS r.neighbors

We will use a predetermined file naming convention throughout this procedure for simplicity. You can use a naming convention of your choice.

  • Step 1 - Re-class from NLCD values to FlightGear values and all man made data to "grasscover" for a clean land cover base.
  1. Open Raster Calculator
  2. Under "Output layer" choose a new filename to save the new raster to. We'll use Kansas_reclass-grass.tiff.
  3. Copy the following into the "Raster Calculator Expression" box. Where Kansas_4326-84@1 is the name of your raw, correct CRS NLCD.

("Kansas_4326-84@1" = 11) * 41 + ("Kansas_4326-84@1" = 12) * 34 + ("Kansas_4326-84@1" = 21) * 26 + ("Kansas_4326-84@1" = 22) * 26 + ("Kansas_4326-84@1" = 23) * 26 + ("Kansas_4326-84@1" = 24) * 26 + ("Kansas_4326-84@1" = 31) * 27 + ("Kansas_4326-84@1" = 41) * 23 + ("Kansas_4326-84@1" = 42) * 24 + ("Kansas_4326-84@1" = 43) * 25 + ("Kansas_4326-84@1" = 51) * 30 + ("Kansas_4326-84@1" = 52) * 29 + ("Kansas_4326-84@1" = 71) * 26 + ("Kansas_4326-84@1" = 72) * 32 + ("Kansas_4326-84@1" = 73) * 31 + ("Kansas_4326-84@1" = 74) * 31 + ("Kansas_4326-84@1" = 75) * 32 + ("Kansas_4326-84@1" = 81) * 18 + ("Kansas_4326-84@1" = 82) * 19 + ("Kansas_4326-84@1" = 90) * 25 + ("Kansas_4326-84@1" = 95) * 35 

   Click on "OK". When finished you will have a new raster with all man made clutter removed and changed to "grasscover".
  • Step 2 - Make a new Urban layer of all all man made clutter.
  1. Open Raster Calculator
  2. Under "Output layer" choose a new filename to save the new raster to. We'll use Kansas_urban.tiff.
  3. Copy the following into the "Raster Calculator Expression" box. Where Kansas_4326-84@1 is the name of your raw, correct CRS NLCD.

("Kansas_4326-84@1" = 21) * 10 + ("Kansas_4326-84@1" = 22) * 10 + ("Kansas_4326-84@1" = 23) * 1 + ("Kansas_4326-84@1" = 24) * 1 

   Click on "Run". When finished you will have a new raster with only all man made clutter.
  • Step 3 - Make a new raster of leaving only towns and larger man made structures.
  1. Click on the Kansas_urban-only layer so it is the active layer.
  2. Click on GRASS -> Raster (r.*) -> r.neighbors
  3. Change the "Neighbourhood operation" to "median" and the "Neighbourhood size" to "7". You might have to adjust the size value to get your desired results.
  4. Under Neighbors choose a new filename to save the new raster to. We'll use Kansas_urban-only.tiff.
   Click on "Run". When finished you will have a new raster with only towns and larger man made structures.
  • Step 4 - Combine the clean, raw, reclassed base raster and the new town and larger structure raster into a new raster.
  1. Open Raster Calculator
  2. Under "Output layer" choose a new filename to save the new raster to. We'll use Kansas_adjusted-combined.tiff.
  3. Copy the following into the "Raster Calculator Expression" box. Where Kansas_urban-only@1 and Kansas_reclass-grass@1 are the names of previously processed step 1 and step 3 data.

("Kansas_urban-only@1" < 1) * "Kansas_reclass-grass@1" + ("Kansas_urban-only@1" != 0) * "Kansas_urban-only@1" 

   Click on "OK". When finished you will have a new raster with only towns and larger man made structures, road easements of grasscover and other default unsmoothed land cover.
  • Step 5 - Upsample 4x to a new higher resolution raster that can then be smoothed.
  1. Click on the Kansas_adjusted-combined layer.
  2. Under Layer "File name" choose a new filename to save this new raster to. We'll use Kansas_final-prep-4x.tiff.
  3. Change the "Resolution (Current:layer) -> Horizontal and Vertical to a value that equals, current "Layer Resolution" / 4.
   Click on "OK". When finished you will have a new up-scaled higher resolution raster.
  • Step 6 - Smooth the Upsampled "final prep" layer.
  1. Click on the Kansas_final-prep-4x layer so it is the active layer.
  2. Click on GRASS -> Raster (r.*) -> r.neighbors
  3. Change the "Neighbourhood operation" to "median" and the "Neighbourhood size" to "7". You might have to adjust the size value to get your desired results.
  4. Under "Neighbors" choose a new filename to save the new raster to. We'll use Kansas_4326-84-hd.tiff.
   Click on "Run". When finished you will have the final smoothed, higher resolution raster to use for building your scenery.

Optional HD Water Option

  • Step 7 - Obtain and load hi resolution vector layer

Make sure that the vector layer and the raster layer you will eventually merge to have the same projection. ** Extent can be different if you use the option below.

  1. Use Top Menu: "Raster" -> "Conversion" -> "Rasterize (vector to raster)" or Processing Toolbax: GDAL -> "Vector conversion" -> "Rasterize (vector to raster)"
  2. Input layer = Kansas_water_4326-84
  3. Fixed value to burn = 41 (water)
  4. Output raster size units = "Georeferenced units"
  5. Width/Horizontal resolution = 0.00008309125 (4x the base NLCD resolution )
  6. Height/Vertical resolution = 0.00008309125 (4x the base NLCD resolution)
  7. ** Output extent = Select the raster layer you will eventually merge with as the "Output extent".
  8. Under "Rasterized" choose a new filename to save the new raster to. We'll use Kansas_4326-84-hd-water.tiff.
   Click "Run" to generate the hi resolution raster layer.
  • Step 8 - Reclass hi res, smoothed water to grass
  1. Open Raster Calculator
  2. Under "Output layer" choose a new filename to save the new raster to. We'll use Kansas_4326-84-hd-nowater.tiff.
  3. Copy the following into the "Raster Calculator Expression" box. Where Kansas_4326-84-hd is the name of your hi resolution, smoothed NLCD that includes the water data.

"Kansas_4326-84-hd@1" * ("Kansas_4326-84-hd@1" != 41) + 26 * ("Kansas_4326-84-hd@1 = 41") 

   Click on "OK". When finished you will have a new raster with the water layer changed to grassland. Another choice would be to change it to sand.
  • Step 9 - Combine the hi resolution no water raster and the hi resolution water raster.
  1. Use Top Menu: "Raster" -> "Miscellaneous" -> "Merge" or Processing Toolbax: GDAL -> Raster Miscellaneous -> Merge
  2. Input layers = Select "Kansas_4326-84-hd-nowater@1" and Kansas_4326-84-hd-water.tiff
  3. Output data type = "byte".
  4. Under "Merged" choose a new filename to save the new raster to. We'll use Kansas_4326-84-hdwater.tiff.
   Click "Run" to generate the new merged hi resolution raster layer.

Setting up Virtual Planet Builder using docker

Generating the scenery uses an OSG tool called Virtual Planet Builder. Fortunately, there is a docker image so you don't need to build this yourself. Instead it will run as a container inside docker.

To set this up:

  1. Install Docker on your platform.
  2. Create an account on https://hub.docker.com/. (Note that you will need to click on an email verification link before you can log in for the first time)
  3. On your own machine, log into docker, and then pull the docker image
docker login

docker pull flightgear/ws30-vbp-generator:latest

You now have a docker image on your own machine you can run.

Running the Container

To generate terrain you need to run the Virtual Planet Builder tool within the container.

Firstly, get the container running from the directory containing your data and output directories: 

docker run --rm --mount "type=bind,source=`pwd`/data,target=/home/flightgear/data,readonly" --mount "type=bind,source=`pwd`/output,target=/home/flightgear/output" -it flightgear/ws30-vbp-generator:latest /bin/bash

You should now find yourself in a bash shell within your container. You should see data and output directories with are linked to the directories you created earlier: 

flightgear@ddcac77f7d5e:~$ ls
data  output

Generating the Terrain

Once the data is pre-processed, we can generate 1x1 tiles of terrain. For example: 

osgdem --TERRAIN \
--image-ext png
--no-interpolate-imagery \
--disable-error-diffusion \
--geocentric \
--no-mip-mapping \
-t /home/stuart/FlightGear/VPB/data/scratch/corine_WGS84.tif \
-d /home/flightgear/data/SRTM90/srtm_34_02.tif \
-d /home/flightgear/data/SRTM90/srtm_35_01.tif \
-d /home/flightgear/data/SRTM90/srtm_35_02.tif \
-d /home/flightgear/data/SRTM90/srtm_36_01.tif \
-d /home/flightgear/data/SRTM90/srtm_36_02.tif \
-b -4 55 -3 56 \
--PagedLOD \
-l 7 \
--radius-to-max-visible-distance-ratio 3 \
-o /home/flightgear/output/vpb/w010n50/w004n55/ws_w004n55.osgb

Running osgdem is in more detail here: Virtual Planet Builder#Running VPB.

After doing this you should have a file output/vpb/w010n50/w004n50/ws_w004n50.osgb, plus a host of sub-directories. This is a 1x1 tile of terrain.

to leave the container simply type exit.

Creating line features

Line features such as roads, railways and rivers are simply stored as a series of lat/lon points in an STG file using a specific STG verb. We get the data of this from Openstreetmap, and we have some simple scripts to generate the files.

The scripts are already included in the docker image at /home/flightgear/scripts/.

Running the scripts

The genroads.py script generates all line features. Simply set the output directory and the lat/lon box. Note that the output directory should be a "Terrain" directory. E.g. 

./scripts/genroads.py ./output/Terrain -4 55 -3 56

Packaging the Scenery

Once you have the terrain and line features they should be packaged in a scenery directory in vpb and Terrain sub-directories respectively. E.g. 

MyCoolScenery/Terrain
MyCoolScenery/vpb

It is good practise to document the data sources used in scenery generation. Some source licenses require attribution of the original data source for anything derived, published or distributed.

To assist in fulfilling these license obligations, you can create a source.xml file in the scenery directory which includes attribution information. This will then be available from within the simulator under Help->Scenery Sources, and may fulfil the attribution requirements of your license. Note that you are responsible for fulfilling any license requirements from the data, not FlightGear.

The format of the file is straightforward: 

<?xml version="1.0"?>
<PropertyList>
    <source>
        <name>Corine Land Cover (CLC) 2018, Version 2020_20u1</name>
        <link>http://web.archive.org/web/20221112175615/https://land.copernicus.eu/pan-european/corine-land-cover/clc2018?tab=metadata%2A</link>
        <license>GMES Open License</license>
    </source>
    <source>
        <name>NASADEM Merged DEM Global 1 arc second V001</name>
        <link>https://www.earthdata.nasa.gov/</link>
        <license>Public Domain</license>
    </source>
    <source>
        <name>OpenStreetMap</name>
        <link>https://www.openstreetmap.org/copyright</link>
        <license>Open Data Commons Open Database License</license>
    </source>
</PropertyList>

Running FlightGear with the new WS3.0 Terrain

To test the new terrain, simply include the appropriate scenery directory (e.g. MyCoolScenery) in in your scenery path and run FlightGear with the --prop:/scenery/use-vpb=true to enable WS3.0.

Retrieved from "https://wiki.flightgear.org/w/index.php?title=Howto:Create_WS3.0_terrain&oldid=137458"