Experimental terrain engine

IMPORTANT: Some, and possibly most, of the features/ideas discussed below are likely to be affected, and possibly even deprecated, by the ongoing work on deprecating TerraGear favor of VPB (Virtual Terrain Builder). Please see: Post FlightGear 2020.2 LTS changes for further information You are advised not to start working on anything directly related to this without first discussing/coordinating your ideas with other FlightGear contributors using the FlightGear developers mailing list. talk page.

Experimental Terrain Engine

Started in	09/2016
Description	Experimental Terrain Engine
Maintainer(s)	psadro_gm
Contributor(s)	psadro_gm (since 09/2016)
Status	Under active development as of 10/2016

Goals

The new experimental terrain engine main goals are:

1. a shader based landclass engine utilizing raster encoded fileds to identify landclass, and smoothness of transitions between them
2. a regular mesh that can be generated on the fly from standard gdal raster images
3. utilize the OpenSceneGraph Database Pager for handling multiple levels of detail (see PagedLOD).

Digital Elevation Modem (DEM )

Tools to created LOD tile hierarchy

demconvert can create highest LOD based on .HGT files from viewfinderpanorama
demconvert can create lower levels of detail based on higher LODs ( simplifies via gdalwarp API )

NOTE: There may be an issue generating low LOD in the tool, or loading them in Flightgear. I am seeing tile border 'valleys' from space.

pagedLOD (.SPT) hierarchy

pagedLOD database is created at subsystem init time in Scenery/terrain_pgt class

NOTE: There may be an issue generating low LOD in the tool, or loading them in Flightgear. I am seeing tile border 'valleys' from space.

FDM groundcache init

On startup, the terrain is initialized via subsystem init.
This populates the pagedLOD tree for the whole world. What it does NOT do, is start paging in terrain.

1. Need to set camera location
2. Need to start the pager ( main loop )

The FDM is also started via subsystem init, but queries the terrain engine until scenery is available given a certain radius around the aircraft. On first attempt, the groundcache is initialized with the first LOD ( very rough ) giving incorrect altitude.

The entire scenegraph is loaded before FDM initialization, and the database pager is traversing the pagedLOD nodes, but the viewpoint is such that all of the pagedLOD nodes are culled. It looks like aircraft final positioning ( or a switch to cockpit view ) is starting the actual terrain loading. I'll need to move up the camera positioning to start terrain loading before fdm is initialized.

Ugh - the original altitude was a hack I added to get moving... I can get a more accurate position by querying the DEM, but this isn't accurate enough - I'm still to high off the ground for a good trim, and I bounce off the ground on init - breaking new Cessna's gear.

I will try to implement the following:

1. get rough estimate to position the aircraft viewPos - DONE!
2. use scenery_available() to wait for the highest level pagedLOD to load - next
3. use the scenery intersect ( as before ) to get exact contact point

Scenery Available

Subclass Terrain nodes in the scenegraph to express terrain LOD and area covered in lon/lat?

Integration

1. modular terrain engine.

The user should be able to select the legacy terrain engine ( tile cache based ), or the new engine based on a property. In the flightgear/Scenery source, the Scenery object instantiates the selected terrain engine. The terrain object is an abstract base class, and new terrain engines derive from this class.

The following APIs are defined for all subsystems inheriting from SGSubsystem _(doxygen)

See also Howto:Create new subsystems.

    // Implementation of SGSubsystem. - called from Scenery
    virtual void init ( osg::Group* terrain ) = 0;
    virtual void reinit() = 0;
    virtual void shutdown () = 0;
    virtual void bind () = 0;
    virtual void unbind () = 0;
    virtual void update (double dt) = 0;

These APIs are called from various other flightgear modules, like the FDM or Nasal (scripting). They are used to query altitude and materials at given locations

    /// Compute the elevation of the scenery at geodetic latitude lat,
    /// geodetic longitude lon and not higher than max_alt.
    /// If the exact flag is set to true, the scenery center is moved to
    /// gain a higher accuracy of that query. The center is restored past
    /// that to the original value.
    /// The altitude hit is returned in the alt argument.
    /// The method returns true if the scenery is available for the given
    /// lat/lon pair. If there is no scenery for that point, the altitude
    /// value is undefined. 
    /// All values are meant to be in meters or degrees.
    virtual bool get_elevation_m(const SGGeod& geod, double& alt,
                                 const simgear::BVHMaterial** material,
                                 const osg::Node* butNotFrom = 0) = 0;

    /// Compute the elevation of the scenery below the cartesian point pos.
    /// you the returned scenery altitude is not higher than the position
    /// pos plus an offset given with max_altoff.
    /// If the exact flag is set to true, the scenery center is moved to
    /// gain a higher accuracy of that query. The center is restored past
    /// that to the original value.
    /// The altitude hit is returned in the alt argument.
    /// The method returns true if the scenery is available for the given
    /// lat/lon pair. If there is no scenery for that point, the altitude
    /// value is undefined.
    /// All values are meant to be in meters.
    virtual bool get_cart_elevation_m(const SGVec3d& pos, double max_altoff,
                                      double& elevation,
                                      const simgear::BVHMaterial** material,
                                      const osg::Node* butNotFrom = 0) = 0;

    /// Compute the nearest intersection point of the line starting from 
    /// start going in direction dir with the terrain.
    /// The input and output values should be in cartesian coordinates in the
    /// usual earth centered wgs84 coordinate system. Units are meters.
    /// On success, true is returned.
    virtual bool get_cart_ground_intersection(const SGVec3d& start, const SGVec3d& dir,
                                              SGVec3d& nearestHit,
                                              const osg::Node* butNotFrom = 0) = 0;

These APIs are called to determine the state of the scenery, or to schedule new scenery Scheduling new scenery from flightgear may or may not be necessary - depending on the engine.

    /// Returns true if scenery is available for the given lat, lon position
    /// within a range of range_m.
    /// lat and lon are expected to be in degrees.
    virtual bool scenery_available(const SGGeod& position, double range_m) = 0;

    // tile mgr api
    virtual bool schedule_scenery(const SGGeod& position, double range_m, double duration=0.0) = 0;

Finally, the effect engine notifies the terrain engine when the material library has changed. This occurs when moving from one regional texture definition to another.

Note  how this can be done without being under flightgear control is TBD. We would need to setup the database pager with this info on terrain initialization.

    // tile is in a different regional texture definition.
    virtual void materialLibChanged() = 0;

Enabling the engine

By default, the current engine is the tile-manager based terrain (.stg/.btg files) To enable and tune the new engine, the following properties are added:

Property	Type	Value(s)	Description
/sim/scenery/engine	String	"tilecache"	Select normal engine
		"pagedLOD"	Select new terrain engine
/sim/scenery/lod-levels	String	"1 3 5 7"	a list of levels - between 1 and 9 - Selects which levels to use.
/sim/scenery/lod-range-mult	String	"2"	the range in which to move to/from new LOD. The bounding radius of the tile is multiplied by this value to calculate the range from the center of the tile.
/sim/scenery/lod-level-res	String	"3 5 9 17 33 49 65 93 129"	"This list defines the resolution of each LOD level. In this instance, level 1 is a 2 x 2 grid. (3 samples on both x and y axis)

The Digital Elevation Model

The actual DEM is a collection of levels (independent of the LOD level) Multiple DEM levels are not required, but highly advised. The ViewFinderPanorama DEM is ~50GB. The entire planet is always loaded, so terrasync would need to download all 60 GB the first time to be able to model the whole world. Each LOD request opens a 'session' with the DEM. The DEM will pick an appropriate DEM level to fulfill the session request based on the size of the request. Sessions are cached, so multiple elevation requests using the same session will not cause addition file open/reads.

For example, if you request anything between 1/8 x 1/8 degree and 1 x 1 degree sessions, you'll get the highest resolution DEM ( as vfp tiles are 1x1 ). If you request 12x12 degrees, you'll get a significantly reduced DEM resolution - as you are most likely generating a tile VERY far away from the viewer.

Currently, only one Mesh Hierarchy is supported. Paths are searched via the normal fg-scenery paths list. The first scenery path to contain a valid hierarchy is selected. The directories are searched in scenery-path/DEM/

Under this directory are sub-directories named level_xx. Within each subdirectory is a description file (deminfo.txt), along with each tile. An example description file is for the original .hgt files from viewfinder panoramas:

1 1 1201 1201 hgt

The format is size-lon( in degrees ) size-lat( in degrees ), res-lon, res-y, file type

Currently, I have generated 5 levels from the vfp data. The largest tile size being 60x60 degrees - 18 tiles cover the whole planet. Current disk space for the levels is as follows:

level dir	size of tile (degrees)	size on disk
./level_01	1x1	50G
./level_02	2x2	4.8G
./level_03	4x4	1.2G
./level_04	12x12	149M
./level_05	60x60	6.8M

Note that level_01 is uncompressed .hgt format. The generated levels are compressed geotiff format. Final solution will be to use geotiff for level 1 as well. I expect the first level to be ~20GB, as each level reduces x and y resolution by a factor of 2.

Trying it out

I have uploaded a small DEM section covering Hawaii for testing: the tarball is here: hawaii_dem. Extract the files in a subdirctory, and add this to your fg-scenery path:

Here's an example script I use:

LOG_CLASS=--log-class=terrain,flight
LOG_LEVEL=--log-level=info

FGROOT=--fg-root=${HOME}/Development/flightgear-fgdata/
SCENERY=--fg-scenery=${HOME}/Development/hawaiidem/

# HAWAII
START_POS="--lon=-158.0 --lat=21.0 --altitude=4000"

TERRAIN_ENGINE="pagedLOD"
LOD_RES="1"
LOD_RANGEMULT="2"
LOD_LEVELS="1 2 3 4 5 6 7 8 9"
LOD_TEXTURING="bluemarble"
  or
LOD_TEXTURING="debug"
ZFAR="9600000"

fgfs ${FGROOT} ${SCENERY} ${LOG_LEVEL} ${LOG_CLASS} ${START_POS} --timeofday=noon --aircraft=ufo --prop:double:/sim/rendering/camera-group/zfar=${ZFAR} --prop:/sim/scenery/engine="${TERRAIN_ENGINE}" --prop:/sim/scenery/lod-levels="${LOD_LEVELS}" --prop:/sim/scenery/lod-res="${LOD_RES}" --prop:/sim/scenery/lod-range-mult="${LOD_RANGEMULT}"

Status / Gallery

psadro_gm has the normal calculations, now. Everything is being generated on the fly. psadro_gm sees very little CPU usage. Still lot's to do - e.g. need to sample 1 extra pixel in the DEM so edges of tiles get correct normals ( you can see the edges )

Also, psadro_gm hasn't created skirts, yet, so there are gaps between levels.

psadro_gm will add to Mathais' tunables to select grid spacing at each LOD level, to see if we can minimie the popping. Note: the texture is just 1 4096x4096 sheet for the whole planet. It'll be nice to see what we can do with procedural texturing.

References

References