|Description||Improved autogen support for FlightGear using OSM data|
|Contributor(s)||radi, vanosten, Soitanen, portreekid|
|Status||Under active development as of 02/2016|
|OpenStreetMap use in FlightGear|
This Python script takes OpenStreetMap floorplans and creates 3d buildings ready for use in FlightGear.
It's at a rather early stage of development. There's no GUI, everything is controlled by an input file. But it produces realistic city layouts (after all, it uses realistic data). The whole process -- from scratch to flying in FG -- for a city the size of LOWI takes perhaps 30 min, including maybe a total of 15 min manual work.
It has been tested with Dresden, Germany (EDDC) and Innsbruck, Austria (LOWI). Both areas are now populated with about 50,000 buildings. Rendering this in FG is quite demanding. The FG process eats ~2.8GB RAM when flying in those areas, the download is ~50 MB each.
- 1 News
- 2 Features
- 3 Planned Features
- 4 LOD Scheme
- 5 Ideas
- 6 Install
- 7 Install on Windows
- 8 Workflow
- 9 Support and troubleshooting
- 10 Improve frame rate
- 11 Contributing
- 12 Related content
- textures reside in a separate 
- the recommended elevation probing method is ELEV_MODE=FgelevCaching
- roads.py is now quite usable
- SKIP_LIST can contain OSM_IDs
I'm currently adding textured roads, railroads, intersections and bridges.
| the shader is now on osm2city's git (in fgdata/). Again -- all credit goes to Emilian. He is improving the shader side, I will continue on the model side. So this is very much work in progress.
The shader requires the default renderer. No ALS/Rembrandt yet.
Get the LOWI roads pack (including the shader) here:
The fgdata/ folder goes to your $FGDATA.
In roads.xml enable either the lightmap or the traffic shader.
Following Mathias' suggestion at FS Weekend 2013, I've now changed the code such that it merges all buildings per (osm2city) tile into one object, reducing the number of drawables from O(10k) to O(10). That indeed gives a nice speed-up. In fact, I'm overwhelmed by what's possible now -- here's a scene looking down at LOWI from FL300 showing 60k buildings. Plain Scenery 2.0 gives 19 fps on i7 Intel HD 5000 2560x1440. With buildings framerate goes down to 14.
The second scene, approaching LOWI from the east, went from 30 fps without buildings down to 20 fps.
Currently data is processed offline beforehand. Basically, it parses the OSM
xml, generates a list of building outlines, discards some based on their area,
simplifies the outlines, clusters them into ~500x500m blocks and different LODs,
then writes .ac, .xml, and .stgs.
OSM parsing is by far the most expensive,
easily taking 10 minutes for 50k buildings. Once that's done, the remaining parts take maybe 1 minute in total.
(Some optimization gave a huge speedup).
At the moment, the code knows only the floor plans.
No streets, no runways, no
land-use. But it'll certainly process such data in the future, and then could
use some heuristics (some OSM buildings are labeled "Terminal 1" or so) to apply
terminal/hangar textures to buildings at airports. This way we could rather easily populate some airports with
'semi-generic' terminal/hangar buildings.
- reads buildings from OSM. Honors height and level tags, reads relations ('buildings with holes')
- reads existing .stg, won't place OSM building if there's a static model nearby
- reads pre-calculated terrain elevation: places buildings at correct elevation
- simplify/automate elevation probing by using fgelev
- LOD animation based on building height and area (see below)
- cluster a number of buildings into a single .ac files. Clusters overlap to alleviate sharp LOD borders
- complex sloped roof generation (using Olivier Teboul's implementation of the straight skeleton algorithm)
- texture manager: DB of facade/roof textures
- ~10 different, hi-res facade textures, some roof textures
- find matching texture for given building (number of levels, modern/old building, etc)
- find matching roof texture for given facade texture
- basic lightmap support
- obstruction lights on tall buildings
- command line interface and parameters file (thanks to forum user vanosten)
- shows statistics on processed buildings
- writes .ac, .xml, .stg
(in random order)
- more complex facade generation. Currently, all sides get same texture Not done
- Rembrandt lighting Not done
- put a piece of matching ground texture around buildings ('garden')
- put shared models if/where OSM indicates so: gas stations... Not done
- geometry cleanup, simplify too complex buildings Done
- use residential/industrial/commercial tags/areas. ATM, all is residential. Not done
- geometry cleanup, simplify too complex buildings Done
- Batch processing of greater areas including downloads Done
- use more LOD levels, write them to different .ac so users can easily reduce building density, therefore improve performance
- mid-term: develop this into a city-engine that procedurally generates a city based on OSM roads. Not done
FlightGear knows three standard LOD: bare, rough and detail. 'Bare' sets the drawing distance of the terrain, which may easily be 50 km or more. Drawing buildings 50 km out makes little sense (unless they are really tall), so we shouldn't use this level here. Of the remaining two standard levels, 'rough' is used for large and/or tall buildings, and 'detail' for smaller ones.
Osm2city can generate complex roof shapes. This increases the poly count further, and I believe it's a good idea to use another LOD 'roof' for complex roofs. Fortunately, we can change every aspect of FlightGear, and adding another LOD is easy. Use the FG command line
to set the distance for 'roof' to 2 km. If you want to adjust it via FG's GUI, copy static-lod.xml (from osm2city's git repo) to $FGDATA/gui/dialogs.
- nearby hi-rise objects of similar shape get same texture? Probably too special a feature.
- bump map/reflection effects?
- a mode that generally uses shared models? If we find a shared models of matching floorplan, use this instead of creating a new, individual building? Will break clusters.
- a terrasync-like thing? Users get OSM buildings
(where available) on-the-fly? Might be quite costly in terms of runtime performance and bandwidth. The OSM download (buildings only!) is ~40MB for the 25x25km LOWI area.
1. dependencies: Install the following packages (names from Debian packages):
python-numpy python-shapely python-matplotlib python-scipy python-pil python-networkx curl python-enum34
3. add the directory with osm2city modules to your PYTHONPATH (unless your PYTHONPATH already contains . (the dot)) You can add
into your hidden configuration file .bashrc located in your home directory.
4. soft link osm2city-data/tex and osm2city-data/tex.src into osm2city/. Typically you will have to "adapt" and execute these commands.
cd osm2city ln -sf ../osm2city-data/tex.src ln -sf ../osm2city-data/tex
Then the sub-directory of osm2city will appears like that
osm2city/ tex.src/ tex/ osm2city.py textures.py ...
If you're running FG >= 3.3, or git newer than 10 Oct 2014, and have more than 4 GB RAM, you're done installing now. Otherwise, you will most likely need to set-up and use the old-fashioned way of elevation probing, which requires you to
5. copy elev.nas to $FGDATA/Nasal/
Install on Windows
In general, you'll want to follow the above install section. This section lists some additional Windows particularities.
Osm2city is pure python, so install Python if you have not already done so https://www.python.org/downloads/release/python-278/
|Caution Python 2.7 is recommended unless you're an experienced Python developer willing to update the Python scripts to support Python 3. Also pay attention to versions and cpu builds of the below helper files. Make sure they match your OS build and Python version.|
Install the following packages, then proceed with step 2. above.
There are five main steps:
- get OSM data
- adjust input file
- if neccessary, create elevation grid and run FG to probe elevation data.
- run osm2city
install the generated sceneryinstalls itself
We'll now walk through these steps in detail:
get OSM data
- decide on a region you want to populate. Get its lon/lat coordinates.
- create a project directory. We will use LOWI/ in this example.
- get corresponding OSM data in its native .xml format.
There are several options to get OSM data. You may find the XAPI Query Builder helpful, it will generate a template download URL that you can fetch with wget:
wget -O buildings.osm http://www.overpass-api.de/api/xapi?*[key=value][bbox=11.16898,47.20837,11.79108,47.38161]
Note that you either need to give a valid key=value pair, or remove [key=value] altogether. The alternative
wget -O buildings.osm http://www.overpass-api.de/api/xapi?map?bbox=11.16898,47.20837,11.79108,47.38161
will result in a file compatible with JOSM, at the cost of a larger download. If unsure, use the first option.
If you use Osmosis and cut the area with --bounding-box then you need to use completeWays=yes.
Put this data into LOWI/buildings.osm
Adjust input file
- Now adjust the input file. You can generate a commented default parameters file by
parameters.py -d > LOWI/params.ini
Edit this file, adjusting (at least) the following:
- PREFIX = "LOWI" (the project directory)
- BOUNDARY_EAST, BOUNDARY_NORTH, BOUNDARY_SOUTH, BOUNDARY_WEST
- OSM_FILE = "buildings.osm"
- set PATH_TO_SCENERY to a standard $FG_SCENERY path. This is the path where we'll get the elevation from. Most likely you'll want to use your TerraSync path here. Suppose the area you want to populate is /home/user/fgfs/scenery/TerraSync/Objects/e010n40/e011n47/, PATH_TO_SCENERY would then be "/home/user/fgfs/scenery/TerraSync" .
- set PATH_TO_OUTPUT to a different scenery path, e.g. "/home/user/fgfs/scenery/osm2city" . Osm2city will generate its objects there. First, this avoids TerraSync overwriting the generated scenery. Second, you can easily remove that folder in case you're not happy with the result. Make sure to include that path in your $FG_SCENERY.
- you might want to set MAX_OBJECTS to a small number (say, 100) for your first try. This will limit the number of buildings parsed, and speed up the whole process.
- you also need to set up elevation probing in the parameters file, described in the next section.
|Note Python does not recognize environment variables, please use full paths in the parameters file (no $HOME etc).|
Probe elevation for your region. Pick one of the three available modes for ELEV_MODE in params.ini. If you have enough memory (4GB should be fine, but YMMV) and run FG git later than 10 Oct 2014, try FgelevCaching. Failing that, try Manual. (portreekid, what's the 3rd mode good for?)
|Caution There are a lot of problems reported currently with fgelev. I can't really test for the next month or so. See troubleshooting or use ELEV_MODE = "Manual".|
- ELEV_MODE = "FgelevCaching"
- Set FG_ELEV to your fgelev executable. For example: FG_ELEV = "/usr/local/fg/bin/fgelev" or just FG_ELEV = "fgelev" if fgelev is already in your OS's path.
- Set PATH_TO_SCENERY (Missing terrain will result in error message)
- make sure the environment variable $FG ROOT points to your FG base path, (e.g. export FG_ROOT=/usr/share/games/flightgear/ on Linux)
- ELEV_MODE = "Manual"
- run tools.py (tools.py -f LOWI/params.ini), this will create a file elev.in. Copy elev.in to $FG_ROOT/Nasal/
- Open $FG_ROOT/Nasal/elev.nas in a text editor. Point the in variable to your elev.in file e.g. var in = "/path/to/$FG_ROOT/Nasal/elev.in".
- tools.py will tell you to hide a certain scenery/Objects/... folder, to prevent probing elevation on top of existing objects. Simply rename the folder
- run FG, open debug->nasal console, enter: elev.get_elevation(), press execute. Might take a long time, depending on the area scanned. This will write elevation data to /tmp/elev.out (which is actually just plain 5 column data: lon,lat,x,y,elevation). Put elev.out into LOWI/
- unhide, rename the objects folder
- ELEV_MODE = "Telnet"
- Run setup.py
- Start FG with the properties service on Port 5501
- Run tools.py, this will create a elev.in, start the nasal script and copy the elev.out back to your project directory
- Make sure you're in osm2city's directory and run
python osm2city.py -f LOWI/params.ini
The parsed OSM data is cached to file LOWI/buildings.osm.pkl which is used on next startup if you set USE_PKL = 1. If all goes well, files like LOWIcity0101.ac and their corresponding .xml, along with .stg files, were created and moved to the correct location in $FG_SCENERY
- copy or link the tex/ folder into the objects folder where these files were created
- run roads.py in the same fashion for roads
copy roads.eff to the same Objects\e0XXnXX\e0XXnXX\ directoryroads.eff is copied automatically
- Band-aid fix for the roads is to adjust their height in roads.py change AGL_ofs to a higher number e.g. 0.5
- the moving cars (TRAFFIC_SHADER_ENABLE = True) are only visible when you disable Atmospheric light scattering (ALS)
Support and troubleshooting
Time permitting, we can provide limited support on the Forums. Please don't use PM for support questions.
Before you ask a question, make sure you read this wiki page. If your question is not covered here, tell us as much as possible about the problem you're facing. Which steps have you taken to solve it? What is the exact error message? What operating system? Screenshots?
We will not cover general questions like "I don't know how to install XY" -- search the Internet.
CRITICAL:root:1173 -95.5767 37.0965 CRITICAL:root:fgelev returned <>, resulting in list index out of range. Did fgelev start OK (Record : 1173)?
The first line is the last input to fgelev. Try running fgelev directly:
/path/to/fgelev --expire 1000 --fg-root $FG_ROOT --fg-scenery $FG_SCENERY
then enter the line last sent to fgelev.
1173 -95.5767 37.0965
The response should look like
Buildings are generated, but they sit below the ground. I'm using ELEV_MODE = FgelevCaching.
Try running fgelev directly:
/path/to/fgelev --expire 1000 --fg-root $FG_ROOT --fg-scenery $FG_SCENERY
0 $lon $lat
where $lon and $lat are coordinates in the area you want to generate. See if it returns a sane value. If it does, double-check PATH_TO_SCENERY and $FG_ROOT.
Buildings/roads have no texture. I get errors "osgDB ac3d reader: could not find texture "tex/atlas_facades.png".
Copy or link the tex/ folder into the objects folder where the .stg files were created, such that you have $FG_SCENERY/Objects/eXXXnXX/eXXXnXX/tex/. Note that buildings might have been created in more than one folder, e.g. in e011n47 and e011n48. You must have a tex/ folder in each of them. On Windows it is sufficient to have one copy in $FG_ROOT.
ImportError: No module named calc_tile
Copy calc_tile.py into batch_processing folder
Improve frame rate
- adjust LOD ranges in FG
- decrease MAX_OBJECTS, increase LOD_*
(- use larger/smaller tiles? Tradeoff between distance calculations/xml parsing and GPU)
You know some python? Or you're keen on writing docs? Contact radi on the FG forum. None of the above? Take pictures of buildings where you live and create textures! I'm especially interested in south-east asian style architecture, as I'm planning to populate Hong Kong Kai Tak (VHXX) in the future.
- Areas populated with osm2city scenery
- AutoGen Scenery for FlightGear
- OpenStreetMap buildings
- Random Buildings
- Using OSM Vector Data in FlightGear
- OSM buidings EHLE
- osm2city.py development
- LOWI city buildings
- [Proposal] Autogen Cities using PixelCity
- Procedural buildings in OSM before part of Scenery