Osm2city.py
| Started in | 11/2013 |
|---|---|
| Description | Improved autogen support for FlightGear using OSM data |
| Contributor(s) | radi, vanosten, Soitanen, portreekid |
| Status | Under active development as of 02/2016 |
| Topic branches: | |
| $FG_SRC | https://gitlab.com/osm2city/osm2city/ |
| fgdata | https://gitlab.com/osm2city/osm2city-data/ |
| OpenStreetMap use in FlightGear |
|---|
Take openstreetmap floorplans, create 3d buildings ready for use in FlightGear.
Development repository on gitorous. Written in python 2.7, developed on Linux. It should also run on Mac OSX. Windows users, see below.
It's at a rather early stage of development. There's no GUI, everything is controlled by an input file. But it produces realistic city layouts (after all, it uses realistic data). The whole process -- from scratch to flying in FG -- for a city the size of LOWI takes perhaps one hour, including maybe a total of 15 mins manual work.
It has been tested with Dresden, Germany (EDDC) and Innsbruck, Austria (LOWI). Both areas are now populated with about 50,000 buildings. Rendering this in FG is quite demanding. The FG process eats ~2.8GB RAM when flying in those areas, the download is ~50 MB each.
Status 04/2014
Following Mathias' suggestion at FS Weekend 2013, I've now changed the code such that it merges all buildings per (osm2city) tile into one object, reducing the number of drawables from O(10k) to O(10). That indeed gives a nice speed-up. In fact, I'm overwhelmed by what's possible now -- here's a scene looking down at LOWI from FL300 showing 60k buildings. I've lost LOD and textures underway, but will fix that soon. Plain Scenery 2.0 gives 19 fps on i7 Intel HD 5000 2560x1440. With buildings framerate goes down to 14.
The second scene, approaching LOWI from the east, went from 30 fps without buildings down to 20 fps.
Status 10/2013
Please see the development repository at: https://gitorious.org/fg-radi/osm2city
Currently data is processed offline beforehand. Basically, it parses the OSM
xml, generates a list of building outlines, discards some based on their area,
simplifies the outlines, clusters them into ~500x500m blocks and different LODs,
then writes .ac, .xml, and .stgs. OSM parsing is by far the most expensive,
easily taking 10 minutes for 50k buildings. Once that's done, the remaining parts take maybe 1 minute in total.
(Some optimization gave a huge speedup).
At the moment, the code knows only the floor plans. No streets, no runways, no land-use. But it'll certainly process such data in the future, and then could use some heuristics (some OSM buildings are labeled "Terminal 1" or so) to apply terminal/hangar textures to buildings at airports. This way we could rather easily populate some airports with 'semi-generic' terminal/hangar buildings.
Features
- reads buildings from OSM. Honors height and level tags.
- reads existing .stg, won't place OSM building if there's a static model nearby
- reads pre-calculated terrain elevation: places buildings at correct elevation
- LOD animation based on building height and area (see below)
- cluster a number of buildings into a single .ac files. Clusters overlap to alleviate sharp LOD borders
- complex sloped roof generation (using Olivier Teboul's implementation of the straight skeleton algorithm)
- texture manager: DB of facade/roof textures
- ~10 different, hi-res facade textures, some roof textures
- find matching texture for given building (number of levels, modern/old building, etc)
- find matching roof texture for given facade texture
- basic lightmap support
- obstruction lights on tall buildings
- shows statistics on processed buildings
- writes .ac, .xml, .stg
LOD Scheme
FlightGear knows three standard LOD: bare, rough and detail. 'Bare' sets the drawing distance of the terrain, which may easily be 50 km or more. Drawing buildings 50 km out makes little sense (unless they are really tall), so we shouldn't use this level here. Of the remaining two standard levels, 'rough' is used for large and/or tall buildings, and 'detail' for smaller ones.
In the near future, osm2city will generate more complex roof shapes. This will increase the poly count further, and I believe it's a good idea to use another LOD 'roof' for complex roofs. Fortunately, we can change every aspect of FlightGear, and adding another LOD is easy. Use the FG command line
--prop:double:/sim/rendering/static-lod/roof=2000
to set the distance for 'roof' to 2 km. If you want to adjust it via FG's GUI, copy static-lod.xml (from osm2city's git repo) to $FGDATA/gui/dialogs.
Planned Features
- make command line interface. Currently, everything is hard-coded.
- 2013/08: now uses parameters file, thanks to forum user vanosten
also parse OSM 'relation' tag and create 'buildings with holes'(done)more complex roof shapes, put these into separate LOD(done)- 2013/10 started: simplify/automate elevation probing by using fgelev
- more complex facade generation. Currently, all sides get same texture
- Rembrandt lighting
- put shared models if/where OSM indicates so: gas stations...
- use residential/industrial/commercial tags/areas. ATM, all is residential.
geometry cleanup, simplify too complex buildings (what is "too complex"?)(done)- Batch processing of greater areas including downloads (PortreeKid)
- use more LOD levels, write them to different .ac so users can easily reduce building density, therefore improve performance
- 2013/05: started: now roofs go into separate LOD
- put large buildings into one ac, sort/rate buildings by stand-out-ness
- then ship light/med/full .stg
- mid-term: develop this into a city-engine that procedurally generates a city based on OSM roads.
- long-term: integrate into FG to do all this on the fly.
Ideas
- nearby hi-rise objects of similar shape get same texture? Probably too special a feature.
- bump map/reflection effects?
- a mode that generally uses shared models? If we find a shared models of matching floorplan, use this instead of creating a new, individual building? Will break clusters.
- a terrasync-like thing? Users get OSM buildings
(where available) on-the-fly? Might be quite costly in terms of runtime performance and bandwidth. The OSM download (buildings only!) is ~40MB for the 25x25km LOWI area.
Install
- dependencies: Install the following packages (names from Debian packages):
python-imposm python-imposm-parserpython-numpy python-shapely python-matplotlib python-scipy python-pil
- update 04/2014: vanosten replaced the imposm parser by a pythons SAX
- get osm2city from gitorious
- add the directory with osm2city modules to your PYTHONPATH (unless your PYTHONPATH already contains .)
- copy elev.nas to $FGDATA/Nasal/
- Change the first two lines of elev.nas to match specific directories
with write access (e.g. $FG_HOME/Export). See IORules and $FG_HOME
Install on Windows
osm2city is pure python , but it depends on imposm.parser for OSM parsing. Some users report problems with imposm.parser on Windows -- YMMV. In the future, we might replace imposm.parser with a standard XML parser.
Install the following packages.
http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy
http://www.lfd.uci.edu/~gohlke/pythonlibs/#shapely
http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy-stack
Workflow
There are five main steps:
- get OSM data
- adjust input file
- create elevation grid, run FG to probe elevation data
- run osm2city
- install the generated scenery
We'll now walk through these steps in detail:
get OSM data
- decide on a region you want to populate. Get its lon/lat coordinates.
- create a project directory. We will use LOWI/ in this example.
- get corresponding OSM data in its native .xml format.
There are several options to get OSM data. You may find the XAPI Query Builder helpful, it will generate a download URL that you can fetch with wget:
wget -O buildings.osm http://www.overpass-api.de/api/xapi?map?bbox=11.16898,47.20837,11.79108,47.38161
Put this data into LOWI/buildings.osm
Adjust input file
- Now adjust the input file. Copy the template params.ini to LOWI/. Edit this file, adjusting (at least) the following (check comments in params.ini):
- PREFIX = LOWI (the project directory)
- BOUNDARY_EAST, BOUNDARY_NORTH, BOUNDARY_SOUTH, BOUNDARY_WEST
- OSM_FILE = buildings.osm
- set PATH_TO_SCENERY to a standard FG_SCENERY path. Suppose the area you want to populate is /home/user/fgfs/scenery/Objects/e010n40/e011n47/, PATH_TO_SCENERY would then be /home/user/fgfs/scenery/ .
- you might want to set MAX_OBJECTS to a small number (say, 100) for your first try. This will limit the number of buildings parsed, and speed up the whole process.
Probe elevation
- probe elevation for this region:
enter lon/lat for your area in the call to raster_glob() near tools.py:241(FIXED: now read from input file)- run tools.py, this will create a file elev.in. Copy elev.in to $FGDATA/Nasal/
- tools.py will tell you to hide a certain scenery/Objects/... folder, to prevent probing elevation on top of existing objects
- run FG, open debug->nasal console, enter: elev.get(), press execute. Might take a long time, depending on the area scanned. This will write elevation data to /tmp/elev.xml (which is actually just plain 5 column data: lon,lat,x,y,elevation). Put elev.xml into LOWI/
Run osm2city
- run osm2city.py. Parsing OSM data
again takes quite looong (10 minutes or more for 50k buildings)is now pretty fast, but the result is cached to file buildings.pkl. Next startup will bemuch faster if you move buildings.pkl to LOWI/buildings.pkl andeven faster if you set USE_PKL = 1.
If all goes well, files like LOWIcity0101.ac and their corresponding .xml, along with .stg files, were created and moved to the correct location in $FG_SCENERY
Improve frame rate
- adjust LOD ranges in FG
- decrease MAX_OBJECTS, increase LOD_*
(- use larger/smaller tiles? Tradeoff between distance calculations/xml parsing and GPU)
Contributing
You know some python? Or you're keen on writing docs? Or just want to use osm2city, but got stuck somewhere? Contact radi on the FG forum. None of the above? Take pictures of buildings where you live! I'm especially interested in south-east asian style architecture, as I'm planning to populate Hong Kong Kai Tak (VHXX) in the future.