Rendering system improvements
This page is intended to be a talk about possible technical changes to the FlightGear rendering engine to allow better appearance and performance. Please keep feature requests etc on some other page.
Current state
This section describes how the scenery is currently rendered.
Rendering system
Flightgear has mostly one pass rendering system, where the whole scenery is rendered in one pass. Lighting is applied directly when rendering, and currently, there is support for only one light (the sun).
Cameras
By default, FlightGear has at least three cameras: Near camera, far camera, and GUI camera. The first two are used to render the scenery and the last one is used to render all(?) 2D stuff, like menus.
Scenery cameras render so that the far camera is rendered first and then the rear camera. The transition is at 100 meters by default (IIRC). Near camera has near at around 1cm and the far camera has far at ~12km.
The near/far camera system is in place because of buffer precision issues, which appear if one wants to render things very near and very far. See [1].
Rendering order
Scenery is rendered in following order (this is from my memory, there might be some errors....)
- Skydome
- Sky itself
- Stars
- Moon and sun halos
- Moon and sun
- Terrain and opaque objects
- Clouds (3D) (Back-to-front) (*)
- Translucent objects (Back-to-front) (**)
- Lights
- Cockpit (***)
(*) 2D clouds are rendered differently, the rendering order is from bottom to top and top to bottom, the render bins do not very well match 3D cloud bin leading to artifacts. (**) Translucent objects should be rendered in the same bin as 3D clouds, but currently, they are rendered after clouds, so they can be seen through the clouds which looks wrong. (***) The cockpit mostly fits the near camera and that's why it's rendered last.
This order leads to a significant amount of overdraw, the terrain is drawn over the sky and cockpit is then drawn over the terrain. With heavy shaders this overdraw is slow.
Other rendering systems
One pass multi lights
In this method, the scene is rendered in one pass and shaders/lighting models take all lights into account.
- No overdraw
- Limited amount of lights -> Suits for outdoor scenes with only sun/moon
- OpenGL fixed pipeline supports 10 lights
- With shaders it depends on the available uniforms/varyings. Depending on the shader complexity, maybe 4 lights would be possible
- Lighting model depends on the object, can have various different models
- Lighting is calculated for all pixels for all lights, even if they are unlit
- Must do sorting by light volume to select best lights
- Works with translucent objects
Multi-pass one light
Render only one light at the time, rendering the whole scene with additive blending for all lights.
- Infinite amount of lights
- Huge amount of overdraw
- Needs heavy sorting and culling to improve the speed
- Works with complex shaders
- Lighting model depends on the object, can have various different models
- Depending on the culling, lighting is only calculated for objects that are in the light
- Has issues with translucent objects
Deferred shading
In deferred shading, the whole scene is first rendered with a simple shader that writes diffuse color, normal, depth, and other parameters to multiple textures (G-buffer). Then, lighting is calculated with those parameters. See for example [2] and [3].
- Infinite amount of lights
- One big light is about as heavy as many small ones
- Suits for indoor scenes with multiple small lights
- Rendering target precision is an issue (especially normal buffer)
- Large memory footprint
- Geometry pass shader is/should be simple and fast
- One lighting shader for all geometry -> limited lighting models (maybe 4 different lighting types)
- No overdraw in the lighting pass
- Does not work with translucent objects
Mixed
A mixed system combines some of the above to give good performance and visual appearance. I think something like this could suit our needs:
- Render scene with one pass, use two lights: sun and moon
- Render diffuse color and depth (and normal?) to texture targets -> allows post-processing effects
- Render all other small lights using those textures -> the light cone approach
- Texture targets allow recovering the original color during night
- Storing normal and depth allows rendering smooth light in the light cone
- Light cones can be easily culled by visibility
- Allows various lighting models in the sun/moon pass, light cones have all only basic lighting model
- Light cones need stencil operations and changing the front/backface culling -> might be slow?
- Fog rendered as a post-process quad (after opaque geometry) -> same fog for every object
- Translucent objects?
Rendering systems in other projects
Currently, just a placeholder to write some notes about other projects and what we might learn from them.
Outerra
- Continuous noise coordinates -> continuous noise function
- Slope, curvature etc as vertex attributes
- Terrain refinement on load time and on shader
- Trees -> Billboards -> Baked texture
- Limited amount of materials (4?) -> Easy to mix on shader level
I don't really understand how are roads implemented in Outerra, some kind of shader level spline interpolator, but how can the number of splines be large enough?
Links:
- About procedural terrain refinement: [5]
- Horizontal displacement: [6], [7]
- Logarithmic z-buffer: [8]
- Floating point z-buffer: [9]
X-Plane
- X-Plane 10 uses deferred shading for global illumination, see link below.
- Seven LOD levels for clouds, from 1mx1m upwards
Links:
- Auto-variation in terrain textures: [11], [12]
- Visiblity (how far is far enough?): [13]
- Global illumination: [14], videos [15]
- Object performance: [16], [17]
Other things
New rendering order
To limit the overdraw, I suggest following rendering order:
- Opaque parts of the cockpit
- Terrain
- Skydome
- Clouds (3D)/translucent objects
- Translucent parts of the cockpit
With near/far cameras there might need to use stencil operations or similar to limit rendering only to parts visible from the cockpit.
Get rid of near/far camera
Ways to get rid of near and far cameras.
Custom z function
Use custom z function in the shader, for example log(z).
- Better precision -> no z fighting
- Cannot use early z kill in the shader -> All pixels are evaluated -> Slow
- Shaders can easily modify z value -> relief mappign etc
An interesting read: [18]
Floating point z-buffer
- Link to NVidia extension, that allows unclamped depth values to be written into the depth buffer. It seems to be a draft and not accepted, though: [19]
- This extension might be related too: [20]
- Tim Moore has done some things with fp-buffer: [21] and an example:[22]
Cockpit only near camera
Render only the inside view using near the camera, everything else with far camera. Put the transition to somewhere around 0.5-1.0m.
- Is the precision then sufficient?
- No z-fighting in the instruments
- Can render everything else in a single pass
- Might need to use stencil operations with cockpit if the cockpit is rendered first
Ideas to boost the performance of the renderer.
- If an object uses only one texture image for one object and only one material and no animations
- => Use a smart combining algorithm to combine hundreds of similar objects to one object ([23])
- Only vertices of the original array, copy face array for each clone, then a BIND_PER_PRIMITIVE attribute containing the center of the clone and a quaternion of its rotation
- Shader places the object correctly, no state changes during the rendering of ~100 objects => fast
- Similar system is in use for the trees and clouds
- Most likely breaks height-over-terrain and another collision checking
Grass and vegetation
- Make tree blobs larger, i.e. more trees on a single object
- Remove the second pass of the tree rendering effect if possible
Clouds
- Volume rendering [26]
- Render far away clouds to a texture and then render that image as a billboard
- Needs to refresh the cloud texture every now and then
Lighting models
- Rendering of wet materials [27]
- Apply a more "metallic" shader as a default for aircraft models? Link?
- Atmospheric scattering, at least for skydome, maybe for terrain too (Post process if using mixed-mode?)
- Different models are described here (taken from Shader requests): [28]. A simple summary:
- Plastic: Phong or Blinn-Phong (currently the only model)
- Metals and Plastics: Cook-Torrance, Strauss
- Rough surfaces: Oren-Nayar
- Clothing, anisotropic surfaces: Ward
- Other models for a probably wide range of materials: Ashikhmin-Shirley