Rendering system improvements

This page is intended to be a talk about possible technical changes to the FlightGear rendering engine to allow better appearance and performance. Please keep feature requests etc on some other page.

Current state

This section describes how the scenery is currently rendered.

Rendering system

Flightgear has mostly one pass rendering system, where the whole scenery is rendered in one pass. Lighting is applied directly when rendering, and currently, there is support for only one light (the sun).

Cameras

By default, FlightGear has at least three cameras: Near camera, far camera, and GUI camera. The first two are used to render the scenery and the last one is used to render all(?) 2D stuff, like menus.

Scenery cameras render so that the far camera is rendered first and then the rear camera. The transition is at 100 meters by default (IIRC). Near camera has near at around 1cm and the far camera has far at ~12km.

The near/far camera system is in place because of buffer precision issues, which appear if one wants to render things very near and very far. See [1].

Rendering order

Scenery is rendered in following order (this is from my memory, there might be some errors....)

1. Skydome
   1. Sky itself
   2. Stars
   3. Moon and sun halos
   4. Moon and sun
2. Terrain and opaque objects
3. Clouds (3D) (Back-to-front) (*)
4. Translucent objects (Back-to-front) (**)
5. Lights
6. Cockpit (***)

(*) 2D clouds are rendered differently, the rendering order is from bottom to top and top to bottom, the render bins do not very well match 3D cloud bin leading to artifacts. (**) Translucent objects should be rendered in the same bin as 3D clouds, but currently, they are rendered after clouds, so they can be seen through the clouds which looks wrong. (***) The cockpit mostly fits the near camera and that's why it's rendered last.

This order leads to a significant amount of overdraw, the terrain is drawn over the sky and cockpit is then drawn over the terrain. With heavy shaders this overdraw is slow.

One pass multi lights

In this method, the scene is rendered in one pass and shaders/lighting models take all lights into account.

• No overdraw
• Limited amount of lights -> Suits for outdoor scenes with only sun/moon
  • OpenGL fixed pipeline supports 10 lights
  • With shaders it depends on the available uniforms/varyings. Depending on the shader complexity, maybe 4 lights would be possible
• Lighting model depends on the object, can have various different models
• Lighting is calculated for all pixels for all lights, even if they are unlit
• Must do sorting by light volume to select best lights
• Works with translucent objects

Multi-pass one light

Render only one light at the time, rendering the whole scene with additive blending for all lights.

• Infinite amount of lights
• Huge amount of overdraw
• Needs heavy sorting and culling to improve the speed
• Works with complex shaders
• Lighting model depends on the object, can have various different models
• Depending on the culling, lighting is only calculated for objects that are in the light
• Has issues with translucent objects

Deferred shading

In deferred shading, the whole scene is first rendered with a simple shader that writes diffuse color, normal, depth, and other parameters to multiple textures (G-buffer). Then, lighting is calculated with those parameters. See for example [2] and [3].

• Infinite amount of lights
• One big light is about as heavy as many small ones
• Suits for indoor scenes with multiple small lights
• Rendering target precision is an issue (especially normal buffer)
• Large memory footprint
• Geometry pass shader is/should be simple and fast
• One lighting shader for all geometry -> limited lighting models (maybe 4 different lighting types)
• No overdraw in the lighting pass
• Does not work with translucent objects

Mixed

A mixed system combines some of the above to give good performance and visual appearance. I think something like this could suit our needs:

• Render scene with one pass, use two lights: sun and moon
• Render diffuse color and depth (and normal?) to texture targets -> allows post-processing effects
• Render all other small lights using those textures -> the light cone approach
  • Texture targets allow recovering the original color during night
  • Storing normal and depth allows rendering smooth light in the light cone
  • Light cones can be easily culled by visibility
• Allows various lighting models in the sun/moon pass, light cones have all only basic lighting model
• Light cones need stencil operations and changing the front/backface culling -> might be slow?
• Fog rendered as a post-process quad (after opaque geometry) -> same fog for every object
• Translucent objects?

Rendering systems in other projects

Currently, just a placeholder to write some notes about other projects and what we might learn from them.

Outerra

[4]

• Continuous noise coordinates -> continuous noise function
• Slope, curvature etc as vertex attributes
• Terrain refinement on load time and on shader
• Trees -> Billboards -> Baked texture
• Limited amount of materials (4?) -> Easy to mix on shader level

I don't really understand how are roads implemented in Outerra, some kind of shader level spline interpolator, but how can the number of splines be large enough?

Links:

• About procedural terrain refinement: [5]
• Horizontal displacement: [6], [7]
• Logarithmic z-buffer: [8]
• Floating point z-buffer: [9]

X-Plane

[10]

• X-Plane 10 uses deferred shading for global illumination, see link below.
• Seven LOD levels for clouds, from 1mx1m upwards

Links:

• Auto-variation in terrain textures: [11], [12]
• Visiblity (how far is far enough?): [13]
• Global illumination: [14], videos [15]
• Object performance: [16], [17]

New rendering order

To limit the overdraw, I suggest following rendering order:

1. Opaque parts of the cockpit
2. Terrain
3. Skydome
4. Clouds (3D)/translucent objects
5. Translucent parts of the cockpit

With near/far cameras there might need to use stencil operations or similar to limit rendering only to parts visible from the cockpit.

Get rid of near/far camera

Ways to get rid of near and far cameras.

Custom z function

Use custom z function in the shader, for example log(z).

• Better precision -> no z fighting
• Cannot use early z kill in the shader -> All pixels are evaluated -> Slow
• Shaders can easily modify z value -> relief mappign etc

An interesting read: [18]

Floating point z-buffer

• Link to NVidia extension, that allows unclamped depth values to be written into the depth buffer. It seems to be a draft and not accepted, though: [19]
• This extension might be related too: [20]
• Tim Moore has done some things with fp-buffer: [21] and an example:[22]

Cockpit only near camera

Render only the inside view using near the camera, everything else with far camera. Put the transition to somewhere around 0.5-1.0m.

• Is the precision then sufficient?
• No z-fighting in the instruments
• Can render everything else in a single pass
• Might need to use stencil operations with cockpit if the cockpit is rendered first

Performance or eye candy related things

Ideas to boost the performance of the renderer.

Shared object performance

• If an object uses only one texture image for one object and only one material and no animations
• => Use a smart combining algorithm to combine hundreds of similar objects to one object ([23])
• Only vertices of the original array, copy face array for each clone, then a BIND_PER_PRIMITIVE attribute containing the center of the clone and a quaternion of its rotation
• Shader places the object correctly, no state changes during the rendering of ~100 objects => fast
• Similar system is in use for the trees and clouds
• Most likely breaks height-over-terrain and another collision checking

Grass and vegetation

[24] and [25]

• Make tree blobs larger, i.e. more trees on a single object
• Remove the second pass of the tree rendering effect if possible

Clouds

• Volume rendering [26]
• Render far away clouds to a texture and then render that image as a billboard
• Needs to refresh the cloud texture every now and then

Lighting models

• Rendering of wet materials [27]
• Apply a more "metallic" shader as a default for aircraft models? Link?
• Atmospheric scattering, at least for skydome, maybe for terrain too (Post process if using mixed-mode?)
• Different models are described here (taken from Shader requests): [28]. A simple summary:
  • Plastic: Phong or Blinn-Phong (currently the only model)
  • Metals and Plastics: Cook-Torrance, Strauss
  • Rough surfaces: Oren-Nayar
  • Clothing, anisotropic surfaces: Ward
  • Other models for a probably wide range of materials: Ashikhmin-Shirley

Related wiki pages

• Proposals:Scenery related
• Proposals:Eye Candy related
• Howto: Improve Framerates
• TerraGear texmapped line data
• Shader requests