FlightGear wiki - User contributions [en]

User:Fredb

2013-02-20T11:13:50Z

Fredb:

{{User
|name = Frédéric Bouvier
|location = France}}
[[Building using CMake - Windows]]

[[Project Rembrandt]]

==Development plan:==
* Restore near/far camera using depth range instead of clear to work with Rembrandt. Distance to viewer computation will change to cope with the two ranges. fg_Planes uniform need to convey three planes instead of two. Maybe fg_Planes_n and fg_Planes_f will be created (note: middle plane needs different parameters for the two ranges).
* Use geometry shader and instancing to compute shadow map to a texture array. Capitalize on OGL 4 features. (Seen in the OpenGL Insights book)
* Use geometry shader and instancing to compute multiple shadow maps for additional light sources.

[[Category: FlightGear Core developers]]

Fr/Projet Rembrandt

2013-01-06T11:25:22Z

Fredb: /* Pipeline configurable */

{{WIP}}
== Pourquoi ce nom? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] était un peintre hollandais du XVIIème siècle, maître incontesté du [http://en.wikipedia.org/wiki/Chiaroscuro "clair-obscur"].

Ce projet a pour but de changer la manière de rendre les lumières et les [[Shadows|ombres]] qu'utilise [[FlightGear]], et d'essayer d'imiter le style de Rembrandt dans FG.

== De quoi s'agit-il ? ==
L'idée maîtresse du projet est d'implémenter le [http://en.wikipedia.org/wiki/Deferred_shading rendu différé] au sein de FlightGear.
Depuis le début, FlightGear utilise un moteur de rendu classique, "en avant", qui affiche toutes les propriétés d'un objet en une fois (ombre, éclairage, brouillard, ...), ce qui rend difficile le rendu d'un ombrage plus sophistiqué (voir le [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) du fait que tous les paramètres de [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ l'équation lumineuse] doivent être traités en une seule fois.

[[Image:project_rembrandt_1.png|thumb|300px|Vue principale avec le contenu des buffers affichés dans les coins]]
Au contraire, le rendu différé cherche à séparer les opérations en étapes simplifiées, et à collecter les résultats intermédiaires dans des buffers cachés, qui peuvent être utilisés par les étapes suivantes.

;Le premier stade est celui de la Géométrie:
: nous rendons la scène entière dans 4 textures, en utilisant le rendu multi-cibles pour les traiter en un seul passage: un pour le buffer de profondeur, un pour les normals (en bas à gauche de l'image), un pour les couleurs "diffuses" (en bas à gauche) et un pour les couleurs "spéculaires" (en haut à droite).

;Le stade suivant est celui de l'ombre :
: nous rendons encore la scène dans une texture profondeur concernant les éclairages. Il y aura une texture pour chaque lumière diffusant des ombres.

;Ensuite, c'est le stade de la lumière, avec plusieurs sous-stades :
:*Sky pass: Le ciel est dessiné d'abord selon la méthode classique.
:*Ambient pass: la mémoire-tampon "diffuse" est modulée avec la couleur ambiante de la scène, et est dessinée comme un quad texturé aligné sur l'écran
:*Sunlight pass: Un second quad aligné à l'écran est dessiné et un shader examine la position pour calculer sa couleur "diffuse" et "specular" en utilisant le "normal" stocké dans le premier stade. La couleur résultante et mélangée avec la passe précédente. Les ombres sont examinées ici par comparaison de la position du pixel avec la position du cache lumière stockée dans la carte shadows
:*Fog pass: un nouvel écran quad est dessiné et la position du pixel est calculée pour évaluer la quantité de brouillard du pixel. La couleur du brouillard est mélangée avec le résultat de la phase précédente
:*stade de la lumière additionnelle: le scene graph sera traversé une autre fois pour afficher les volumes de lumière (cône ou tronc de cône pour les taches de lumière, sphère pour les lumières omni directionnelles), et leur shader ajoutera la lumière composée uniquement par les pixels éclairés.
:*stade brouillard: un nouveau quad aligné sur l'écran est dessiné, et la position du pixel est calculée pour évaluer la quantité de brouillard du pixel. La couleur du brouillard est mélangée avec le résultat du stade précédent.
:*Stade des objets transparents: les objets transparents (et les nuages) sont enfin rendus en utilisant la méthode classique.

:Tous les données de lumière sont accumulées dans une seule mémoire tampon, qui sera utilisée pour le dernier stade en addition au stade Géométrie.

;A la fin, le stade affichage, avec l'option effet post-traitement :
:Les résultats des précédents tampons sont poussés dans le tampon principal pour être affichés, optionnellement modifiés pour montrer Glow, Motion blur, HDR, redout ou blackout, occlusion ambiante de l'espace écran, anti-aliasing, etc...

Dans FG, on termine le pipeline du rendu par l'affichage du [[Menubar|GUI]] et du [[HUD]].

Tous ces phases sont plus précisément décrites dans [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] c'est la base du code actuel, avec quelques ajouts ou modifications.

== Avertissements ==
Le rendu différé n'affiche pas la transparence. Pour le moment, les nuages doivent être éclairés et ombrés par eux-mêmes. Les surfaces transparentes sont alpha-testées et non mélangées. Elles doivent être amenées dans leur propre bac sur l'image composite.

La partition profondeur ne marche pas non plus, à cause de la mémoire tampon profondeur, qui doit être gardée pour retenir la position de zone de vue, et, pour le moment, z-fighting est tout à fait visible. La partition profondeur sans rangée de profondeur superposée pourrait être la solution, et devrait être essayée.

Le passage de la lumière peut rendre certains MFD illisibles (ce qui utilisent une couleur émissive) car flous. Ils devront être traités comme transparents.

== Mise en œuvre ==
=== Répertoires ===
Le code est la principale branche du [[FlightGear and Git|répertoire officiel]]. Aucune autre localisation n'existe.

=== Rendu des surfaces transparentes ===
[[Image:project_rembrandt_5.png|thumb|300px|Surfaces transparentes desssinées d'après des objets opaques]]
Les surfaces transparentes sont détectées par les plugins OSG loader et reçoivent des suggestions de rendu TRANSPARENT_BIN
Au stade collection, le collecteur commande les surfaces transparentes dans le le bac transparent. Dans un retour, attaché à la caméra Géometry, après la traversée scénographe, les bacs transparent sont enlevés de la phase rendu et sauvegardés dans un espace temporaire. Dans un retour attaché la caméra Éclairage, après la traversée sceneraph, les bacs transparents sauvegardent au prochain stade, sont ajoutés au stade rendu de la caméra éclairageavec un numéro d'ordre. Ainsi, les surfaces transparentes sont dessinées au sommet de la scene éclairée à partir de G-buffer .

=== Consommation de mémoire ===
Pour chaque caméra définie du groupe, il y a une carte d'ombre séparée, de telle sorte que l'utilisation de la mémoire vidéo soit:
* G-tampon et tampon lumière: 20 bytes par pixel. Pour un écran HD (1920x1080) la mémoire requise est de 40 Mb
* Carte d'ombre 3 x taille_carte_ombre x shadow_map_size bytes (si la taille est 8192, la taille totale de la mémoire est 192 Mb
Ne pas compter les textures, liste d'affichage ou tampons vertex pour modèles et terrains

3 écrans HD ont besoin de 120 Mb de mémoire pour les tampons (ombre exclue), il vous faut 3x8192x8192x3 = 576 Mb (megabytes) de mémoire pour les seuls shadows.

Si vous voyez des messages d'erreur pendant le démarrage, ou si FlightGear ne démarre pas bien, c'est probablement parceque vous n'avez pas assez de mémoire vidéo libre. Réduisez la taille de la carte d'ombre dans préférences.xml

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

Et mettez 4096 ou 2048 à la place.Vous pouvez aussi utiliser un paramètre de démarrage: --prop:/sim/rendering/shadows/map-size=2048

=== Pipeline configurable ===

Le rendu Rembrandt utilise un fichier XML pour installer son pipeline pour chaque fenêtre décrite dans le groupe de cameras. Ce fichier décrit la façon dont les tampons intermédiaires sont établis, et comment les différentes étapes du rendu sont agencées. Le schéma général du fichier pipeline est le suivant :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Tampons (Buffers)====
Un tampon est une texture utilisée comme zone de stockage dans le GPU. Sa taille est habituellement un multiple de la taille de l'écran, mais une taille fixe est prise en charge ( typique pour la carte d'ombres). La description d'un tampon suit :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Phases ====
Une phase est une unité de rendu d'un groupe de tampons. La plupart des phases sont pré-définies et leur type n'est pas libre. Quand un type n'est pas spécifié, le nom est utilisé? Les types de phases sont :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|La phase geometry initialise la plupart des tampons et agit sur les objets réels et la géométrie. Les objets transparents sont placés à côté, et seront utilisés tels quels dans la phase lumière. Le reste de la géométrie opaque est rendu avec les effets standard, permettant de mettre la donnée sensible dans les tampons.
|-
|ombre
|Dans cette phase, la géométrie est rendue dans la carte normale à partir de la perspective du soleil.
|-
|lumière
|Cette phase utilise les tampons remplis par les phases précédrntes pour éclairer chaque pixel de la scène. Le résultat est rendu dans un autre tampon pour permettre des effets ultérieurs.
|-
|plein écran
|Les phases de ce type sont utilisés pour changer la scène entière, ou transformer une donnée à partir d'un tampon particulier.
|-
|affichage
|Le rendu final de la scène à l'écran est jugé à l'écran, ou la texture est définie dans le groupe caméra .
|}

Ci-dessous, une description de phase :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Les phases sont rendues dans les tampons (sauf pur la phase affichage). Les pièces jointes disent quels tampons sony affectés par chaque phase.

===== Pièces jointes =====
La pièce jointe décrit la liaison entre tampon et poit d'attachement :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Permissions(passes) =====
Les Passes sont seulement disponibles dans la phase <tt>lighting</tt>. trois sortes de phases sont autorisées :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Rendus de la voûte céleste, solei, lune, planète, étoiles et nuages
|-
|lumières
|Rendus des taches additionnelles et points lumineux
|-
|plein écran
|Le pass fullscreen analogue à une phase fullscreen sauf qu'il est rendu dans des tampons attachés à la phase lumière
|}
Un pass est défini ci-dessous :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

Une phase lumière typique est une succession de 5 pass:
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass pour lumière ambiante
# <tt>fullscreen</tt> pass pour lumière solaire (et ombres)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass pour brouillard

Chaque effet attaché au plein écran définit le mélange fait entre le pass et la somme précédente de rendu.

== Démarrage de FlightGear avec Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|dialogue Rembrandt]]
Le rendu Rembrandt est maintenant intégré dans le répertoire principal, mais doit être activé pour démarrer. Il y a deux façons de l'activer (une seule est nécessaire):
* <code>--enable-rembrandt</code> (lorsqu'on utilise [[FGRun]], on peut ajouter ceci derrière le <tt>FG_EXECUTABLE</tt> sur la première page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (avec FGRun on peut ajouter ceci via <tt>Advanced > Properties</tt> sur la dernière page).

Le <tt>View > Rendering Options > Rembrandt Options</tt> dialogue vous permet de basculer et d'ajuster les différentes options offertes par Rembrandt.

Rembrandt est très gourmand en ressources GPU et son démarrage peut échouer avec les options par défaut. Le symptome le plus fréquent est un message OSG dans la console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

On peut voir aussi :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

Des options additionnelles peuvent éviter ces problèmes :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Certaines vieilles cartes NVidia comme 7600GT, n'ont pas assez de résolution pour la profondeur, et on voit des "rideaux de brouillard" à quelques mètres devant. Une astuce consiste à encoder la profondeur dans une autre texture et donner la valeur correcte ensuite. Cette option permet cela.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Désactive les ombres tout à fait.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Évite le message "error C6013" sur les vieilles cartes en manque de résolution dans le cockpitt. Mettez <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> pour changer la taille de la carte d'ombre. Plus de taille, et moins de résolution (la valeur par défaut est 5 mètres)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Définnissez la taille de la carte d'ombres. Les valeurs utilles sont 1024, 2048, 4096 ou 8192. Peu de cartes supportent 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Définissez le nombre de cascades de la carte ombre. Moins de cascades signifie moins de temps passé à générer la carte shadow, mais aussi signifie une qualité moindre de l'ombre. Mettez entre 1 et 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Définissez la taille de cascade de la carte shadow pour chaque cascade. Les valeurs par défaut sont 5m, 50m, 500m et 5000m pour 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|Par défaut, Rembrandt utilise normalement des tampons de 8 bits (ainsi, la propriété est vraiment définie par défaut). Ceci peut créer des artefacts pour les éclairages specular. Si c'est inacceptable et si le GPU le supporte, mettez false pour avoir une meilleure précision pour les normales et les effets s'appuyant sur une direction normale.
|}

== Lignes directrices pour les écritures shader ==
=== Uniformes prédéfinis ===
Ces uniformes glsl n'ont pas besoin d'être déclarés dans le fichier effet
{| class="wikitable"
!Nom
!Type
!Objectif
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|En pass plein écran seulement, view matrix utilisé our transformer la position écran en direction vue
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|En pass plein écran seulement, view matrix inverse employé pour transformer la position écran en direction vue
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|En pass plein écran seulement, projection matri inverse employé pour transformer la position écran en direction vue
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position de la caméra en coordonnées cartésiennes
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position de la caméra dans l'espace, exprimé en coordonnées géodésiques (longitude et latitude en radians, élévation en mètres)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Dire si la profondeur est rangée dans une texture profondeur ou une texture couleur
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Utilisé pour convertir la valeur du tampon profondeur en une profondeur qui peut être utilisée pour calculer la position dans l'espace du fragment.
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions du tampon, utilisé pour convertir gl_FragCoord dans la taille [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Definie par OSG, employé seulement lors du travail sur la géométrie réelle
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Definie par OSG, employé seulement lors du travail sur la géométrie réelle
|}
.Ils doivent encore être déclarés dans le fragment du vertex shader

=== Fonctions utilitaires ===

Pour faciliter la maintenance des shaders plusieurs fonctions utilitaires sont disponibles pour le fragment shader. Ces fonctions sont placées ensemble dans deux fichiers : <tt>gbuffer-functions.frag</tt> et <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Utilisé pour encoder youtes les valeurs de G-Buffer dans material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Employé pour compresser normals à l'intérieur de G-Buffer dans material shaders. Normalement appelé de <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruit normals à partir de G-Buffer. Employé dans shaders plein écran et shaders lumière

;<tt>vec3 float_to_color(in float f)</tt>
:Encode les valeurs float dans la taille [0..1] dans les 24 bits d'une couleur. cette fonction est employée par <tt>encode_gbuffer()</tt> si le <tt>/sim/rendering/use-color-for_depth</tt> est vrai, pour les vieilles cartes qui ne fournissent pas d'information profondeur avec assez de résolution dans les shaders fullscreen ou lumière.

;<tt>float color_to_float(vec3 color)</tt>
:Décode les valeurs float dans la taille [0..1] dans les 24 bits d'une couleur. cette fonction est employée par <tt>position()</tt> si le <tt>/sim/rendering/use-color-for_depth</tt> est vrai, pour les vieilles cartes qui ne fournissent pas d'information profondeur avec assez de résolution dans les shaders fullscreen ou lumière.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruit la position eye space à partir de la direction view position a partir de la view direction et la profondeur lue du tampon profondeur

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
::Reconstruit la position eye space à partir de la direction view et la profondeur encodée dans une couleur lue du tampon profondeur

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruit la position eye space à partir de la direction vue et du tampon profondeur (profondeur réelle ou couleur selon la valeur de <tt>/sim/rendering/use-color-for_depth</tt>) à un fragment donné sur l'écran, donné par <tt>coords</tt>

==== Usage ====

Pour les shaders material, il faut fournir à la fois <tt>gbuffer-functions.frag</tt> et <tt>gbuffer-encode.frag</tt> dans le fichier effet, comme suit :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

Pour les shaders fullscreen passes, on doit fournir seulement <tt>gbuffer-functions.frag</tt>, comme ceci :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

Dans la principale fonction du shader, les fonctions référencées doivent être écrites en premier. Avec no #include files, le prototype de la fonction entière nécessite d'être typé :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);
main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Phase Geometry ===
La phase Geometry est là pour remplir le G-tampon. '''L'ombrage n'intervient pas à cette phase, et le calcul lumière ou brouillard ne doit pas faire partie du shader '''. L'opération requise dans le fragment Shader est de remplir avec une valeur sensible chaque tampon individuel :

{| class="wikitable" style="text-align: center;"
|pràfondeur (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment profondeur
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

Ceci est la couche par défaut attendue par le shader . material Id peut être utilisé pour détecter une couche différente

=== Pass lumière additionnelle ===
Il y aura là un seul shader pour chaque type de lumière utilisé. L'objectif est de créer des lumières comme animations dans le fichier model XML. Le shader lumière retrouvera la scène geometry en combinant la position espace écran en vue de rayon espace par l'inversion de la matrix projection (une fonction plus secourable sera fournie), et le fragment profondeur à cette position écran lit à partir du tampon profondeur. Avec l'aide du fragment normal, la couleur diffuse et specular et les propriétés de la lumière que le shader introduit, il sera possible d'ajouter au tampon lumière l'apport du rendu lumière.

===Pass brouillard===
En utilisant le fragment profondeur, il sera possible de calculer une distribution brouillard. Pour le moment, il n'y a que l'équation simple brouillard.

===Pass fleur===
C'est un effet double pass qui rend flou le tampon lumière dans une petite texture. Cette texture est ensuite ajoutée au tampon lumière lors de la phase affichage.

=== Effets requis ===
Plusieurs pass sont mis en œuvre pour l'utilisation du système effet. Dans ce but , des effets sont référencés dans le code core utilisant des noms réservés. Ces effets sont:
{| class="wikitable" cellpadding="3"
!Nom
!Type
!But
|-
|<tt>Effects/ssao</tt>
|Agit sur un quad plein écran
|Calcule l'occlusion ambiante à partir du tampon normal et du tampon profondeur
|-
|<tt>Effects/ambient</tt>
|Agit sur un quad plein écran
|Copie le tampon couleur diffuse multiplié par la lumière ambiante au tampon éclairage. L'occlusion ambiante peut aussi affecter la lumière ambiante .
|-
|<tt>Effects/light-spot</tt>
|Agit sur la géométrie réeelle du volume de la lumière
|Calcule la contribution lumière d'un point lumineux défini dans une animation <tt>light</tt>ayant un <tt>light-type</tt> de '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Agit sur un quad de plein écran
|Calcule le brouillard à partir du C-buffer et les paramètres de l'éclairage
|-
|<tt>Effects/display</tt>
|Agit sur un quad de plein écran
|Rends l'image finale composite à partir du G-buffer et du buffer éclairage
|}

== Lignes-guide pour modeleurs ==

=== Portage d'avion ===
* Rembrandt calcule les ombres => plus de fausses ombres dans le modèle
* Rembrandt calcule l'occlusion ambiante => pas d'occlusion ambiante incluse dans les textures
* Rembrandt est en lumière => les cartes lumière static ne sont pas nécéssaires, la couleur émissive pour voir les modèles n'est pas nécessaire, et pourrait interférer
* Rembrandt est brillant => les couleurs émissives incorrectement utilisées peuvent rendre flou, et rendre du texte illisible. L'intensité de la lumière devra être réglée
* Rembrandt a des besoins précis avec les shaders => les shaders nécessitent un ajustement pour se conformer au nouveau cadre, sinon la vue sera faussée
* Rembrandt ne peut faire des surfaces transparentes => les surfaces transparentes doivent être dûment enregistrées pour les atteindre avec le chemin classique

=== Enregistrement de toutes les surfaces translucides ===

Chaque modèle est par défaut rendu en utilisant l'effet <tt>Effects/model-default</tt>. Cet effet initialise le G-buffer, en ignorant les surfaces transparentes, en mettant alpha test et rendu de toute la geometrie dans la poubelle. Il n'est pas possible de rediriger le rendu vers les bacs transparents lorsque la texture associée a un canal alpha car la plupart des modeles emploient un atlas simple texture et même les parties opaques sont rendues avec texture avec alpha canal.

Si un modèle nécessite des surfaces translucides ou transparentes , ces objets surface doivent être assignés à un effet différent qui place explicitement le bac rendu à "DepthSortedBin", ou met la suggestion rendu à "transparent". Ceci dit au moteur de rendu de rendre cet objet opérant avant le rendu, ainsi, la clarté et le brouillard doivent être activés, et si un programme shader est utilisé, et s'ils ont été calculés en manière classique. Le <tt>Effects/model-transparent</tt> peut être utilisé pour enregistrer simlement les surfaces transparentes/translucides. Vous attribuez cet effet à un objet (ou à de multiples objets) comme:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

'''Attention: <Effect> agit seulement sur les objets réels, pas sur des groupes d'objets ou des animations.'''

Si la surface opaque a besoin d'un effet spécial, par exemple appliquer un "bump mapping" (effet de relief sur des textures) cet effet devra utiliser le bac "RenderBin", ou le rendu suggéré met à "opaque", et le G-buffer a besoin d'être initialisé correctement dans la phase Geometry.

=== Soyez sûr que toute la Geometry distribuera l'ombre ===

Pour limiter la quantité de rendu geometry dans la carte shadow, et aussi pour réduire les artefacts (acnée shadow), seulement les faces ne regardant pas le soleil distribuent les ombres. Le test est effectué en utilisant l'orientation normale. Cela veut dire que les polygones à double côté, ou mesh non fermés, seront transparents à la lumière sous certains angles de soleil. Pour éviter ça, les modeleurs peuvent:
*s'assurer que l'objet est toujours dans l'ombre d'autres objets,
*fermer leur mesh, ou
*doubler les polygones avec l'endroit normal à l'opposé

=== Ajouts de lumière à un modèle===

Deux points sont à considérer: l'apparence de la sourceet la zone illuminée. Pour l'apparence de la source lumière, (ce que vous voyez quand vous regardez le bulbe) vous avez besoin d'un modèle avec un emissive material qui produira l'effet brillant et ceci est visible de nuit.

Pour l'effet de la source dans son environnement (l'aire lumineuse), nous devons avoir dans le modèle 3D (le fichier .ac) un volume qui inclut l'effet (Volume Lumière). Cela peut être un grand cône pour les aires lumineuses ou une spère pour une lumière ponctuelle. Il est important que le volume lumière soit fermé, convexe et les normals orientés vers l'extérieur.

Le volume lumière doit être partie de la geometry du modèle et être référencé dans le fichier animation. Pas besoin d'ajouter une couleur ou un effet à ce volume. Le calcul lumière est seulement fait sur les fragments couverts par le volume lumière, mais n'a aucune influence sur la couleur ou l'atténuation de la lumière .

Toutes les animations disponibles sont possibles sur le volume lumière sauf '''material''' et '''texture'''. Il n'est pas possible de changer couleur et lumières, pour le moment; sauf en se branchant sur une autre animation. L'axe et la position sont dans l'espace objet et sont transformés par les animations qui suivent.

==== Taches lumineuses ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Nom
!Objectif
|-
|<tt>type</tt>
|Installe l'animation <tt>'''light'''</tt>
|-
|<tt>light-type</tt>
|C'est une lumière <tt>'''spot'''</tt>
|-
|<tt>name</tt>
|Nom donné à cette animation
|-
|<tt>object-name</tt>
|Nom du volume lumière dans le modèle 3D (typiquement un cône avec un sommet à <tt>position</tt>, le long de l'axe <tt>direction</tt> si <tt>cutoff</tt> est inférieur à 90 degrés, ou une sphère centrée à <tt>position</tt> si <tt>cutoff</tt> est supérieur à 90 degrés )
|-
|<tt>nopreview</tt>
|Cache le volume lumière dans fgrun 3d preview
|-
|<tt>position</tt>
|Dans espace objet, position de la lumière
|-
|<tt>direction</tt>
|Dans espace objet, direction vers le centre de la tache
|-
|<tt>ambient</tt>
|Couleur ambiante de la lumière
|-
|<tt>diffuse</tt>
|Couleur diffuse de la lumière
|-
|<tt>specular</tt>
|Couleur specular de la lumière
|-
|<tt>dim-factor</tt>
|Groupe de paramètres pour contrôler un facteur appliqué en même temps à ambient, diffuse et specular
|-
|<tt>attenuation</tt>
|Trois element vector. <c> element est le facteur constant, <l> element est le facteur linéaire et <q> element est le facteur quadratique. 
Attenuation de couleur à distance d est [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplié par <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir étant vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vecteur à partir de la position lumière vers le point, dans l'espace caméra.
|-
|<tt>near-m</tt>
|Distance minimum d'influence, à partir de la position, en mètres
|-
|<tt>far-m</tt>
|Distance maximum d'influence, à partir de la position, en mètres
|}
|}

==== Points lumineux ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Nom
!Objectif
|-
|<tt>type</tt>
|Installe l'animation <tt>'''light'''</tt>
|-
|<tt>light-type</tt>
|C'est une lumière <tt>'''point'''</tt>
|-
|<tt>name</tt>
|Nom donné à cette animation
|-
|<tt>object-name</tt>
|Nom du volume lumière dans le modèle 3D ( typiquement une sphère centrée sur <tt>position</tt>, avec un rayon de <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Volume lumière cachée dans fgrun 3d preview
|-
|<tt>position</tt>
|Dans espace object , position de la lumière
|-
|<tt>ambient</tt>
|Couleur ambiante de la lumière
|-
|<tt>diffuse</tt>
|Couleur diffuse de la lumière
|-
|<tt>specular</tt>
|Couleur specular de la lumièret
|-
|<tt>dim-factor</tt>
|Groupe de paramètres pour contrôler un facteur appliqué à ambient, diffuse et specular en même temps
|-
|<tt>attenuation</tt>
|Trois element vector. <c> element est le facteur constant <l> element est le facteur linéaire et <q> element est le facteur quadratiquer. 
Attenuation de couleur à distance d est [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Distance minimum d'influence, à partir de la position, en mètres
|-
|<tt>far-m</tt>
|Distance maximum d'influence, à partir de la position, en mètres
|}
|}

==== Considérations de performance et de compatibilité ====
Chaque lumière sur écran équivaut, pour le GPU à redessiner le volume lumière avec un shader, sans recueillir de buffer. Ainsi, chaque lumière vient avec un coût, ce qui est minime individuellement parlant, mais non négligeable quand jl y en a beaucoup. Ce coût augmente avec la taille du volume lumière

De plus, il est raisonnable de permettre à un modèle de fonctionner avec le moteur de rendu classique qui ne sait rien des lumières et rendrait les volumes lumière comme une autre géométrie. Ainsi, c'est une bonne pratique de compléter chaque animation lumière par une animation sélective vérifiant :
*si Rembrandt est activé
*si le mode de qualité choisi correspond avec les possibilités de la lumière
*si la lumi§re doit être on ou off selon les autres paramètres de la simulation (position du soleil; position de du commutateur, présence de puissance...)

Une bonne propriété est de crèer un reflet des préférences de l'utilisateur sur qualité vs performance concernant les lumières, et un curseur dédié est ajouté dans le dialogue shader.
[[File:Fgfs-shaders-with-light.jpg|thumb|Light quality slider]]
La propriété à utiliser est:
/sim/rendering/shaders/lights
Le curseur qualité va de 0 (pas de lumière) à 4 (pleine lumière). Un simple réverbère d'aéroport apparaît à 1. Quelques lampes de pont à 2. Toutes les lampes de pont simples à 3, et les plus sophistiquées à 4

Exemple:
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSource</name>
<object-name>LightVolume</object-name>
...
</animation>

<animation>
<type>select</type>

<object-name>LightSource</object-name>
<condition>
<and>

<property>/sim/rendering/rembrandt/enabled</property>

<greater-than>
<property>/sim/rendering/shaders/lights</property>
<value>3.0</value>
</greater-than>

<greater-than>
<property>/sim/time/sun-angle-rad</property>
<value>1.57</value>
</greater-than>
</and>
</condition>
</animation>

</syntaxhighlight>

=== Tutoriels ===
[[User:F-JJTH|F-JJTH]] a compilé son expérience et celle de l'équipe de la PAF d'amélioration d'avions dans {{fr}} [http://equipe-flightgear.forumactif.com/t1034-howto-adapter-un-appareil-pour-rembrandt this tutorial].

== TODO Liste ==

=== Tâches à compléter ===
*Fixer le rendu shadow quand on emploie plusieurs liens en OSG
*Implementer Cascaded Shadow Map (a besoin d'être optimisée - frustum calculation et nuit)
*Voir ce qui arrive avec brillance dans le brouillard: une sorte de terrain inconnu crée des patches blancs dans le buffer emission-problème de génération de scène
*Test multi-écran (mostly done)
*Restorer splashscreen
*Dessiner des objets transparents avec rendu en premier (peut nécessiter la capture du bac transparence à partir de la phase géométrie et le placer dans la phase affichage) (OK - nécessite une contribution modèle)
*Trouver une solution pour couleur ambient et emissive de material (peut nécessiter un buffer additionnel)
*Fournir un shader pour objets transparents qui puisse rendre le buffer emissive infaisable (en utilisant MRT) La phase lumière ne peut utiliser MRT
*Utiliser un buffer stencil pour limiter la taille lumière (non fait dans shader lumière)
**nécessaire pour la lumière de cockpit pour réaliser des faux shadowws et éviter d'éclairer le runway de la cabine
*Utiliser un système effet à la place des shaders lourdement codés
*Intensités globales de brillance, ou occlusion ambiante via curseur dans le dialogue rendu

=== Tâches à court terme ===
*Convertir les shaders existants en rendu différé
*Éviter de redessiner les objets opaques dans la phase lumière. Améliorer le masque node OSG .
*S'occuper des particules et précipitations
*Fixer le brouillard sur les nuages
*Fixer les matrices shadow dans multi-screen
*IMettre en œuvre un lightfield shader

=== Idées à long terme (non classées) ===
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore depth partitioning using depth ranges
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallerie ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendice ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[en:Project Rembrandt]]

Fr/Projet Rembrandt

2013-01-06T11:22:24Z

Fredb: /* De quoi s'agit-il ? */

{{WIP}}
== Pourquoi ce nom? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] était un peintre hollandais du XVIIème siècle, maître incontesté du [http://en.wikipedia.org/wiki/Chiaroscuro "clair-obscur"].

Ce projet a pour but de changer la manière de rendre les lumières et les [[Shadows|ombres]] qu'utilise [[FlightGear]], et d'essayer d'imiter le style de Rembrandt dans FG.

== De quoi s'agit-il ? ==
L'idée maîtresse du projet est d'implémenter le [http://en.wikipedia.org/wiki/Deferred_shading rendu différé] au sein de FlightGear.
Depuis le début, FlightGear utilise un moteur de rendu classique, "en avant", qui affiche toutes les propriétés d'un objet en une fois (ombre, éclairage, brouillard, ...), ce qui rend difficile le rendu d'un ombrage plus sophistiqué (voir le [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) du fait que tous les paramètres de [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ l'équation lumineuse] doivent être traités en une seule fois.

[[Image:project_rembrandt_1.png|thumb|300px|Vue principale avec le contenu des buffers affichés dans les coins]]
Au contraire, le rendu différé cherche à séparer les opérations en étapes simplifiées, et à collecter les résultats intermédiaires dans des buffers cachés, qui peuvent être utilisés par les étapes suivantes.

;Le premier stade est celui de la Géométrie:
: nous rendons la scène entière dans 4 textures, en utilisant le rendu multi-cibles pour les traiter en un seul passage: un pour le buffer de profondeur, un pour les normals (en bas à gauche de l'image), un pour les couleurs "diffuses" (en bas à gauche) et un pour les couleurs "spéculaires" (en haut à droite).

;Le stade suivant est celui de l'ombre :
: nous rendons encore la scène dans une texture profondeur concernant les éclairages. Il y aura une texture pour chaque lumière diffusant des ombres.

;Ensuite, c'est le stade de la lumière, avec plusieurs sous-stades :
:*Sky pass: Le ciel est dessiné d'abord selon la méthode classique.
:*Ambient pass: la mémoire-tampon "diffuse" est modulée avec la couleur ambiante de la scène, et est dessinée comme un quad texturé aligné sur l'écran
:*Sunlight pass: Un second quad aligné à l'écran est dessiné et un shader examine la position pour calculer sa couleur "diffuse" et "specular" en utilisant le "normal" stocké dans le premier stade. La couleur résultante et mélangée avec la passe précédente. Les ombres sont examinées ici par comparaison de la position du pixel avec la position du cache lumière stockée dans la carte shadows
:*Fog pass: un nouvel écran quad est dessiné et la position du pixel est calculée pour évaluer la quantité de brouillard du pixel. La couleur du brouillard est mélangée avec le résultat de la phase précédente
:*stade de la lumière additionnelle: le scene graph sera traversé une autre fois pour afficher les volumes de lumière (cône ou tronc de cône pour les taches de lumière, sphère pour les lumières omni directionnelles), et leur shader ajoutera la lumière composée uniquement par les pixels éclairés.
:*stade brouillard: un nouveau quad aligné sur l'écran est dessiné, et la position du pixel est calculée pour évaluer la quantité de brouillard du pixel. La couleur du brouillard est mélangée avec le résultat du stade précédent.
:*Stade des objets transparents: les objets transparents (et les nuages) sont enfin rendus en utilisant la méthode classique.

:Tous les données de lumière sont accumulées dans une seule mémoire tampon, qui sera utilisée pour le dernier stade en addition au stade Géométrie.

;A la fin, le stade affichage, avec l'option effet post-traitement :
:Les résultats des précédents tampons sont poussés dans le tampon principal pour être affichés, optionnellement modifiés pour montrer Glow, Motion blur, HDR, redout ou blackout, occlusion ambiante de l'espace écran, anti-aliasing, etc...

Dans FG, on termine le pipeline du rendu par l'affichage du [[Menubar|GUI]] et du [[HUD]].

Tous ces phases sont plus précisément décrites dans [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] c'est la base du code actuel, avec quelques ajouts ou modifications.

== Avertissements ==
Le rendu différé n'affiche pas la transparence. Pour le moment, les nuages doivent être éclairés et ombrés par eux-mêmes. Les surfaces transparentes sont alpha-testées et non mélangées. Elles doivent être amenées dans leur propre bac sur l'image composite.

La partition profondeur ne marche pas non plus, à cause de la mémoire tampon profondeur, qui doit être gardée pour retenir la position de zone de vue, et, pour le moment, z-fighting est tout à fait visible. La partition profondeur sans rangée de profondeur superposée pourrait être la solution, et devrait être essayée.

Le passage de la lumière peut rendre certains MFD illisibles (ce qui utilisent une couleur émissive) car flous. Ils devront être traités comme transparents.

== Mise en œuvre ==
=== Répertoires ===
Le code est la principale branche du [[FlightGear and Git|répertoire officiel]]. Aucune autre localisation n'existe.

=== Rendu des surfaces transparentes ===
[[Image:project_rembrandt_5.png|thumb|300px|Surfaces transparentes desssinées d'après des objets opaques]]
Les surfaces transparentes sont détectées par les plugins OSG loader et reçoivent des suggestions de rendu TRANSPARENT_BIN
Au stade collection, le collecteur commande les surfaces transparentes dans le le bac transparent. Dans un retour, attaché à la caméra Géometry, après la traversée scénographe, les bacs transparent sont enlevés de la phase rendu et sauvegardés dans un espace temporaire. Dans un retour attaché la caméra Éclairage, après la traversée sceneraph, les bacs transparents sauvegardent au prochain stade, sont ajoutés au stade rendu de la caméra éclairageavec un numéro d'ordre. Ainsi, les surfaces transparentes sont dessinées au sommet de la scene éclairée à partir de G-buffer .

=== Consommation de mémoire ===
Pour chaque caméra définie du groupe, il y a une carte d'ombre séparée, de telle sorte que l'utilisation de la mémoire vidéo soit:
* G-tampon et tampon lumière: 20 bytes par pixel. Pour un écran HD (1920x1080) la mémoire requise est de 40 Mb
* Carte d'ombre 3 x taille_carte_ombre x shadow_map_size bytes (si la taille est 8192, la taille totale de la mémoire est 192 Mb
Ne pas compter les textures, liste d'affichage ou tampons vertex pour modèles et terrains

3 écrans HD ont besoin de 120 Mb de mémoire pour les tampons (ombre exclue), il vous faut 3x8192x8192x3 = 576 Mb (megabytes) de mémoire pour les seuls shadows.

Si vous voyez des messages d'erreur pendant le démarrage, ou si FlightGear ne démarre pas bien, c'est probablement parceque vous n'avez pas assez de mémoire vidéo libre. Réduisez la taille de la carte d'ombre dans préférences.xml

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

Et mettez 4096 ou 2048 à la place.Vous pouvez aussi utiliser un paramètre de démarrage: --prop:/sim/rendering/shadows/map-size=2048

=== Pipeline configurable ===

Le rendu Rembrandt utilise un fichier XML pour installer son pipeline pour chaque fenêtre décrite dans le groupe de cameras. Ce fichier mntre la façon dont les tampons intermédiaires sont établis, et comment les différentes étapes du rendu sont agencés. Le schéma général du fichier pipeline est le suivant :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Tampons (Buffers)====
Un tampon est une texture utilisée comme zone de stockage dans le GPU. Sa taille est habituellement un multiple de la taille de l'écran, mais une taille fixe est prise en charge ( typique pour la carte d'ombres). La description d'un tampon suit :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Phases ====
Une phase est une unité de rendu d'un groupe de tampons. La plupart des phases sont pré-définies et leur type n'est pas libre. Quand un type n'est pas spécifié, le nom est utilisé? Les types de phases sont :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|La phase geometry initialise la plupart des tampons et agit sur les objets réels et la géométrie. Les objets transparents sont placés à côté, et seront utilisés tels quels dans la phase lumière. Le reste de la géométrie opaque est rendu avec les effets standard, permettant de mettre la donnée sensible dans les tampons.
|-
|ombre
|Dans cette phase, la géométrie est rendue dans la carte normale à partir de la perspective du soleil.
|-
|lumière
|Cette phase utilise les tampons remplis par les phases précédrntes pour éclairer chaque pixel de la scène. Le résultat est rendu dans un autre tampon pour permettre des effets ultérieurs.
|-
|plein écran
|Les phases de ce type sont utilisés pour changer la scène entière, ou transformer une donnée à partir d'un tampon particulier.
|-
|affichage
|Le rendu final de la scène à l'écran est jugé à l'écran, ou la texture est définie dans le groupe caméra .
|}

Ci-dessous, une description de phase :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Les phases sont rendues dans les tampons (sauf pur la phase affichage). Les pièces jointes disent quels tampons sony affectés par chaque phase.

===== Pièces jointes =====
La pièce jointe décrit la liaison entre tampon et poit d'attachement :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Permissions(passes) =====
Les Passes sont seulement disponibles dans la phase <tt>lighting</tt>. trois sortes de phases sont autorisées :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Rendus de la voûte céleste, solei, lune, planète, étoiles et nuages
|-
|lumières
|Rendus des taches additionnelles et points lumineux
|-
|plein écran
|Le pass fullscreen analogue à une phase fullscreen sauf qu'il est rendu dans des tampons attachés à la phase lumière
|}
Un pass est défini ci-dessous :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

Une phase lumière typique est une succession de 5 pass:
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass pour lumière ambiante
# <tt>fullscreen</tt> pass pour lumière solaire (et ombres)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass pour brouillard

Chaque effet attaché au plein écran définit le mélange fait entre le pass et la somme précédente de rendu.

== Démarrage de FlightGear avec Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|dialogue Rembrandt]]
Le rendu Rembrandt est maintenant intégré dans le répertoire principal, mais doit être activé pour démarrer. Il y a deux façons de l'activer (une seule est nécessaire):
* <code>--enable-rembrandt</code> (lorsqu'on utilise [[FGRun]], on peut ajouter ceci derrière le <tt>FG_EXECUTABLE</tt> sur la première page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (avec FGRun on peut ajouter ceci via <tt>Advanced > Properties</tt> sur la dernière page).

Le <tt>View > Rendering Options > Rembrandt Options</tt> dialogue vous permet de basculer et d'ajuster les différentes options offertes par Rembrandt.

Rembrandt est très gourmand en ressources GPU et son démarrage peut échouer avec les options par défaut. Le symptome le plus fréquent est un message OSG dans la console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

On peut voir aussi :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

Des options additionnelles peuvent éviter ces problèmes :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Certaines vieilles cartes NVidia comme 7600GT, n'ont pas assez de résolution pour la profondeur, et on voit des "rideaux de brouillard" à quelques mètres devant. Une astuce consiste à encoder la profondeur dans une autre texture et donner la valeur correcte ensuite. Cette option permet cela.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Désactive les ombres tout à fait.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Évite le message "error C6013" sur les vieilles cartes en manque de résolution dans le cockpitt. Mettez <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> pour changer la taille de la carte d'ombre. Plus de taille, et moins de résolution (la valeur par défaut est 5 mètres)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Définnissez la taille de la carte d'ombres. Les valeurs utilles sont 1024, 2048, 4096 ou 8192. Peu de cartes supportent 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Définissez le nombre de cascades de la carte ombre. Moins de cascades signifie moins de temps passé à générer la carte shadow, mais aussi signifie une qualité moindre de l'ombre. Mettez entre 1 et 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Définissez la taille de cascade de la carte shadow pour chaque cascade. Les valeurs par défaut sont 5m, 50m, 500m et 5000m pour 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|Par défaut, Rembrandt utilise normalement des tampons de 8 bits (ainsi, la propriété est vraiment définie par défaut). Ceci peut créer des artefacts pour les éclairages specular. Si c'est inacceptable et si le GPU le supporte, mettez false pour avoir une meilleure précision pour les normales et les effets s'appuyant sur une direction normale.
|}

== Lignes directrices pour les écritures shader ==
=== Uniformes prédéfinis ===
Ces uniformes glsl n'ont pas besoin d'être déclarés dans le fichier effet
{| class="wikitable"
!Nom
!Type
!Objectif
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|En pass plein écran seulement, view matrix utilisé our transformer la position écran en direction vue
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|En pass plein écran seulement, view matrix inverse employé pour transformer la position écran en direction vue
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|En pass plein écran seulement, projection matri inverse employé pour transformer la position écran en direction vue
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position de la caméra en coordonnées cartésiennes
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position de la caméra dans l'espace, exprimé en coordonnées géodésiques (longitude et latitude en radians, élévation en mètres)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Dire si la profondeur est rangée dans une texture profondeur ou une texture couleur
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Utilisé pour convertir la valeur du tampon profondeur en une profondeur qui peut être utilisée pour calculer la position dans l'espace du fragment.
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions du tampon, utilisé pour convertir gl_FragCoord dans la taille [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Definie par OSG, employé seulement lors du travail sur la géométrie réelle
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Definie par OSG, employé seulement lors du travail sur la géométrie réelle
|}
.Ils doivent encore être déclarés dans le fragment du vertex shader

=== Fonctions utilitaires ===

Pour faciliter la maintenance des shaders plusieurs fonctions utilitaires sont disponibles pour le fragment shader. Ces fonctions sont placées ensemble dans deux fichiers : <tt>gbuffer-functions.frag</tt> et <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Utilisé pour encoder youtes les valeurs de G-Buffer dans material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Employé pour compresser normals à l'intérieur de G-Buffer dans material shaders. Normalement appelé de <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruit normals à partir de G-Buffer. Employé dans shaders plein écran et shaders lumière

;<tt>vec3 float_to_color(in float f)</tt>
:Encode les valeurs float dans la taille [0..1] dans les 24 bits d'une couleur. cette fonction est employée par <tt>encode_gbuffer()</tt> si le <tt>/sim/rendering/use-color-for_depth</tt> est vrai, pour les vieilles cartes qui ne fournissent pas d'information profondeur avec assez de résolution dans les shaders fullscreen ou lumière.

;<tt>float color_to_float(vec3 color)</tt>
:Décode les valeurs float dans la taille [0..1] dans les 24 bits d'une couleur. cette fonction est employée par <tt>position()</tt> si le <tt>/sim/rendering/use-color-for_depth</tt> est vrai, pour les vieilles cartes qui ne fournissent pas d'information profondeur avec assez de résolution dans les shaders fullscreen ou lumière.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruit la position eye space à partir de la direction view position a partir de la view direction et la profondeur lue du tampon profondeur

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
::Reconstruit la position eye space à partir de la direction view et la profondeur encodée dans une couleur lue du tampon profondeur

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruit la position eye space à partir de la direction vue et du tampon profondeur (profondeur réelle ou couleur selon la valeur de <tt>/sim/rendering/use-color-for_depth</tt>) à un fragment donné sur l'écran, donné par <tt>coords</tt>

==== Usage ====

Pour les shaders material, il faut fournir à la fois <tt>gbuffer-functions.frag</tt> et <tt>gbuffer-encode.frag</tt> dans le fichier effet, comme suit :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

Pour les shaders fullscreen passes, on doit fournir seulement <tt>gbuffer-functions.frag</tt>, comme ceci :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

Dans la principale fonction du shader, les fonctions référencées doivent être écrites en premier. Avec no #include files, le prototype de la fonction entière nécessite d'être typé :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);
main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Phase Geometry ===
La phase Geometry est là pour remplir le G-tampon. '''L'ombrage n'intervient pas à cette phase, et le calcul lumière ou brouillard ne doit pas faire partie du shader '''. L'opération requise dans le fragment Shader est de remplir avec une valeur sensible chaque tampon individuel :

{| class="wikitable" style="text-align: center;"
|pràfondeur (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment profondeur
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

Ceci est la couche par défaut attendue par le shader . material Id peut être utilisé pour détecter une couche différente

=== Pass lumière additionnelle ===
Il y aura là un seul shader pour chaque type de lumière utilisé. L'objectif est de créer des lumières comme animations dans le fichier model XML. Le shader lumière retrouvera la scène geometry en combinant la position espace écran en vue de rayon espace par l'inversion de la matrix projection (une fonction plus secourable sera fournie), et le fragment profondeur à cette position écran lit à partir du tampon profondeur. Avec l'aide du fragment normal, la couleur diffuse et specular et les propriétés de la lumière que le shader introduit, il sera possible d'ajouter au tampon lumière l'apport du rendu lumière.

===Pass brouillard===
En utilisant le fragment profondeur, il sera possible de calculer une distribution brouillard. Pour le moment, il n'y a que l'équation simple brouillard.

===Pass fleur===
C'est un effet double pass qui rend flou le tampon lumière dans une petite texture. Cette texture est ensuite ajoutée au tampon lumière lors de la phase affichage.

=== Effets requis ===
Plusieurs pass sont mis en œuvre pour l'utilisation du système effet. Dans ce but , des effets sont référencés dans le code core utilisant des noms réservés. Ces effets sont:
{| class="wikitable" cellpadding="3"
!Nom
!Type
!But
|-
|<tt>Effects/ssao</tt>
|Agit sur un quad plein écran
|Calcule l'occlusion ambiante à partir du tampon normal et du tampon profondeur
|-
|<tt>Effects/ambient</tt>
|Agit sur un quad plein écran
|Copie le tampon couleur diffuse multiplié par la lumière ambiante au tampon éclairage. L'occlusion ambiante peut aussi affecter la lumière ambiante .
|-
|<tt>Effects/light-spot</tt>
|Agit sur la géométrie réeelle du volume de la lumière
|Calcule la contribution lumière d'un point lumineux défini dans une animation <tt>light</tt>ayant un <tt>light-type</tt> de '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Agit sur un quad de plein écran
|Calcule le brouillard à partir du C-buffer et les paramètres de l'éclairage
|-
|<tt>Effects/display</tt>
|Agit sur un quad de plein écran
|Rends l'image finale composite à partir du G-buffer et du buffer éclairage
|}

== Lignes-guide pour modeleurs ==

=== Portage d'avion ===
* Rembrandt calcule les ombres => plus de fausses ombres dans le modèle
* Rembrandt calcule l'occlusion ambiante => pas d'occlusion ambiante incluse dans les textures
* Rembrandt est en lumière => les cartes lumière static ne sont pas nécéssaires, la couleur émissive pour voir les modèles n'est pas nécessaire, et pourrait interférer
* Rembrandt est brillant => les couleurs émissives incorrectement utilisées peuvent rendre flou, et rendre du texte illisible. L'intensité de la lumière devra être réglée
* Rembrandt a des besoins précis avec les shaders => les shaders nécessitent un ajustement pour se conformer au nouveau cadre, sinon la vue sera faussée
* Rembrandt ne peut faire des surfaces transparentes => les surfaces transparentes doivent être dûment enregistrées pour les atteindre avec le chemin classique

=== Enregistrement de toutes les surfaces translucides ===

Chaque modèle est par défaut rendu en utilisant l'effet <tt>Effects/model-default</tt>. Cet effet initialise le G-buffer, en ignorant les surfaces transparentes, en mettant alpha test et rendu de toute la geometrie dans la poubelle. Il n'est pas possible de rediriger le rendu vers les bacs transparents lorsque la texture associée a un canal alpha car la plupart des modeles emploient un atlas simple texture et même les parties opaques sont rendues avec texture avec alpha canal.

Si un modèle nécessite des surfaces translucides ou transparentes , ces objets surface doivent être assignés à un effet différent qui place explicitement le bac rendu à "DepthSortedBin", ou met la suggestion rendu à "transparent". Ceci dit au moteur de rendu de rendre cet objet opérant avant le rendu, ainsi, la clarté et le brouillard doivent être activés, et si un programme shader est utilisé, et s'ils ont été calculés en manière classique. Le <tt>Effects/model-transparent</tt> peut être utilisé pour enregistrer simlement les surfaces transparentes/translucides. Vous attribuez cet effet à un objet (ou à de multiples objets) comme:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

'''Attention: <Effect> agit seulement sur les objets réels, pas sur des groupes d'objets ou des animations.'''

Si la surface opaque a besoin d'un effet spécial, par exemple appliquer un "bump mapping" (effet de relief sur des textures) cet effet devra utiliser le bac "RenderBin", ou le rendu suggéré met à "opaque", et le G-buffer a besoin d'être initialisé correctement dans la phase Geometry.

=== Soyez sûr que toute la Geometry distribuera l'ombre ===

Pour limiter la quantité de rendu geometry dans la carte shadow, et aussi pour réduire les artefacts (acnée shadow), seulement les faces ne regardant pas le soleil distribuent les ombres. Le test est effectué en utilisant l'orientation normale. Cela veut dire que les polygones à double côté, ou mesh non fermés, seront transparents à la lumière sous certains angles de soleil. Pour éviter ça, les modeleurs peuvent:
*s'assurer que l'objet est toujours dans l'ombre d'autres objets,
*fermer leur mesh, ou
*doubler les polygones avec l'endroit normal à l'opposé

=== Ajouts de lumière à un modèle===

Deux points sont à considérer: l'apparence de la sourceet la zone illuminée. Pour l'apparence de la source lumière, (ce que vous voyez quand vous regardez le bulbe) vous avez besoin d'un modèle avec un emissive material qui produira l'effet brillant et ceci est visible de nuit.

Pour l'effet de la source dans son environnement (l'aire lumineuse), nous devons avoir dans le modèle 3D (le fichier .ac) un volume qui inclut l'effet (Volume Lumière). Cela peut être un grand cône pour les aires lumineuses ou une spère pour une lumière ponctuelle. Il est important que le volume lumière soit fermé, convexe et les normals orientés vers l'extérieur.

Le volume lumière doit être partie de la geometry du modèle et être référencé dans le fichier animation. Pas besoin d'ajouter une couleur ou un effet à ce volume. Le calcul lumière est seulement fait sur les fragments couverts par le volume lumière, mais n'a aucune influence sur la couleur ou l'atténuation de la lumière .

Toutes les animations disponibles sont possibles sur le volume lumière sauf '''material''' et '''texture'''. Il n'est pas possible de changer couleur et lumières, pour le moment; sauf en se branchant sur une autre animation. L'axe et la position sont dans l'espace objet et sont transformés par les animations qui suivent.

==== Taches lumineuses ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Nom
!Objectif
|-
|<tt>type</tt>
|Installe l'animation <tt>'''light'''</tt>
|-
|<tt>light-type</tt>
|C'est une lumière <tt>'''spot'''</tt>
|-
|<tt>name</tt>
|Nom donné à cette animation
|-
|<tt>object-name</tt>
|Nom du volume lumière dans le modèle 3D (typiquement un cône avec un sommet à <tt>position</tt>, le long de l'axe <tt>direction</tt> si <tt>cutoff</tt> est inférieur à 90 degrés, ou une sphère centrée à <tt>position</tt> si <tt>cutoff</tt> est supérieur à 90 degrés )
|-
|<tt>nopreview</tt>
|Cache le volume lumière dans fgrun 3d preview
|-
|<tt>position</tt>
|Dans espace objet, position de la lumière
|-
|<tt>direction</tt>
|Dans espace objet, direction vers le centre de la tache
|-
|<tt>ambient</tt>
|Couleur ambiante de la lumière
|-
|<tt>diffuse</tt>
|Couleur diffuse de la lumière
|-
|<tt>specular</tt>
|Couleur specular de la lumière
|-
|<tt>dim-factor</tt>
|Groupe de paramètres pour contrôler un facteur appliqué en même temps à ambient, diffuse et specular
|-
|<tt>attenuation</tt>
|Trois element vector. <c> element est le facteur constant, <l> element est le facteur linéaire et <q> element est le facteur quadratique. 
Attenuation de couleur à distance d est [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplié par <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir étant vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vecteur à partir de la position lumière vers le point, dans l'espace caméra.
|-
|<tt>near-m</tt>
|Distance minimum d'influence, à partir de la position, en mètres
|-
|<tt>far-m</tt>
|Distance maximum d'influence, à partir de la position, en mètres
|}
|}

==== Points lumineux ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Nom
!Objectif
|-
|<tt>type</tt>
|Installe l'animation <tt>'''light'''</tt>
|-
|<tt>light-type</tt>
|C'est une lumière <tt>'''point'''</tt>
|-
|<tt>name</tt>
|Nom donné à cette animation
|-
|<tt>object-name</tt>
|Nom du volume lumière dans le modèle 3D ( typiquement une sphère centrée sur <tt>position</tt>, avec un rayon de <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Volume lumière cachée dans fgrun 3d preview
|-
|<tt>position</tt>
|Dans espace object , position de la lumière
|-
|<tt>ambient</tt>
|Couleur ambiante de la lumière
|-
|<tt>diffuse</tt>
|Couleur diffuse de la lumière
|-
|<tt>specular</tt>
|Couleur specular de la lumièret
|-
|<tt>dim-factor</tt>
|Groupe de paramètres pour contrôler un facteur appliqué à ambient, diffuse et specular en même temps
|-
|<tt>attenuation</tt>
|Trois element vector. <c> element est le facteur constant <l> element est le facteur linéaire et <q> element est le facteur quadratiquer. 
Attenuation de couleur à distance d est [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Distance minimum d'influence, à partir de la position, en mètres
|-
|<tt>far-m</tt>
|Distance maximum d'influence, à partir de la position, en mètres
|}
|}

==== Considérations de performance et de compatibilité ====
Chaque lumière sur écran équivaut, pour le GPU à redessiner le volume lumière avec un shader, sans recueillir de buffer. Ainsi, chaque lumière vient avec un coût, ce qui est minime individuellement parlant, mais non négligeable quand jl y en a beaucoup. Ce coût augmente avec la taille du volume lumière

De plus, il est raisonnable de permettre à un modèle de fonctionner avec le moteur de rendu classique qui ne sait rien des lumières et rendrait les volumes lumière comme une autre géométrie. Ainsi, c'est une bonne pratique de compléter chaque animation lumière par une animation sélective vérifiant :
*si Rembrandt est activé
*si le mode de qualité choisi correspond avec les possibilités de la lumière
*si la lumi§re doit être on ou off selon les autres paramètres de la simulation (position du soleil; position de du commutateur, présence de puissance...)

Une bonne propriété est de crèer un reflet des préférences de l'utilisateur sur qualité vs performance concernant les lumières, et un curseur dédié est ajouté dans le dialogue shader.
[[File:Fgfs-shaders-with-light.jpg|thumb|Light quality slider]]
La propriété à utiliser est:
/sim/rendering/shaders/lights
Le curseur qualité va de 0 (pas de lumière) à 4 (pleine lumière). Un simple réverbère d'aéroport apparaît à 1. Quelques lampes de pont à 2. Toutes les lampes de pont simples à 3, et les plus sophistiquées à 4

Exemple:
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSource</name>
<object-name>LightVolume</object-name>
...
</animation>

<animation>
<type>select</type>

<object-name>LightSource</object-name>
<condition>
<and>

<property>/sim/rendering/rembrandt/enabled</property>

<greater-than>
<property>/sim/rendering/shaders/lights</property>
<value>3.0</value>
</greater-than>

<greater-than>
<property>/sim/time/sun-angle-rad</property>
<value>1.57</value>
</greater-than>
</and>
</condition>
</animation>

</syntaxhighlight>

=== Tutoriels ===
[[User:F-JJTH|F-JJTH]] a compilé son expérience et celle de l'équipe de la PAF d'amélioration d'avions dans {{fr}} [http://equipe-flightgear.forumactif.com/t1034-howto-adapter-un-appareil-pour-rembrandt this tutorial].

== TODO Liste ==

=== Tâches à compléter ===
*Fixer le rendu shadow quand on emploie plusieurs liens en OSG
*Implementer Cascaded Shadow Map (a besoin d'être optimisée - frustum calculation et nuit)
*Voir ce qui arrive avec brillance dans le brouillard: une sorte de terrain inconnu crée des patches blancs dans le buffer emission-problème de génération de scène
*Test multi-écran (mostly done)
*Restorer splashscreen
*Dessiner des objets transparents avec rendu en premier (peut nécessiter la capture du bac transparence à partir de la phase géométrie et le placer dans la phase affichage) (OK - nécessite une contribution modèle)
*Trouver une solution pour couleur ambient et emissive de material (peut nécessiter un buffer additionnel)
*Fournir un shader pour objets transparents qui puisse rendre le buffer emissive infaisable (en utilisant MRT) La phase lumière ne peut utiliser MRT
*Utiliser un buffer stencil pour limiter la taille lumière (non fait dans shader lumière)
**nécessaire pour la lumière de cockpit pour réaliser des faux shadowws et éviter d'éclairer le runway de la cabine
*Utiliser un système effet à la place des shaders lourdement codés
*Intensités globales de brillance, ou occlusion ambiante via curseur dans le dialogue rendu

=== Tâches à court terme ===
*Convertir les shaders existants en rendu différé
*Éviter de redessiner les objets opaques dans la phase lumière. Améliorer le masque node OSG .
*S'occuper des particules et précipitations
*Fixer le brouillard sur les nuages
*Fixer les matrices shadow dans multi-screen
*IMettre en œuvre un lightfield shader

=== Idées à long terme (non classées) ===
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore depth partitioning using depth ranges
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallerie ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendice ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[en:Project Rembrandt]]

Fr/Projet Rembrandt

2013-01-06T11:20:47Z

Fredb: /* De quoi s'agit-il ? */

{{WIP}}
== Pourquoi ce nom? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] était un peintre hollandais du XVIIème siècle, maître incontesté du [http://en.wikipedia.org/wiki/Chiaroscuro "clair-obscur"].

Ce projet a pour but de changer la manière de rendre les lumières et les [[Shadows|ombres]] qu'utilise [[FlightGear]], et d'essayer d'imiter le style de Rembrandt dans FG.

== De quoi s'agit-il ? ==
L'idée maîtresse du projet est d'implémenter le [http://en.wikipedia.org/wiki/Deferred_shading rendu différé] au sein de FlightGear.
Depuis le début, FlightGear utilise un moteur de rendu classique, "en avant", qui affiche toutes les propriétés d'un objet en une fois (ombre, éclairage, brouillard, ...), ce qui rend difficile le rendu d'un ombrage plus sophistiqué (voir le [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) du fait que tous les paramètres de [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ l'équation lumineuse] doivent être traités en une seule fois.

[[Image:project_rembrandt_1.png|thumb|300px|Vue principale avec le contenu des buffers affichés dans les coins]]
Au contraire, le rendu différé cherche à séparer les opérations en étapes simplifiées, et à collecter les résultats intermédiaires dans des buffers cachés, qui peuvent être utilisés par la prochain étape.

;Le premier stade est celui de la Géométrie:
: nous rendons la scène entière dans 4 textures, en utilisant le rendu multi-cibles pour les traiter en un seul passage: un pour le buffer de profondeur, un pour les normals (en bas à gauche de l'image), un pour les couleurs "diffuses" (en bas à gauche) et un pour les couleurs "spéculaires" (en haut à droite).

;Le stade suivant est celui de l'ombre :
: nous rendons encore la scène dans une texture profondeur concernant les éclairages. Il y aura une texture pour chaque lumière diffusant des ombres.

;Ensuite, c'est le stade de la lumière, avec plusieurs sous-stades :
:*Sky pass: Le ciel est dessiné d'abord selon la méthode classique.
:*Ambient pass: la mémoire-tampon "diffuse" est modulée avec la couleur ambiante de la scène, et est dessinée comme un quad texturé aligné sur l'écran
:*Sunlight pass: Un second quad aligné à l'écran est dessiné et un shader examine la position pour calculer sa couleur "diffuse" et "specular" en utilisant le "normal" stocké dans le premier stade. La couleur résultante et mélangée avec la passe précédente. Les ombres sont examinées ici par comparaison de la position du pixel avec la position du cache lumière stockée dans la carte shadows
:*Fog pass: un nouvel écran quad est dessiné et la position du pixel est calculée pour évaluer la quantité de brouillard du pixel. La couleur du brouillard est mélangée avec le résultat de la phase précédente
:*stade de la lumière additionnelle: le scene graph sera traversé une autre fois pour afficher les volumes de lumière (cône ou tronc de cône pour les taches de lumière, sphère pour les lumières omni directionnelles), et leur shader ajoutera la lumière composée uniquement par les pixels éclairés.
:*stade brouillard: un nouveau quad aligné sur l'écran est dessiné, et la position du pixel est calculée pour évaluer la quantité de brouillard du pixel. La couleur du brouillard est mélangée avec le résultat du stade précédent.
:*Stade des objets transparents: les objets transparents (et les nuages) sont enfin rendus en utilisant la méthode classique.

:Tous les données de lumière sont accumulées dans une seule mémoire tampon, qui sera utilisée pour le dernier stade en addition au stade Géométrie.

;A la fin, le stade affichage, avec l'option effet post-traitement :
:Les résultats des précédents tampons sont poussés dans le tampon principal pour être affichés, optionnellement modifiés pour montrer Glow, Motion blur, HDR, redout ou blackout, occlusion ambiante de l'espace écran, anti-aliasing, etc...

Dans FG, on termine le pipeline du rendu par l'affichage du [[Menubar|GUI]] et du [[HUD]].

Tous ces phases sont plus précisément décrites dans [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] c'est la base du code actuel, avec quelques ajouts ou modifications.

== Avertissements ==
Le rendu différé n'affiche pas la transparence. Pour le moment, les nuages doivent être éclairés et ombrés par eux-mêmes. Les surfaces transparentes sont alpha-testées et non mélangées. Elles doivent être amenées dans leur propre bac sur l'image composite.

La partition profondeur ne marche pas non plus, à cause de la mémoire tampon profondeur, qui doit être gardée pour retenir la position de zone de vue, et, pour le moment, z-fighting est tout à fait visible. La partition profondeur sans rangée de profondeur superposée pourrait être la solution, et devrait être essayée.

Le passage de la lumière peut rendre certains MFD illisibles (ce qui utilisent une couleur émissive) car flous. Ils devront être traités comme transparents.

== Mise en œuvre ==
=== Répertoires ===
Le code est la principale branche du [[FlightGear and Git|répertoire officiel]]. Aucune autre localisation n'existe.

=== Rendu des surfaces transparentes ===
[[Image:project_rembrandt_5.png|thumb|300px|Surfaces transparentes desssinées d'après des objets opaques]]
Les surfaces transparentes sont détectées par les plugins OSG loader et reçoivent des suggestions de rendu TRANSPARENT_BIN
Au stade collection, le collecteur commande les surfaces transparentes dans le le bac transparent. Dans un retour, attaché à la caméra Géometry, après la traversée scénographe, les bacs transparent sont enlevés de la phase rendu et sauvegardés dans un espace temporaire. Dans un retour attaché la caméra Éclairage, après la traversée sceneraph, les bacs transparents sauvegardent au prochain stade, sont ajoutés au stade rendu de la caméra éclairageavec un numéro d'ordre. Ainsi, les surfaces transparentes sont dessinées au sommet de la scene éclairée à partir de G-buffer .

=== Consommation de mémoire ===
Pour chaque caméra définie du groupe, il y a une carte d'ombre séparée, de telle sorte que l'utilisation de la mémoire vidéo soit:
* G-tampon et tampon lumière: 20 bytes par pixel. Pour un écran HD (1920x1080) la mémoire requise est de 40 Mb
* Carte d'ombre 3 x taille_carte_ombre x shadow_map_size bytes (si la taille est 8192, la taille totale de la mémoire est 192 Mb
Ne pas compter les textures, liste d'affichage ou tampons vertex pour modèles et terrains

3 écrans HD ont besoin de 120 Mb de mémoire pour les tampons (ombre exclue), il vous faut 3x8192x8192x3 = 576 Mb (megabytes) de mémoire pour les seuls shadows.

Si vous voyez des messages d'erreur pendant le démarrage, ou si FlightGear ne démarre pas bien, c'est probablement parceque vous n'avez pas assez de mémoire vidéo libre. Réduisez la taille de la carte d'ombre dans préférences.xml

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

Et mettez 4096 ou 2048 à la place.Vous pouvez aussi utiliser un paramètre de démarrage: --prop:/sim/rendering/shadows/map-size=2048

=== Pipeline configurable ===

Le rendu Rembrandt utilise un fichier XML pour installer son pipeline pour chaque fenêtre décrite dans le groupe de cameras. Ce fichier mntre la façon dont les tampons intermédiaires sont établis, et comment les différentes étapes du rendu sont agencés. Le schéma général du fichier pipeline est le suivant :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Tampons (Buffers)====
Un tampon est une texture utilisée comme zone de stockage dans le GPU. Sa taille est habituellement un multiple de la taille de l'écran, mais une taille fixe est prise en charge ( typique pour la carte d'ombres). La description d'un tampon suit :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Phases ====
Une phase est une unité de rendu d'un groupe de tampons. La plupart des phases sont pré-définies et leur type n'est pas libre. Quand un type n'est pas spécifié, le nom est utilisé? Les types de phases sont :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|La phase geometry initialise la plupart des tampons et agit sur les objets réels et la géométrie. Les objets transparents sont placés à côté, et seront utilisés tels quels dans la phase lumière. Le reste de la géométrie opaque est rendu avec les effets standard, permettant de mettre la donnée sensible dans les tampons.
|-
|ombre
|Dans cette phase, la géométrie est rendue dans la carte normale à partir de la perspective du soleil.
|-
|lumière
|Cette phase utilise les tampons remplis par les phases précédrntes pour éclairer chaque pixel de la scène. Le résultat est rendu dans un autre tampon pour permettre des effets ultérieurs.
|-
|plein écran
|Les phases de ce type sont utilisés pour changer la scène entière, ou transformer une donnée à partir d'un tampon particulier.
|-
|affichage
|Le rendu final de la scène à l'écran est jugé à l'écran, ou la texture est définie dans le groupe caméra .
|}

Ci-dessous, une description de phase :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Les phases sont rendues dans les tampons (sauf pur la phase affichage). Les pièces jointes disent quels tampons sony affectés par chaque phase.

===== Pièces jointes =====
La pièce jointe décrit la liaison entre tampon et poit d'attachement :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Permissions(passes) =====
Les Passes sont seulement disponibles dans la phase <tt>lighting</tt>. trois sortes de phases sont autorisées :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Rendus de la voûte céleste, solei, lune, planète, étoiles et nuages
|-
|lumières
|Rendus des taches additionnelles et points lumineux
|-
|plein écran
|Le pass fullscreen analogue à une phase fullscreen sauf qu'il est rendu dans des tampons attachés à la phase lumière
|}
Un pass est défini ci-dessous :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

Une phase lumière typique est une succession de 5 pass:
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass pour lumière ambiante
# <tt>fullscreen</tt> pass pour lumière solaire (et ombres)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass pour brouillard

Chaque effet attaché au plein écran définit le mélange fait entre le pass et la somme précédente de rendu.

== Démarrage de FlightGear avec Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|dialogue Rembrandt]]
Le rendu Rembrandt est maintenant intégré dans le répertoire principal, mais doit être activé pour démarrer. Il y a deux façons de l'activer (une seule est nécessaire):
* <code>--enable-rembrandt</code> (lorsqu'on utilise [[FGRun]], on peut ajouter ceci derrière le <tt>FG_EXECUTABLE</tt> sur la première page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (avec FGRun on peut ajouter ceci via <tt>Advanced > Properties</tt> sur la dernière page).

Le <tt>View > Rendering Options > Rembrandt Options</tt> dialogue vous permet de basculer et d'ajuster les différentes options offertes par Rembrandt.

Rembrandt est très gourmand en ressources GPU et son démarrage peut échouer avec les options par défaut. Le symptome le plus fréquent est un message OSG dans la console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

On peut voir aussi :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

Des options additionnelles peuvent éviter ces problèmes :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Certaines vieilles cartes NVidia comme 7600GT, n'ont pas assez de résolution pour la profondeur, et on voit des "rideaux de brouillard" à quelques mètres devant. Une astuce consiste à encoder la profondeur dans une autre texture et donner la valeur correcte ensuite. Cette option permet cela.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Désactive les ombres tout à fait.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Évite le message "error C6013" sur les vieilles cartes en manque de résolution dans le cockpitt. Mettez <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> pour changer la taille de la carte d'ombre. Plus de taille, et moins de résolution (la valeur par défaut est 5 mètres)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Définnissez la taille de la carte d'ombres. Les valeurs utilles sont 1024, 2048, 4096 ou 8192. Peu de cartes supportent 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Définissez le nombre de cascades de la carte ombre. Moins de cascades signifie moins de temps passé à générer la carte shadow, mais aussi signifie une qualité moindre de l'ombre. Mettez entre 1 et 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Définissez la taille de cascade de la carte shadow pour chaque cascade. Les valeurs par défaut sont 5m, 50m, 500m et 5000m pour 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|Par défaut, Rembrandt utilise normalement des tampons de 8 bits (ainsi, la propriété est vraiment définie par défaut). Ceci peut créer des artefacts pour les éclairages specular. Si c'est inacceptable et si le GPU le supporte, mettez false pour avoir une meilleure précision pour les normales et les effets s'appuyant sur une direction normale.
|}

== Lignes directrices pour les écritures shader ==
=== Uniformes prédéfinis ===
Ces uniformes glsl n'ont pas besoin d'être déclarés dans le fichier effet
{| class="wikitable"
!Nom
!Type
!Objectif
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|En pass plein écran seulement, view matrix utilisé our transformer la position écran en direction vue
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|En pass plein écran seulement, view matrix inverse employé pour transformer la position écran en direction vue
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|En pass plein écran seulement, projection matri inverse employé pour transformer la position écran en direction vue
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position de la caméra en coordonnées cartésiennes
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position de la caméra dans l'espace, exprimé en coordonnées géodésiques (longitude et latitude en radians, élévation en mètres)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Dire si la profondeur est rangée dans une texture profondeur ou une texture couleur
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Utilisé pour convertir la valeur du tampon profondeur en une profondeur qui peut être utilisée pour calculer la position dans l'espace du fragment.
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions du tampon, utilisé pour convertir gl_FragCoord dans la taille [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Definie par OSG, employé seulement lors du travail sur la géométrie réelle
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Definie par OSG, employé seulement lors du travail sur la géométrie réelle
|}
.Ils doivent encore être déclarés dans le fragment du vertex shader

=== Fonctions utilitaires ===

Pour faciliter la maintenance des shaders plusieurs fonctions utilitaires sont disponibles pour le fragment shader. Ces fonctions sont placées ensemble dans deux fichiers : <tt>gbuffer-functions.frag</tt> et <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Utilisé pour encoder youtes les valeurs de G-Buffer dans material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Employé pour compresser normals à l'intérieur de G-Buffer dans material shaders. Normalement appelé de <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruit normals à partir de G-Buffer. Employé dans shaders plein écran et shaders lumière

;<tt>vec3 float_to_color(in float f)</tt>
:Encode les valeurs float dans la taille [0..1] dans les 24 bits d'une couleur. cette fonction est employée par <tt>encode_gbuffer()</tt> si le <tt>/sim/rendering/use-color-for_depth</tt> est vrai, pour les vieilles cartes qui ne fournissent pas d'information profondeur avec assez de résolution dans les shaders fullscreen ou lumière.

;<tt>float color_to_float(vec3 color)</tt>
:Décode les valeurs float dans la taille [0..1] dans les 24 bits d'une couleur. cette fonction est employée par <tt>position()</tt> si le <tt>/sim/rendering/use-color-for_depth</tt> est vrai, pour les vieilles cartes qui ne fournissent pas d'information profondeur avec assez de résolution dans les shaders fullscreen ou lumière.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruit la position eye space à partir de la direction view position a partir de la view direction et la profondeur lue du tampon profondeur

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
::Reconstruit la position eye space à partir de la direction view et la profondeur encodée dans une couleur lue du tampon profondeur

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruit la position eye space à partir de la direction vue et du tampon profondeur (profondeur réelle ou couleur selon la valeur de <tt>/sim/rendering/use-color-for_depth</tt>) à un fragment donné sur l'écran, donné par <tt>coords</tt>

==== Usage ====

Pour les shaders material, il faut fournir à la fois <tt>gbuffer-functions.frag</tt> et <tt>gbuffer-encode.frag</tt> dans le fichier effet, comme suit :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

Pour les shaders fullscreen passes, on doit fournir seulement <tt>gbuffer-functions.frag</tt>, comme ceci :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

Dans la principale fonction du shader, les fonctions référencées doivent être écrites en premier. Avec no #include files, le prototype de la fonction entière nécessite d'être typé :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);
main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Phase Geometry ===
La phase Geometry est là pour remplir le G-tampon. '''L'ombrage n'intervient pas à cette phase, et le calcul lumière ou brouillard ne doit pas faire partie du shader '''. L'opération requise dans le fragment Shader est de remplir avec une valeur sensible chaque tampon individuel :

{| class="wikitable" style="text-align: center;"
|pràfondeur (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment profondeur
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

Ceci est la couche par défaut attendue par le shader . material Id peut être utilisé pour détecter une couche différente

=== Pass lumière additionnelle ===
Il y aura là un seul shader pour chaque type de lumière utilisé. L'objectif est de créer des lumières comme animations dans le fichier model XML. Le shader lumière retrouvera la scène geometry en combinant la position espace écran en vue de rayon espace par l'inversion de la matrix projection (une fonction plus secourable sera fournie), et le fragment profondeur à cette position écran lit à partir du tampon profondeur. Avec l'aide du fragment normal, la couleur diffuse et specular et les propriétés de la lumière que le shader introduit, il sera possible d'ajouter au tampon lumière l'apport du rendu lumière.

===Pass brouillard===
En utilisant le fragment profondeur, il sera possible de calculer une distribution brouillard. Pour le moment, il n'y a que l'équation simple brouillard.

===Pass fleur===
C'est un effet double pass qui rend flou le tampon lumière dans une petite texture. Cette texture est ensuite ajoutée au tampon lumière lors de la phase affichage.

=== Effets requis ===
Plusieurs pass sont mis en œuvre pour l'utilisation du système effet. Dans ce but , des effets sont référencés dans le code core utilisant des noms réservés. Ces effets sont:
{| class="wikitable" cellpadding="3"
!Nom
!Type
!But
|-
|<tt>Effects/ssao</tt>
|Agit sur un quad plein écran
|Calcule l'occlusion ambiante à partir du tampon normal et du tampon profondeur
|-
|<tt>Effects/ambient</tt>
|Agit sur un quad plein écran
|Copie le tampon couleur diffuse multiplié par la lumière ambiante au tampon éclairage. L'occlusion ambiante peut aussi affecter la lumière ambiante .
|-
|<tt>Effects/light-spot</tt>
|Agit sur la géométrie réeelle du volume de la lumière
|Calcule la contribution lumière d'un point lumineux défini dans une animation <tt>light</tt>ayant un <tt>light-type</tt> de '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Agit sur un quad de plein écran
|Calcule le brouillard à partir du C-buffer et les paramètres de l'éclairage
|-
|<tt>Effects/display</tt>
|Agit sur un quad de plein écran
|Rends l'image finale composite à partir du G-buffer et du buffer éclairage
|}

== Lignes-guide pour modeleurs ==

=== Portage d'avion ===
* Rembrandt calcule les ombres => plus de fausses ombres dans le modèle
* Rembrandt calcule l'occlusion ambiante => pas d'occlusion ambiante incluse dans les textures
* Rembrandt est en lumière => les cartes lumière static ne sont pas nécéssaires, la couleur émissive pour voir les modèles n'est pas nécessaire, et pourrait interférer
* Rembrandt est brillant => les couleurs émissives incorrectement utilisées peuvent rendre flou, et rendre du texte illisible. L'intensité de la lumière devra être réglée
* Rembrandt a des besoins précis avec les shaders => les shaders nécessitent un ajustement pour se conformer au nouveau cadre, sinon la vue sera faussée
* Rembrandt ne peut faire des surfaces transparentes => les surfaces transparentes doivent être dûment enregistrées pour les atteindre avec le chemin classique

=== Enregistrement de toutes les surfaces translucides ===

Chaque modèle est par défaut rendu en utilisant l'effet <tt>Effects/model-default</tt>. Cet effet initialise le G-buffer, en ignorant les surfaces transparentes, en mettant alpha test et rendu de toute la geometrie dans la poubelle. Il n'est pas possible de rediriger le rendu vers les bacs transparents lorsque la texture associée a un canal alpha car la plupart des modeles emploient un atlas simple texture et même les parties opaques sont rendues avec texture avec alpha canal.

Si un modèle nécessite des surfaces translucides ou transparentes , ces objets surface doivent être assignés à un effet différent qui place explicitement le bac rendu à "DepthSortedBin", ou met la suggestion rendu à "transparent". Ceci dit au moteur de rendu de rendre cet objet opérant avant le rendu, ainsi, la clarté et le brouillard doivent être activés, et si un programme shader est utilisé, et s'ils ont été calculés en manière classique. Le <tt>Effects/model-transparent</tt> peut être utilisé pour enregistrer simlement les surfaces transparentes/translucides. Vous attribuez cet effet à un objet (ou à de multiples objets) comme:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

'''Attention: <Effect> agit seulement sur les objets réels, pas sur des groupes d'objets ou des animations.'''

Si la surface opaque a besoin d'un effet spécial, par exemple appliquer un "bump mapping" (effet de relief sur des textures) cet effet devra utiliser le bac "RenderBin", ou le rendu suggéré met à "opaque", et le G-buffer a besoin d'être initialisé correctement dans la phase Geometry.

=== Soyez sûr que toute la Geometry distribuera l'ombre ===

Pour limiter la quantité de rendu geometry dans la carte shadow, et aussi pour réduire les artefacts (acnée shadow), seulement les faces ne regardant pas le soleil distribuent les ombres. Le test est effectué en utilisant l'orientation normale. Cela veut dire que les polygones à double côté, ou mesh non fermés, seront transparents à la lumière sous certains angles de soleil. Pour éviter ça, les modeleurs peuvent:
*s'assurer que l'objet est toujours dans l'ombre d'autres objets,
*fermer leur mesh, ou
*doubler les polygones avec l'endroit normal à l'opposé

=== Ajouts de lumière à un modèle===

Deux points sont à considérer: l'apparence de la sourceet la zone illuminée. Pour l'apparence de la source lumière, (ce que vous voyez quand vous regardez le bulbe) vous avez besoin d'un modèle avec un emissive material qui produira l'effet brillant et ceci est visible de nuit.

Pour l'effet de la source dans son environnement (l'aire lumineuse), nous devons avoir dans le modèle 3D (le fichier .ac) un volume qui inclut l'effet (Volume Lumière). Cela peut être un grand cône pour les aires lumineuses ou une spère pour une lumière ponctuelle. Il est important que le volume lumière soit fermé, convexe et les normals orientés vers l'extérieur.

Le volume lumière doit être partie de la geometry du modèle et être référencé dans le fichier animation. Pas besoin d'ajouter une couleur ou un effet à ce volume. Le calcul lumière est seulement fait sur les fragments couverts par le volume lumière, mais n'a aucune influence sur la couleur ou l'atténuation de la lumière .

Toutes les animations disponibles sont possibles sur le volume lumière sauf '''material''' et '''texture'''. Il n'est pas possible de changer couleur et lumières, pour le moment; sauf en se branchant sur une autre animation. L'axe et la position sont dans l'espace objet et sont transformés par les animations qui suivent.

==== Taches lumineuses ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Nom
!Objectif
|-
|<tt>type</tt>
|Installe l'animation <tt>'''light'''</tt>
|-
|<tt>light-type</tt>
|C'est une lumière <tt>'''spot'''</tt>
|-
|<tt>name</tt>
|Nom donné à cette animation
|-
|<tt>object-name</tt>
|Nom du volume lumière dans le modèle 3D (typiquement un cône avec un sommet à <tt>position</tt>, le long de l'axe <tt>direction</tt> si <tt>cutoff</tt> est inférieur à 90 degrés, ou une sphère centrée à <tt>position</tt> si <tt>cutoff</tt> est supérieur à 90 degrés )
|-
|<tt>nopreview</tt>
|Cache le volume lumière dans fgrun 3d preview
|-
|<tt>position</tt>
|Dans espace objet, position de la lumière
|-
|<tt>direction</tt>
|Dans espace objet, direction vers le centre de la tache
|-
|<tt>ambient</tt>
|Couleur ambiante de la lumière
|-
|<tt>diffuse</tt>
|Couleur diffuse de la lumière
|-
|<tt>specular</tt>
|Couleur specular de la lumière
|-
|<tt>dim-factor</tt>
|Groupe de paramètres pour contrôler un facteur appliqué en même temps à ambient, diffuse et specular
|-
|<tt>attenuation</tt>
|Trois element vector. <c> element est le facteur constant, <l> element est le facteur linéaire et <q> element est le facteur quadratique. 
Attenuation de couleur à distance d est [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplié par <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir étant vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vecteur à partir de la position lumière vers le point, dans l'espace caméra.
|-
|<tt>near-m</tt>
|Distance minimum d'influence, à partir de la position, en mètres
|-
|<tt>far-m</tt>
|Distance maximum d'influence, à partir de la position, en mètres
|}
|}

==== Points lumineux ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Nom
!Objectif
|-
|<tt>type</tt>
|Installe l'animation <tt>'''light'''</tt>
|-
|<tt>light-type</tt>
|C'est une lumière <tt>'''point'''</tt>
|-
|<tt>name</tt>
|Nom donné à cette animation
|-
|<tt>object-name</tt>
|Nom du volume lumière dans le modèle 3D ( typiquement une sphère centrée sur <tt>position</tt>, avec un rayon de <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Volume lumière cachée dans fgrun 3d preview
|-
|<tt>position</tt>
|Dans espace object , position de la lumière
|-
|<tt>ambient</tt>
|Couleur ambiante de la lumière
|-
|<tt>diffuse</tt>
|Couleur diffuse de la lumière
|-
|<tt>specular</tt>
|Couleur specular de la lumièret
|-
|<tt>dim-factor</tt>
|Groupe de paramètres pour contrôler un facteur appliqué à ambient, diffuse et specular en même temps
|-
|<tt>attenuation</tt>
|Trois element vector. <c> element est le facteur constant <l> element est le facteur linéaire et <q> element est le facteur quadratiquer. 
Attenuation de couleur à distance d est [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Distance minimum d'influence, à partir de la position, en mètres
|-
|<tt>far-m</tt>
|Distance maximum d'influence, à partir de la position, en mètres
|}
|}

==== Considérations de performance et de compatibilité ====
Chaque lumière sur écran équivaut, pour le GPU à redessiner le volume lumière avec un shader, sans recueillir de buffer. Ainsi, chaque lumière vient avec un coût, ce qui est minime individuellement parlant, mais non négligeable quand jl y en a beaucoup. Ce coût augmente avec la taille du volume lumière

De plus, il est raisonnable de permettre à un modèle de fonctionner avec le moteur de rendu classique qui ne sait rien des lumières et rendrait les volumes lumière comme une autre géométrie. Ainsi, c'est une bonne pratique de compléter chaque animation lumière par une animation sélective vérifiant :
*si Rembrandt est activé
*si le mode de qualité choisi correspond avec les possibilités de la lumière
*si la lumi§re doit être on ou off selon les autres paramètres de la simulation (position du soleil; position de du commutateur, présence de puissance...)

Une bonne propriété est de crèer un reflet des préférences de l'utilisateur sur qualité vs performance concernant les lumières, et un curseur dédié est ajouté dans le dialogue shader.
[[File:Fgfs-shaders-with-light.jpg|thumb|Light quality slider]]
La propriété à utiliser est:
/sim/rendering/shaders/lights
Le curseur qualité va de 0 (pas de lumière) à 4 (pleine lumière). Un simple réverbère d'aéroport apparaît à 1. Quelques lampes de pont à 2. Toutes les lampes de pont simples à 3, et les plus sophistiquées à 4

Exemple:
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSource</name>
<object-name>LightVolume</object-name>
...
</animation>

<animation>
<type>select</type>

<object-name>LightSource</object-name>
<condition>
<and>

<property>/sim/rendering/rembrandt/enabled</property>

<greater-than>
<property>/sim/rendering/shaders/lights</property>
<value>3.0</value>
</greater-than>

<greater-than>
<property>/sim/time/sun-angle-rad</property>
<value>1.57</value>
</greater-than>
</and>
</condition>
</animation>

</syntaxhighlight>

=== Tutoriels ===
[[User:F-JJTH|F-JJTH]] a compilé son expérience et celle de l'équipe de la PAF d'amélioration d'avions dans {{fr}} [http://equipe-flightgear.forumactif.com/t1034-howto-adapter-un-appareil-pour-rembrandt this tutorial].

== TODO Liste ==

=== Tâches à compléter ===
*Fixer le rendu shadow quand on emploie plusieurs liens en OSG
*Implementer Cascaded Shadow Map (a besoin d'être optimisée - frustum calculation et nuit)
*Voir ce qui arrive avec brillance dans le brouillard: une sorte de terrain inconnu crée des patches blancs dans le buffer emission-problème de génération de scène
*Test multi-écran (mostly done)
*Restorer splashscreen
*Dessiner des objets transparents avec rendu en premier (peut nécessiter la capture du bac transparence à partir de la phase géométrie et le placer dans la phase affichage) (OK - nécessite une contribution modèle)
*Trouver une solution pour couleur ambient et emissive de material (peut nécessiter un buffer additionnel)
*Fournir un shader pour objets transparents qui puisse rendre le buffer emissive infaisable (en utilisant MRT) La phase lumière ne peut utiliser MRT
*Utiliser un buffer stencil pour limiter la taille lumière (non fait dans shader lumière)
**nécessaire pour la lumière de cockpit pour réaliser des faux shadowws et éviter d'éclairer le runway de la cabine
*Utiliser un système effet à la place des shaders lourdement codés
*Intensités globales de brillance, ou occlusion ambiante via curseur dans le dialogue rendu

=== Tâches à court terme ===
*Convertir les shaders existants en rendu différé
*Éviter de redessiner les objets opaques dans la phase lumière. Améliorer le masque node OSG .
*S'occuper des particules et précipitations
*Fixer le brouillard sur les nuages
*Fixer les matrices shadow dans multi-screen
*IMettre en œuvre un lightfield shader

=== Idées à long terme (non classées) ===
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore depth partitioning using depth ranges
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallerie ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendice ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[en:Project Rembrandt]]

FlightGear Newsletter September 2012

2012-09-23T15:10:11Z

Fredb: /* Project Rembrandt */

{{newsletter}}
{{TOC_right|limit=2}}

''We would like to emphasize that the monthly newsletter can not live without the contributions of FlightGear users and developers. Everyone with a wiki account (free to register) can edit the newsletter and every contribution is welcome. So if you know about any FlightGear related news or projects such as for example updated scenery or aircraft, please do feel invited to add such news to the newsletter.''

== Development news ==
Note to all developers: Please also copy your newsletter additions to the changelog of the upcoming release: [[Changelog_3.0.0]].


=== Random Buildings ===
Shortly after the 2.8 release, a number of users reported severe memory growth issues related to a new feature: [[Random Buildings]].

This was a side-effect of the way in which the buildings were created during tile loading. Stuart has now completely changed the way in which random buildings are generated and displayed to reduce the memory occupancy by using a shader-based scheme similar to random trees. You should now see significantly lower memory occupancy (500 - 2GB less depending on area and density) when running with random buildings enabled.

=== NavCache ===
James has pushed a large change to next, which adds a binary cache of most of the navigation data. The cache is stored in FG_HOME/navdata.cache, and rebuilt if the timestamps on any of the data files change (apt.dat, nav.data, fix.dat and so on). When the cache needs to be rebuilt, startup will take a bit longer than before, but when the cache is valid, startup is /much/ faster, especially for debug builds:

{{cquote|Memory consumption is also lower, since we don't keep airports / fixes / taxiways / runways in memory until they're needed.

This is a fairly large change, so please look out for any bugs related to navaids / startup position / GPS / route-manager searches and lookups either getting different results or failing. I've tested here locally and things seem
sane, and have had positive feedback from various testers on IRC and email, but I still expect to find some edge cases which need further work.

There's future work to move even more data into the cache - eg parking positions - which will further help performance for FMS / map systems since we won't need to parse lots of XML data repeatedly. I'm going to let this change settle down before adding more cached data, however.<ref>{{cite web |url=http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg38330.html |title=[Flightgear-devel] Nav-cache|author=James Turner |date=Wed, 19 Sep 2012}}</ref>}}

<references/>

=== Project Rembrandt ===
The night vision effect, as well as the cinema effects, are now available for 2.8 users. See [http://www.flightgear.org/forums/viewtopic.php?f=47&t=17602| here] for the download links, details and feedback.

=== Canvas System ===


Work has now started to develop a dedicated Nasal module specifically for [[Canvas Maps|mapping/charting purposes]]. A first proof of concept is available in $FG_ROOT/Nasal/canvas/map.nas, but we expect to commit more changes (and a real API) soon. Initially, we'll be collecting all related features in a single "map.nas" module, and refactor/improve things later on.

In particular, the idea is to have a single Nasal module as the backend for all mapping/charting tasks in FlightGear, so that existing hard-coded features can be unified and re-implemented via a single consistent Nasal module that serves as a shared backend.

Ultimately, this will make it possible to re-implement existing features such as [[ATC-FS]], the [[Map]] dialog and even Brisa's [[Airport Diagram Generator]] or the [[Kelpie Flight Planner]] directly in FlightGear, using Nasal scripting and the [[Canvas]] system, without having to write any C++ code - so that people won't need to use external tools or programming languages to create such applications.

In the long run, we want to come up with a very flexible design so that the "map.nas" module cannot only be used for creating maps used in GUI dialogs, but also for creating moving map displays like the existing hard-coded [[Navigation display]] or the [[TCAS]] and WXRADAR displays ("glass" displays), i.e. as cockpit instruments/avionics.
This approach would also make it possible to directly show instruments in GUI dialogs or vice versa.

As a first proof of concept, Stuart has added a Canvas-based map to the airport selection dialog, and Hooray has demonstrated that the same technique could be used to add a map widget to the route manager dialog - so that airport selection and route planning could become more intuitive using the [[Canvas]] system. For additional info, please see: [http://flightgear.org/forums/viewtopic.php?f=71&t=17625].

In addition, ShivaVG (the vector drawing engine behind canvas) has been extended to support some kind of a non-zero fill rule. It's not really non-zero because instead of incrementing and decrementing depending on the orientation of the drawn overlapping regions it simply checks if at least a single region covers a pixel. For our use cases it
should be enough tough.

Some further performance improvements were added: Now the texture is only redrawn if something has changed. So if you don't change the map there shouldn't be any noticeable performance impact anymore.

Also the bounding boxes are now only written to the property tree if they are recalculated. This should take some fair amount of changing properties away.

Seems like now tessellating takes much longer than before due to the amount of data in one single path.

The current problem with zooming is that every position of every path gets recalculated with the updated projection, even though this isn't really needed as the projection doesn't noticeably while just zooming
and scrolling around at a single airport.

=== Flightplan access via Scripting ===
In advance of converting the Map and NavDisplay to use the [[Canvas]], James has improved the "flightplan()" extension function of the [[Nasal]] scripting interpreter to expose the full route-path vector for each flight plan leg, as a vector on the leg .

<syntaxhighlight lang="php">
var fp = flightplan();
for (var i=0; i<fp.getPlanSize(); i += 1)
{
var leg = fp.getWP(i);
debug.dump(leg.path());
}
</syntaxhighlight>

With these changes in place, it is already possible to directly show a SVG image for each waypoint and connect them using OpenVG paths, even without separate "icon" support.

=== High Level Architecture ===

=== FlightGear On Android ===

=== Mailing list digest ===

(by far the easiest option to populate the newsletter with contents is copying/pasting stuff from the forum and the mailing list or the git logs)

=== Usability Improvements ===

[[File:Nasal-console-3.0.png|thumb|The Nasal console in FG 3.0 with copy/paste buttons ]]

There have been a number of improvements made to the GUI this month, all aiming to improve usability.

==== Nasal Console ====
The built-in [[Nasal Console]] has been updated to also add native copy/paste support, a feature that got implemented as part of the ongoing [[Canvas]] effort (see [[Howto:Clipboard access using Nasal]] for details ). This feature makes it now possible to easily copy/paste snippets of code FlightGear, without having to use the "dump" feature.

==== About Dialog ====

[[File:About-dlg-3.0.png|thumb|The about dialog in FG 3.0 with extended graphics information ]]

In addition, the "about" dialog has also been slightly updated so that it now shows a handful of OpenGL-related parameters, this info should make it easier for people to provide useful issue reports, especially related to their system's graphics support. For this very purpose, the dialog box has now also support for copying this info to the clipboard or simply take a screen shot that users can post on the forums.

==== Select Airport ====

[[File:Airport-selection-dialog.png|thumb|The new Airport Selection dialog with Canvas display of the airport]]

The Select Airport and Position on Ground dialogs have been combined. The combined dialog displays the lon/lat/alt of the selected airport, and also uses [[Canvas]] to display an airport diagram.

==== Aircraft Checklists ====

Aircraft typically have a large number of checklists, covering almost every phase of flight. Previously, aircraft authors have added checklists to the help text for the aircraft. This is a bit unwieldy. To improve matters, there is now an Aircraft Checklists dialog under the Help menu specifically designed to organize and display checklists. Aircraft authors can create new checklists easily without having to worry about how they will be displayed. See [[Aircraft Checklists]] for details.

==== Integrated Documentation Browser ====

[[File:Nasal-doc-browser-3.0.png|thumb|New integrated documentation browser]]

Often, especially new FlightGear users are not aware of all the documentation that comes with FlightGear. While the manual is fairly easy to find and pretty accessible (being a PDF file), most of the other documentation in $FG_ROOT/Docs is hardly easy to find and often pretty inaccessible to new users, even though these files are standard text files, because of the non-standard file extensions used by most README* files there.
That's why we have now added a new built-in dialog to FlightGear which shows most plain text files in $FG_ROOT/Docs directly inside FlightGear, without having to start any other applications.

=== Forum digest ===

=== Git digest ===

=== Getting involved as a programmer ===

Please see [[Howto:Start core development]]

== Release ChangeLog ==
This section lists changes committed this month that will be available in the next release, these will be copied to the release changelog shortly before a release (for each month), so that we hopefully get a comprehensive list of new features.

== Interview with a contributor (Thorsten Renk) ==
''In each edition we have an interview with a contributor. Suggestions for possible questions are available on [[interview questions]], you are invited to come up with new questions and interview ideas obviously! Anyone is free to write an interview (with him-/herself or others) for next month's newsletter! If you'd like to help interview a contributor or get interviewed, please do consider adding yourself to the [[list of interview volunteers]]! To keep this going and less awkward, we are currently trying to come up with the convention that former interviewees become next month's interviewers.''

* How long have you been involved in FlightGear?

Since late 2009.

* What are your major interests in FlightGear?

In the broadest sense, modelling of the environment. I've started out modelling clouds and weather, then as I discovered shaders I've also included dust, haze and light scattering effects, and of late I am into improved ways of texturing the terrain.

All these areas are related, so it's actually pretty powerful if the same person knows how to affect all of them - for instance, on a rainy day, we not only see an overcast cloud layer, but also the light beneath the clouds is dimmed, the sunrise has a different color hue, the ground is wet and hence darker than usual and there may be water on the ground. Being able to do all these things with the same environment model is pretty cool.

* What project are you working on right now?

Procedural texturing, i.e. computing the texture mix to be displayed on the terrain inside the shader code - this allows for effects which would be impossible to achieve with the standard way of assigning textures due to the huge memory footprint.

* What do you plan on doing in the future?

Who knows? I have plenty of things I would like to do and in principle know how to do - the problem is just making them run fast enough. It's pretty amazing how fast a graphics card can crunch numbers, but there are limits eventually. And basically every day, looking out of the window and observing the richness in detail on a real sky, I see how inadequate even our most advanced shading models really are. It's amazing how nature solves all the rendering equations in real time.

* Are you happy with the way the FlightGear project is going?

Mostly. I know many people think FlightGear should have a project structure which works more like a commercial project, i.e. with clearly defined project goals and well-defined tasks. Rather, it is a somewhat anarchic collection of different individual projects merged together, often with multiple ways to solve the same problem (we support as many as four different ways to generate clouds for instance) which makes it not always easy for new contributers in need of a feature. But, coming from a research environment, I actually feel quite at home with such a development model - science works the same way, and for instance, while having several independent ways of doing the same thing can be seen as a waste of resources, it can also be seen as a feature - we explore many different ways of doing things which helps us find the best solution.

What I don't like is that there is often a lot of friction - for instance 3-D modellers feel underappreciated since most of the project decisions are made by the coders, there's an ongoing controversy between scripting space (Nasal) and C++ coding, and there are contributors of world scenery vs. custom scenery creators. While some level of discussion seems useful to me, related discussions often reach a point which I find no longer productive.

* What do you enjoy most about developing for FlightGear?

The same thing Tolkien enjoyed about writing 'The Lord of the Rings' - the possibility to do subcreation. The visible FlightGear scenery is a world to explore, and this world is directly and visibly influenced by code I write. I can place myself high up in the air, look down and wave my mouse - and a whole layer of clouds appears. I get to decide what color the sunrise has. That's pretty fascinating as far as that goes.

And actually, I've always enjoyed flying through clouds in flight simulators - in the early ones, they were just so bad, but now I can make them just the way I've always wanted them to be. Flying between two scattered cloud layers is still one of my favourite activities.

* Are there any "hidden features" you have worked on in FlightGear that new users may miss?

Plenty. Several I decided not to release since they are too tricky to control, so flags switch them off. For instance, there's a working model for wave lift in the Advanced Weather code, but there is no code to automatically place it (that's much more tricky to develop), so the wave needs to be placed by hand with a suitable line of Nasal. There are experimental cloud layer definitions which are just too expensive to render on a normal system. Many things I just try, and when I find no good way of implementing them, they stay in the code in a de-activated mode.

* What advice can you give to new developers who want to get started on their first aircraft/new feature/Nasal script?

Be patient. The world doesn't revolve around you, but good work is recognized eventually.

Many people seem to expect that the FlightGear development community is just waiting for their contribution and jump to fill every requirement. But every developer has his own agenda, and the way things actually work is by convincing people that spending their time to support another project is a good idea. For instance, demonstrating that a project is feasible is a good idea before expecting help.

I know it looks pretty unwelcoming if someone wants to contribute and needs to go a long way before getting help. On the other hand, I know the other perspective of investing a lot of work into someone's project, just to see the other person disappear and all that work being lost. So I understand much better that established developers want to see that someone is serious about what he proposes to do before they jump in.

More questions are being collected here: [[Interview questions]].

Stay tuned for next month's interview, featuring FlightGear contributor XXXXXXXX

== Snapshot releases ==
Every now and then, easy-to-install development snapshots are created (usually, twice montlhy). These snapshos depict a recent state of the development version of FlightGear. By using them users can test out features that will be included in the upcoming release. Testers are encouraged to file bugs at [http://code.google.com/p/flightgear-bugs/ the issue tracker].

The snapshot can be download via the links at the bottom of this page: http://www.flightgear.org/download/. Updates and feedback can be found [http://flightgear.org/forums/viewtopic.php?f=28&t=10488&p=144233&hilit=snapshot#p144233 at the forum].

== Translators required ==
{|
|[[File:en.gif]]
|The FlightGear Wiki still needs help for translating it into various languages. If you are interested in making the FlightGear Wiki multi-language then start at [[Help:Translate]].
|-
|[[File:de.gif]]
|Das FlightGear Wiki benötigt immer noch Hilfe bei der Übersetzung in verschiedene Sprachen. Wenn Du Interesse daran hast, das FlightGear Wiki Mehrsprachig zu machen, dann fang doch mit [[:de:Help:Übersetzen|Help:Übersetzen]] an.
|-
|[[File:nl.gif]]
|De FlightGear Wiki kan nog steed hulp gebruiken bij het vertalen van artikelen. Als je interesse hebt om de wiki meertalig te maken, raden we je aan om een kijkje te nemen bij [[:nl:Help:Vertalen|Help:Vertalen]].
|}

== Nasal for newbies ==

== New software tools and projects ==

== FlightGear addons and mods ==

== In the hangar ==

All the way back in May 2011, we addopted a new status-rating system for aircraft. So far, only a few have actually been rated, as can be seen in the list 'hockenberry' set up at [https://docs.google.com/spreadsheet/ccc?key=0ApzphjA4w05ndF94Y2F0bzJTbHQ5QTJXZXJRcUVRbWc&hl=en_US Google Docs]. If you're an aircraft developer and your aircraft is/are not on the list, please consider rating their status. All you'll need to know/do is described at [[Formalizing Aircraft Status]]. If you'd just like to get started contributing to FlightGear, this would also seem like an excellent way to get started.

=== New aircraft ===

=== Updated aircraft ===

=== Liveries ===

== Scenery corner ==
=== OpenStreetMap license change completed ===
On September 12, [[OpenStreetMap]] (OSM) completed its long expected license change. From now on, OSM data is licensed under the [http://opendatacommons.org/licenses/odbl/ Open Database Licence].

* [[OpenStreetMap import]]
* [[OpenStreetMap buildings]]

=== TerraGear GUI received new functions ===
The [[TerraGear GUI]], an user interface to generate FlightGear scenery, received several improvements this month. Besides a couple of (small) bug fixes, two features deserve to be highlighted here:
* The list of materials to choose from now displays the respective textures. This makes it easier to pick the right material.
* Elevation data can now be downloaded for the entire world. The GUI does this automatically, all you need to do is specify a bounding box for the to-be-created scenery.
Feedback is welcome as always!

=== Tools ===
It is now possible to update the geographical data of static models using our webtools, just like you are now used to to with the shared objects. You just have to select the object corresponding to the 3D model, and then click on "update".

=== Airports ===

== Aircraft of the month ==
== Airport of the month ==
== Screenshot of the month ==

== Suggested flights ==
== Aircraft reviews ==

== Wiki updates ==
===New articles===
<DynamicArticleList>
type=new
count=10
</DynamicArticleList>
===New aircraft articles===
<DynamicArticleList>
type=new
count=10
categoryRoot=Aircraft
</DynamicArticleList>
===Most popular newsletters===
<DynamicArticleList>
type=hot
count=5
categoryRoot=FlightGear Newsletter
</DynamicArticleList>

== Community news ==
=== FlightGear on YouTube ===

=== New tutorials and screencasts ===
=== Forum news ===
=== Multiplayer ===
=== Virtual airlines ===
=== FlightGear events ===

== Useful links ==
== And finally ... ==
=== Contributing ===
One of the regular thoughts expressed on the FlightGear forums is "I'd like to contribute but I don't know how to program, and I don't have the time". Unfortunately, there is a common mis-conception that contributing requires programming and lots of free time. In fact, there are a huge range of ways to contribute to the project without needing to write code or spending days working on something.

For ideas on starting to contribute to FlightGear, you may want to check out: [[Volunteer]].

To learn more about how the project works, please see [http://flightgear.org/forums/viewtopic.php?f=42&t=15267#p149971 this short essay] written by Thorsten.

=== Call for volunteers ===
* The [[Flightgear On Android]] team is looking for testers
* The [[Target4Today]] team is looking for volunteers to help improving FlightGear's combat support
* The [[OpenRadar]] project is looking for a new maintainer.
* The [[FGFSPM]] (FlightGear Package Manager) is looking for a new maintainer.

=== Did you know ===

[[Category:FlightGear Newsletter]]

Project Rembrandt

2012-09-15T10:20:28Z

Fredb: /* Appendix */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

The ssao effect provided in v2.8.0 may generate these messages, more often on Mac :

<pre>
FRAGMENT glCompileShader "/Users/xxxx/Desktop/FlightGear.app/Contents/Resources/../Resources/data/Shaders/ssao.frag" FAILED
FRAGMENT Shader "/Users/xxxx/Desktop/FlightGear.app/Contents/Resources/../Resources/data/Shaders/ssao.frag" infolog:
ERROR: 0:20: 'array of 2-component vector of float' : constructor not supported for type
ERROR: 0:20: 'array of 2-component vector of float' : no matching overloaded function found
ERROR: 0:20: 'const 2-component vector of float' : cannot declare arrays of this type
ERROR: 0:20: 'v' : redefinition
ERROR: 0:55: 'reflect' : no matching overloaded function found
ERROR: 0:55: '=' : cannot convert from 'float' to '2-component vector of float'

glLinkProgram "" FAILED
Program "" infolog:
ERROR: One or more attached shaders not successfully compiled
</pre>
In that case, disable ambient occlusion with the command :
<tt>--prop:/sim/rendering/rembrandt/ambient-occlusion-buffers=false</tt>

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

Once your aircraft has been ported, please modify its wiki page and add this symbol to it:
[[File:Rembrandtready.png]].

For a list of converted aircraft, please see [http://flightgear.org/forums/viewtopic.php?f=4&t=17536#p166346].

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

'''Beware: <Effect> only works on real objects, not on groups of objects or animations.'''

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Making sure that all geometry will cast shadow ===

To limit the amount of geometry rendered in the shadow map, and also to reduce artifacts (shadow acne), only faces not facing the sun are casting shadows. The test is made using the normal orientation. That means that double-sided polygons, or mesh that are not closed, will be transparent to light at certain sun angles. To avoid that, modelers can either :
* ensure that the object is always in the shadow of another objects,
* close their mesh, or,
* double polygons with the normal set to the opposite.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Performance and compatibility considerations ====
Every light on screen is equivalent for the GPU of redrawing the light volume with a shader, without z buffer culling. So each light comes with a cost, that is small taken individually but noticeable when a lot of them are visible. That cost also increase with the size of the light volume.

Beside that, it is wise to allow a model to work with the classical renderer that know nothing about lights and would render light volumes like other geometry. So a good practice is to complement each light animation with a select animation checking :
*if Rembrandt is enabled
*if the user selected quality match with the purpose of the light
*if the light should be on or off according to the other parameters of the simulation (position of the sun, position of switch, presence of power, ...)

A quality property is created to reflect the user preferences about quality vs performance concerning lights, and a proper slider is added to the shader dialog.
[[File:Fgfs-shaders-with-light.jpg|thumb|Light quality slider]]
The propery to use is :
/sim/rendering/shaders/lights
The quality slider sets its range from 0 (no lights) to 4 (all lights on). Simple airport lamp post appears at 1. Few bridge lamps at 2, all simple bridge lamp at 3 and more involved one at 4.

Example:
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSource</name>
<object-name>LightVolume</object-name>
...
</animation>

<animation>
<type>select</type>

<object-name>LightSource</object-name>
<condition>
<and>

<property>/sim/rendering/rembrandt/enabled</property>

<greater-than>
<property>/sim/rendering/shaders/lights</property>
<value>3.0</value>
</greater-than>

<greater-than>
<property>/sim/time/sun-angle-rad</property>
<value>1.57</value>
</greater-than>
</and>
</condition>
</animation>

</syntaxhighlight>

=== Tutorials ===
[[User:F-JJTH|F-JJTH]] compiled his experience and the one acquired by the PAF team converting aircraft in this [http://equipe-flightgear.forumactif.com/t1034-howto-adapter-un-appareil-pour-rembrandt tutorial (in French)]

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)
*Fix fog on clouds

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix shadow matrices in multi-screen
*Fix Lights in multi screen (seems same problem than shadows. Forgot a 1/w factor ?)
*Implement lightfield shader
*Add some kind of fog to lights
*Restore depth partitioning using depth ranges
*Move effect of cloud coverage from water shader to sunlight shader
*Allow light masks using textures
*Use stencil double-sided operations to limit the depth of light volumes. Use depth clamp to ensure front faces are always rendered even if the camera is enclosed in the light volume.

=== Long term ideas (unsorted) ===
*Use a separate list of light volumes to avoid traversing the scenegraph again (with transparency problems). Find out how we can detect unloading of models.
*implement volumetric effects by extending lights to arbitrary shaders. Should enable to implement heat haze effect and real wake waves.
*implement strength of glow (in the emissive buffer alpha channel) - use a Poisson-disk distribution to implement variable size blur
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
* {{cite web |url=http://publications.dice.se/publications.asp?show_category=yes&which_category=Rendering |author=DICE |title=Publications on rendering |accessdate=15 August 2012 }}
* {{cite web |url=http://advances.realtimerendering.com/s2012/Epic/The%20Technology%20Behind%20the%20Elemental%20Demo%2016x9.pptx |author=Martin Mittring |title=The Technology behind the Unreal Engine 4 Elemental Demo |accessdate=15 September 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

Project Rembrandt

2012-09-15T10:13:33Z

Fredb: /* Guidelines for modelers */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

The ssao effect provided in v2.8.0 may generate these messages, more often on Mac :

<pre>
FRAGMENT glCompileShader "/Users/xxxx/Desktop/FlightGear.app/Contents/Resources/../Resources/data/Shaders/ssao.frag" FAILED
FRAGMENT Shader "/Users/xxxx/Desktop/FlightGear.app/Contents/Resources/../Resources/data/Shaders/ssao.frag" infolog:
ERROR: 0:20: 'array of 2-component vector of float' : constructor not supported for type
ERROR: 0:20: 'array of 2-component vector of float' : no matching overloaded function found
ERROR: 0:20: 'const 2-component vector of float' : cannot declare arrays of this type
ERROR: 0:20: 'v' : redefinition
ERROR: 0:55: 'reflect' : no matching overloaded function found
ERROR: 0:55: '=' : cannot convert from 'float' to '2-component vector of float'

glLinkProgram "" FAILED
Program "" infolog:
ERROR: One or more attached shaders not successfully compiled
</pre>
In that case, disable ambient occlusion with the command :
<tt>--prop:/sim/rendering/rembrandt/ambient-occlusion-buffers=false</tt>

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

Once your aircraft has been ported, please modify its wiki page and add this symbol to it:
[[File:Rembrandtready.png]].

For a list of converted aircraft, please see [http://flightgear.org/forums/viewtopic.php?f=4&t=17536#p166346].

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

'''Beware: <Effect> only works on real objects, not on groups of objects or animations.'''

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Making sure that all geometry will cast shadow ===

To limit the amount of geometry rendered in the shadow map, and also to reduce artifacts (shadow acne), only faces not facing the sun are casting shadows. The test is made using the normal orientation. That means that double-sided polygons, or mesh that are not closed, will be transparent to light at certain sun angles. To avoid that, modelers can either :
* ensure that the object is always in the shadow of another objects,
* close their mesh, or,
* double polygons with the normal set to the opposite.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Performance and compatibility considerations ====
Every light on screen is equivalent for the GPU of redrawing the light volume with a shader, without z buffer culling. So each light comes with a cost, that is small taken individually but noticeable when a lot of them are visible. That cost also increase with the size of the light volume.

Beside that, it is wise to allow a model to work with the classical renderer that know nothing about lights and would render light volumes like other geometry. So a good practice is to complement each light animation with a select animation checking :
*if Rembrandt is enabled
*if the user selected quality match with the purpose of the light
*if the light should be on or off according to the other parameters of the simulation (position of the sun, position of switch, presence of power, ...)

A quality property is created to reflect the user preferences about quality vs performance concerning lights, and a proper slider is added to the shader dialog.
[[File:Fgfs-shaders-with-light.jpg|thumb|Light quality slider]]
The propery to use is :
/sim/rendering/shaders/lights
The quality slider sets its range from 0 (no lights) to 4 (all lights on). Simple airport lamp post appears at 1. Few bridge lamps at 2, all simple bridge lamp at 3 and more involved one at 4.

Example:
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSource</name>
<object-name>LightVolume</object-name>
...
</animation>

<animation>
<type>select</type>

<object-name>LightSource</object-name>
<condition>
<and>

<property>/sim/rendering/rembrandt/enabled</property>

<greater-than>
<property>/sim/rendering/shaders/lights</property>
<value>3.0</value>
</greater-than>

<greater-than>
<property>/sim/time/sun-angle-rad</property>
<value>1.57</value>
</greater-than>
</and>
</condition>
</animation>

</syntaxhighlight>

=== Tutorials ===
[[User:F-JJTH|F-JJTH]] compiled his experience and the one acquired by the PAF team converting aircraft in this [http://equipe-flightgear.forumactif.com/t1034-howto-adapter-un-appareil-pour-rembrandt tutorial (in French)]

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)
*Fix fog on clouds

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix shadow matrices in multi-screen
*Fix Lights in multi screen (seems same problem than shadows. Forgot a 1/w factor ?)
*Implement lightfield shader
*Add some kind of fog to lights
*Restore depth partitioning using depth ranges
*Move effect of cloud coverage from water shader to sunlight shader
*Allow light masks using textures
*Use stencil double-sided operations to limit the depth of light volumes. Use depth clamp to ensure front faces are always rendered even if the camera is enclosed in the light volume.

=== Long term ideas (unsorted) ===
*Use a separate list of light volumes to avoid traversing the scenegraph again (with transparency problems). Find out how we can detect unloading of models.
*implement volumetric effects by extending lights to arbitrary shaders. Should enable to implement heat haze effect and real wake waves.
*implement strength of glow (in the emissive buffer alpha channel) - use a Poisson-disk distribution to implement variable size blur
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
* {{cite web |url=http://publications.dice.se/publications.asp?show_category=yes&which_category=Rendering |author=DICE |title=Publications on rendering |accessdate=15 August 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

Talk:Release plan

2012-09-04T12:52:56Z

Fredb:

==For the benefit of the unwashed masses...==

Would it be acceptable to discuss this proposition on this page? I've never managed to use the dev list properly - it always ends up being rather messy. --[[User Talk:Armchair Ace| Armchair Ace]] 15:00, 23 May 2011 (EDT)

==What about inserting in this cycle a refactory release?==

Hi, I was thinking: wouldn't it sound gound to add a 'refactoring'/performance/bugfixing specific release in this cycle. Maybe use a 3.1 number for instance. This would enable devs and fellow contributors to focus on fixing bugs reported, and try to make performance better on their projects. --[[User Talk:f-ojac| f-ojac]] 09:33, 21 August 2012 (EDT)

: Well, I like the idea, but I think it cannot be handled right now - due to the required workload for each release, just look at how people and resources are still the limiting factor here, even with "just 2" releases per year. This is because of all the manual work that's involved here. I don't think we'll see more releases without some serious automation work, like mentioned at [[Release plan#Open_items.2C_questions]]. Only after the release process is largely automated, can we expect more releases. Also, it's worth noting that, strictly speaking, refactoring is very different from improving performance or bugfixing. Bug fixing in particular is meant to be done during the code freeze period already. But, refactoring is all about keeping the existing behavior, while improving the design and the architecture to make it more maintainable. Now, having a "performance" release would be cool for a number of reasons,and FG can really be a resource hog. And FG eating up 14+ gb of RAM is just ridiculous and could have an impact on the project's reputation.--[[User:Hooray|Hooray]] 04:42, 21 August 2012 (EDT)

==FGRun inclusion==

fgrun is now hosted in the fg project, and appears as a submodule of fgmeta in the master branch. A Jenkins task, [http://flightgear.simpits.org:8080/job/Windows-rel-test/ Windows-rel-test], is configured to demonstrated the feasibility of including it in the final build process. What about including it in the release plan (same for maclauncher btw) ? -- [[User:Fredb|Fredb]] 08:52, 4 September 2012 (EDT)

Project Rembrandt

2012-09-03T18:43:43Z

Fredb: /* Running Flightgear with Rembrandt */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

The ssao effect provided in v2.8.0 may generate these messages, more often on Mac :

<pre>
FRAGMENT glCompileShader "/Users/xxxx/Desktop/FlightGear.app/Contents/Resources/../Resources/data/Shaders/ssao.frag" FAILED
FRAGMENT Shader "/Users/J2MA/Desktop/FlightGear.app/Contents/Resources/../Resources/data/Shaders/ssao.frag" infolog:
ERROR: 0:20: 'array of 2-component vector of float' : constructor not supported for type
ERROR: 0:20: 'array of 2-component vector of float' : no matching overloaded function found
ERROR: 0:20: 'const 2-component vector of float' : cannot declare arrays of this type
ERROR: 0:20: 'v' : redefinition
ERROR: 0:55: 'reflect' : no matching overloaded function found
ERROR: 0:55: '=' : cannot convert from 'float' to '2-component vector of float'

glLinkProgram "" FAILED
Program "" infolog:
ERROR: One or more attached shaders not successfully compiled
</pre>
In that case, disable ambient occlusion with the command :
<tt>--prop:/sim/rendering/rembrandt/ambient-occlusion-buffers=false</tt>

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

'''Beware: <Effect> only works on real objects, not on groups of objects or animations.'''

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Making sure that all geometry will cast shadow ===

To limit the amount of geometry rendered in the shadow map, and also to reduce artifacts (shadow acne), only faces not facing the sun are casting shadows. The test is made using the normal orientation. That means that double-sided polygons, or mesh that are not closed, will be transparent to light at certain sun angles. To avoid that, modelers can either :
* ensure that the object is always in the shadow of another objects,
* close their mesh, or,
* double polygons with the normal set to the opposite.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Performance and compatibility considerations ====
Every light on screen is equivalent for the GPU of redrawing the light volume with a shader, without z buffer culling. So each light comes with a cost, that is small taken individually but noticeable when a lot of them are visible. That cost also increase with the size of the light volume.

Beside that, it is wise to allow a model to work with the classical renderer that know nothing about lights and would render light volumes like other geometry. So a good practice is to complement each light animation with a select animation checking :
*if Rembrandt is enabled
*if the user selected quality match with the purpose of the light
*if the light should be on or off according to the other parameters of the simulation (position of the sun, position of switch, presence of power, ...)

A quality property is created to reflect the user preferences about quality vs performance concerning lights, and a proper slider is added to the shader dialog.
[[File:Fgfs-shaders-with-light.jpg|thumb|Light quality slider]]
The propery to use is :
/sim/rendering/shaders/lights
The quality slider sets its range from 0 (no lights) to 4 (all lights on). Simple airport lamp post appears at 1. Few bridge lamps at 2, all simple bridge lamp at 3 and more involved one at 4.

Example:
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSource</name>
<object-name>LightVolume</object-name>
...
</animation>

<animation>
<type>select</type>

<object-name>LightSource</object-name>
<condition>
<and>

<property>/sim/rendering/rembrandt/enabled</property>

<greater-than>
<property>/sim/rendering/shaders/lights</property>
<value>3.0</value>
</greater-than>

<greater-than>
<property>/sim/time/sun-angle-rad</property>
<value>1.57</value>
</greater-than>
</and>
</condition>
</animation>

</syntaxhighlight>

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)
*Fix fog on clouds

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix shadow matrices in multi-screen
*Fix Lights in multi screen (seems same problem than shadows. Forgot a 1/w factor ?)
*Implement lightfield shader
*Add some kind of fog to lights
*Restore depth partitioning using depth ranges
*Move effect of cloud coverage from water shader to sunlight shader
*Allow light masks using textures
*Use stencil double-sided operations to limit the depth of light volumes. Use depth clamp to ensure front faces are always rendered even if the camera is enclosed in the light volume.

=== Long term ideas (unsorted) ===
*Use a separate list of light volumes to avoid traversing the scenegraph again (with transparency problems). Find out how we can detect unloading of models.
*implement volumetric effects by extending lights to arbitrary shaders. Should enable to implement heat haze effect and real wake waves.
*implement strength of glow (in the emissive buffer alpha channel) - use a Poisson-disk distribution to implement variable size blur
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
* {{cite web |url=http://publications.dice.se/publications.asp?show_category=yes&which_category=Rendering |author=DICE |title=Publications on rendering |accessdate=15 August 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

FlightGear Launch Control

2012-09-02T21:51:01Z

Fredb:

{{Infobox Software
| title = FlightGear Launch Control
| image = Fgrun-page2.jpg
| alt = The second (aircraft selection) page of FGRun.
| developedby = Bernie Bright, Frederic Bouvier
| initialrelease = 0.3.2 (October 24, 2003)
| latestrelease = 1.6.2 (August 17, 2012)
| writtenin = C++
| os = All 32-bit MS Windows (95/98/NT/2000/XP) All POSIX (Linux/BSD/UNIX-like)
| developmentstatus = Active (2002-)
| license = [[GNU General Public License]]
| website = http://sourceforge.net/projects/fgrun
}}
'''FlightGear Launch Control''' ('''FGRun''') is a graphical frontend for [[FlightGear]], distributed and developed separately at [http://sourceforge.net/projects/fgrun] SourceForge. However, it was included in officially distributed binaries for FlightGear, even though it is not technically a part the FlightGear program, like several other frontends (see also [[FlightGear related projects]]). FGRun allows one to choose options more easily than with the [[Command Line Parameters|command line]].

The application has two main parts:
* a wizard to choose common options
* an advanced section where all options are available for advanced users.

Options chosen are automatically saved in a file and are reused the next time FGRun is started.

For information on compiling this software see [[Building FlightGear Launch Control]].

== The wizard ==
The wizard appears to the user when FGRun is started. The first time, when no path are set, the first page is shown, otherwise, it jumps directly to the second page, but the first one is still accessible by hitting the "Prev" button.

=== Navigation bar ===
The bottom of the wizard shows six buttons. From left to right:
* '''Defaults''' : in case of emergency, clear all options and restore initial path set at installation.
* '''Load''' : Load a set of options previously saved.
* '''Save as...''' : Save the current set of options to a file.
* '''Prev''' : go to previous page of the wizard.
* '''Next''' : go to next page of the wizard. This button is replaced by the '''Run''' button in the last page, to start FlightGear.
* '''Quit''' : end the wizard ( and terrasync if it was started by fgrun ).

=== Page One - Paths ===
[[File:fgrun-page1.jpg|thumb]]
In the first page, the paths needed by FGRun are recalled and can be set. If FGRun was installed with a binary package, either under Windows or Linux, these paths are normally initialized by the installation procedure.

* '''Executable''': the full path to the FlightGear program (<tt>fgfs</tt>). One can choose it directly using the file selection dialog that pops up when the folder button is hit.
* '''[[$FG ROOT|FG_ROOT]]''': the full path to the FlightGear base package (<tt>data/</tt> directory). If this path is wrong, no [[aircraft]] would be displayed and FlightGear won't run.
* '''[[$FG_AIRCRAFT|FG_AIRCRAFT]]''': can link to external aircraft packages. Leave empty when all your aircraft are installed in <tt>[[$FG_ROOT]]/Aircraft</tt>.
* '''[[FG SCENERY]]''': the paths to [[scenery]] folders. These folders usually have a 'Terrain', and optionally a 'Objects' , subfolders. You can add a new path with the 'Add...' button. Priorities among folders are managed with the up and down arrows; the top most folder having the highest priority. If there is no scenery for a location in the top folder, FlightGear will look in the next folder etc.
* '''Airports Cache''': the location of the file that stores the [[:Category:Airports|airport]] ids found in the scenery. It is faster to access that file than to scan scenery directories to build the airport list. It is possible to delete that file if things go wrong. That will force a new scan when the third page will be shown.

'''Hint''': If things go wrong and FlightGear cannot start anymore, and you want to start over with a bare installation, remove or rename the file named 'fgrun.prefs' located in the parent directory of the airport cache file. You will have to reenter path by hand, but the silly options you could have entered will be deleted.

=== Page two - Aircraft Selection ===
[[File:fgrun-page2.jpg|thumb]]
The second page, usually the first page seen by users when starting fgrun, is the aircraft selection page. It shows the list of aircraft found in <tt>[[$FG ROOT]]/Aircraft</tt> and all <tt>FG_AIRCRAFT</tt> directories, and the model of the selected aircraft.

=== Page three - Airport Selection ===
[[File:fgrun-page3.jpg|thumb]]
The third page is the Airport and Runway selection box. It only shows the content of the Airport Cache file, that should be the reflect of the content of the scenery folders. If a new scenery has been installed and doesn't show up, or an old scenery has been deleted and airport are still there, click on the 'Refresh' button to rescan the scenery directories. The scan occurs the first time or when the Airport Cache has been deleted.

The airport's list doesn't seem to appear (or ever worse, it disappears) when the refresh button is pressed. To (re)build this list execute the following line once the FlightGear-data is installed:

zcat /usr/share/FlightGear/Airports/metar.dat.gz > \${HOME}/.fltk/flightgear.org/fgrun/airports.txt

=== Page four - Options and Run ===
[[File:fgrun-page4.jpg|thumb]]
The last page shows common options without the need to go to the Advanced section.

:'''Hint'''
: Usually, people new to FlightGear should stick there and experience a simple flight before going to the advanced section. This way, it will be easier to diagnose problem if, by accident, FlightGear would not start or run. Often problems come from incoherent options choosen, and there is no fool guard beyond this limit. If FG doen't run anyway, it will be time to look at the hardware setup.

The command line is helpful to people wanting to diagnose a problem.

The 'Run' button starts FlightGear with the options shown in the command line box.

== Advanced Options ==
The advanced options dialog can be opened through clicking the 'Advanced' button on the last (fourth) page of the launcher.

=== General ===
[[File:fgrun-wiki-a01.jpg|thumb|General]]
* '''FG_SCENERY:''' Where's your [[scenery]] saved on your computer?
* '''Control:''' Which controller do you use?
* '''Airport:''' Your start airport.
* '''Runway:''' Start runway of your airport (see above).
* '''Aircraft:''' Your flying vehicle/[[aircraft]].
* '''Language:''' The language of FlightGear.

=== Features ===
[[File:fgrun-wiki-a02.jpg|thumb|Features]]
* '''Game Mode:''' The full screen game mode.
* '''Random Objects:''' Enable random [[scenery]] objects.
* '''Sound:''' Enable sound effects like engine sounds.
* '''HUD:''' Enable Heads Up Display (HUD).
* '''AI Traffic:''' Enable [[AI traffic]].

=== Flight Model ===
[[File:fgrun-wiki-a03.jpg|thumb|Flight Model]]

=== Freeze ===
[[File:fgrun-wiki-a04.jpg|thumb|Freeze]]
* '''Freeze:''' Start paused'.

=== Initial Position ===
[[File:fgrun-wiki-a05.jpg|thumb|Initial Position]]
* '''Longitude:''' What's your longitude?
* '''Latitude:''' What's your latitude?
* '''Altitude:''' What's your altitude?
* '''Heading:''' The aircraft heading degrees magnetic.
* '''Airspeed:''' Your airspeed in knots.

=== Rendering ===
[[File:fgrun-wiki-a06.jpg|thumb|Rendering]]
* '''Clouds:''' Are there any clouds up in the air?
* '''3D Clouds:''' Are the clouds 3D?
* '''Full Screen:''' Want to play on full screen?
* '''Wireframe:''' Do you wanna see only the wires of the [[scenery]] and the [[aircraft]]?
* '''Visibility:''' Visibility distance in meters or miles.
* '''Bpp:''' Color depth in Bits per pixel.
* '''FOV:''' Field of View angle.
* '''Materials file:''' The materials XML file (default materials.xml, can be changed to materials-dds.xml for DDS scenery textures).

=== Time ===
[[File:fgrun-wiki-a07.jpg|thumb|Time]]
* '''Time match local:''' Synchronisize simulation time with local time.
* '''Time of day:''' What's the time of the day?

=== Network ===
[[File:fgrun-wiki-a08.jpg|thumb|Network]]
* '''Callsign:''' your callsign for the network. This is used to recognize other pilots and for tracking your flights.

'''See:''' [[Multiplayer Howto]]

=== Input/Output ===
[[File:fgrun-wiki-a09.jpg|thumb|Input/Output]]

=== Avionics ===
[[File:fgrun-wiki-a10.jpg|thumb|Avionics]]

=== Properties ===
[[File:fgrun-wiki-a11.jpg|thumb|Properties]]
This is where you could edit or add new property commands. It's same as configuring the [[command line]] with <tt>--prop:...</tt>.

=== Debugging ===
[[File:fgrun-wiki-a12.jpg|thumb|Debugging]]
* '''Log Level:''' The lower the log level, the less errors you'll see. A higher log level can increase the loading time extremely.
* '''Trace Read Properties:'''
* '''Trace Write Properties:'''
=== Environment ===
[[File:fgrun-wiki-a13.jpg|thumb|Environment]]

=== Weather ===
[[File:fgrun-wiki-a14.jpg|thumb|Weather]]
* '''Wind Direction:''' Direction of the wind.
* '''Heading:''' Directon wind is coming from.
* '''Speed:''' Wind speed in knots (kts).

=== Clouds ===
[[File:fgrun-wiki-a15.jpg|thumb|Clouds]]
* '''Elevation:''' How high (in feet) are the clouds?
* '''Thickness:''' How thick (in feet) are the clouds?

== Related content ==
* [[FGo!]], a simple GUI front-end for FlightGear written in Python
* [[JFlightWizard]], another graphical frontend

== External link ==
* [https://gitorious.org/fg/fgrun Git repository]

[[Category:FlightGear front ends]]
[[Category:GPL software]]

[[de:FlightGear Launch Control]]
[[fr:FlightGear Launch Control]]

FlightGear Launch Control

2012-09-02T21:47:17Z

Fredb: /* Page two - Aircraft Selection */

{{Infobox Software
| title = FlightGear Launch Control
| image = Fgrun-page2.jpg
| alt = The second (aircraft selection) page of FGRun.
| developedby = Bernie Bright, Frederic Bouvier
| initialrelease = 0.3.2 (October 24, 2003)
| latestrelease = 1.5.2 (February 6, 2010)
| writtenin = C++
| os = All 32-bit MS Windows (95/98/NT/2000/XP) All POSIX (Linux/BSD/UNIX-like)
| developmentstatus = Active (2002-)
| license = [[GNU General Public License]]
| website = http://sourceforge.net/projects/fgrun
}}
'''FlightGear Launch Control''' ('''FGRun''') is a graphical frontend for [[FlightGear]], distributed and developed separately at [http://sourceforge.net/projects/fgrun] SourceForge. However, it was included in officially distributed binaries for FlightGear, even though it is not technically a part the FlightGear program, like several other frontends (see also [[FlightGear related projects]]). FGRun allows one to choose options more easily than with the [[Command Line Parameters|command line]].

The application has two main parts:
* a wizard to choose common options
* an advanced section where all options are available for advanced users.

Options chosen are automatically saved in a file and are reused the next time FGRun is started.

For information on compiling this software see [[Building FlightGear Launch Control]].

== The wizard ==
The wizard appears to the user when FGRun is started. The first time, when no path are set, the first page is shown, otherwise, it jumps directly to the second page, but the first one is still accessible by hitting the "Prev" button.

=== Navigation bar ===
The bottom of the wizard shows six buttons. From left to right:
* '''Defaults''' : in case of emergency, clear all options and restore initial path set at installation.
* '''Load''' : Load a set of options previously saved.
* '''Save as...''' : Save the current set of options to a file.
* '''Prev''' : go to previous page of the wizard.
* '''Next''' : go to next page of the wizard. This button is replaced by the '''Run''' button in the last page, to start FlightGear.
* '''Quit''' : end the wizard ( and terrasync if it was started by fgrun ).

=== Page One - Paths ===
[[File:fgrun-page1.jpg|thumb]]
In the first page, the paths needed by FGRun are recalled and can be set. If FGRun was installed with a binary package, either under Windows or Linux, these paths are normally initialized by the installation procedure.

* '''Executable''': the full path to the FlightGear program (<tt>fgfs</tt>). One can choose it directly using the file selection dialog that pops up when the folder button is hit.
* '''[[$FG ROOT|FG_ROOT]]''': the full path to the FlightGear base package (<tt>data/</tt> directory). If this path is wrong, no [[aircraft]] would be displayed and FlightGear won't run.
* '''[[$FG_AIRCRAFT|FG_AIRCRAFT]]''': can link to external aircraft packages. Leave empty when all your aircraft are installed in <tt>[[$FG_ROOT]]/Aircraft</tt>.
* '''[[FG SCENERY]]''': the paths to [[scenery]] folders. These folders usually have a 'Terrain', and optionally a 'Objects' , subfolders. You can add a new path with the 'Add...' button. Priorities among folders are managed with the up and down arrows; the top most folder having the highest priority. If there is no scenery for a location in the top folder, FlightGear will look in the next folder etc.
* '''Airports Cache''': the location of the file that stores the [[:Category:Airports|airport]] ids found in the scenery. It is faster to access that file than to scan scenery directories to build the airport list. It is possible to delete that file if things go wrong. That will force a new scan when the third page will be shown.

'''Hint''': If things go wrong and FlightGear cannot start anymore, and you want to start over with a bare installation, remove or rename the file named 'fgrun.prefs' located in the parent directory of the airport cache file. You will have to reenter path by hand, but the silly options you could have entered will be deleted.

=== Page two - Aircraft Selection ===
[[File:fgrun-page2.jpg|thumb]]
The second page, usually the first page seen by users when starting fgrun, is the aircraft selection page. It shows the list of aircraft found in <tt>[[$FG ROOT]]/Aircraft</tt> and all <tt>FG_AIRCRAFT</tt> directories, and the model of the selected aircraft.

=== Page three - Airport Selection ===
[[File:fgrun-page3.jpg|thumb]]
The third page is the Airport and Runway selection box. It only shows the content of the Airport Cache file, that should be the reflect of the content of the scenery folders. If a new scenery has been installed and doesn't show up, or an old scenery has been deleted and airport are still there, click on the 'Refresh' button to rescan the scenery directories. The scan occurs the first time or when the Airport Cache has been deleted.

The airport's list doesn't seem to appear (or ever worse, it disappears) when the refresh button is pressed. To (re)build this list execute the following line once the FlightGear-data is installed:

zcat /usr/share/FlightGear/Airports/metar.dat.gz > \${HOME}/.fltk/flightgear.org/fgrun/airports.txt

=== Page four - Options and Run ===
[[File:fgrun-page4.jpg|thumb]]
The last page shows common options without the need to go to the Advanced section.

:'''Hint'''
: Usually, people new to FlightGear should stick there and experience a simple flight before going to the advanced section. This way, it will be easier to diagnose problem if, by accident, FlightGear would not start or run. Often problems come from incoherent options choosen, and there is no fool guard beyond this limit. If FG doen't run anyway, it will be time to look at the hardware setup.

The command line is helpful to people wanting to diagnose a problem.

The 'Run' button starts FlightGear with the options shown in the command line box.

== Advanced Options ==
The advanced options dialog can be opened through clicking the 'Advanced' button on the last (fourth) page of the launcher.

=== General ===
[[File:fgrun-wiki-a01.jpg|thumb|General]]
* '''FG_SCENERY:''' Where's your [[scenery]] saved on your computer?
* '''Control:''' Which controller do you use?
* '''Airport:''' Your start airport.
* '''Runway:''' Start runway of your airport (see above).
* '''Aircraft:''' Your flying vehicle/[[aircraft]].
* '''Language:''' The language of FlightGear.

=== Features ===
[[File:fgrun-wiki-a02.jpg|thumb|Features]]
* '''Game Mode:''' The full screen game mode.
* '''Random Objects:''' Enable random [[scenery]] objects.
* '''Sound:''' Enable sound effects like engine sounds.
* '''HUD:''' Enable Heads Up Display (HUD).
* '''AI Traffic:''' Enable [[AI traffic]].

=== Flight Model ===
[[File:fgrun-wiki-a03.jpg|thumb|Flight Model]]

=== Freeze ===
[[File:fgrun-wiki-a04.jpg|thumb|Freeze]]
* '''Freeze:''' Start paused'.

=== Initial Position ===
[[File:fgrun-wiki-a05.jpg|thumb|Initial Position]]
* '''Longitude:''' What's your longitude?
* '''Latitude:''' What's your latitude?
* '''Altitude:''' What's your altitude?
* '''Heading:''' The aircraft heading degrees magnetic.
* '''Airspeed:''' Your airspeed in knots.

=== Rendering ===
[[File:fgrun-wiki-a06.jpg|thumb|Rendering]]
* '''Clouds:''' Are there any clouds up in the air?
* '''3D Clouds:''' Are the clouds 3D?
* '''Full Screen:''' Want to play on full screen?
* '''Wireframe:''' Do you wanna see only the wires of the [[scenery]] and the [[aircraft]]?
* '''Visibility:''' Visibility distance in meters or miles.
* '''Bpp:''' Color depth in Bits per pixel.
* '''FOV:''' Field of View angle.
* '''Materials file:''' The materials XML file (default materials.xml, can be changed to materials-dds.xml for DDS scenery textures).

=== Time ===
[[File:fgrun-wiki-a07.jpg|thumb|Time]]
* '''Time match local:''' Synchronisize simulation time with local time.
* '''Time of day:''' What's the time of the day?

=== Network ===
[[File:fgrun-wiki-a08.jpg|thumb|Network]]
* '''Callsign:''' your callsign for the network. This is used to recognize other pilots and for tracking your flights.

'''See:''' [[Multiplayer Howto]]

=== Input/Output ===
[[File:fgrun-wiki-a09.jpg|thumb|Input/Output]]

=== Avionics ===
[[File:fgrun-wiki-a10.jpg|thumb|Avionics]]

=== Properties ===
[[File:fgrun-wiki-a11.jpg|thumb|Properties]]
This is where you could edit or add new property commands. It's same as configuring the [[command line]] with <tt>--prop:...</tt>.

=== Debugging ===
[[File:fgrun-wiki-a12.jpg|thumb|Debugging]]
* '''Log Level:''' The lower the log level, the less errors you'll see. A higher log level can increase the loading time extremely.
* '''Trace Read Properties:'''
* '''Trace Write Properties:'''
=== Environment ===
[[File:fgrun-wiki-a13.jpg|thumb|Environment]]

=== Weather ===
[[File:fgrun-wiki-a14.jpg|thumb|Weather]]
* '''Wind Direction:''' Direction of the wind.
* '''Heading:''' Directon wind is coming from.
* '''Speed:''' Wind speed in knots (kts).

=== Clouds ===
[[File:fgrun-wiki-a15.jpg|thumb|Clouds]]
* '''Elevation:''' How high (in feet) are the clouds?
* '''Thickness:''' How thick (in feet) are the clouds?

== Related content ==
* [[FGo!]], a simple GUI front-end for FlightGear written in Python
* [[JFlightWizard]], another graphical frontend

== External link ==
* [https://gitorious.org/fg/fgrun Git repository]

[[Category:FlightGear front ends]]
[[Category:GPL software]]

[[de:FlightGear Launch Control]]
[[fr:FlightGear Launch Control]]

Project Rembrandt

2012-09-01T22:18:56Z

Fredb: "error C6013" shouldn't appear because Rembrandt no longer use arrays in fragment shaders for shadows

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

'''Beware: <Effect> only works on real objects, not on groups of objects or animations.'''

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Making sure that all geometry will cast shadow ===

To limit the amount of geometry rendered in the shadow map, and also to reduce artifacts (shadow acne), only faces not facing the sun are casting shadows. The test is made using the normal orientation. That means that double-sided polygons, or mesh that are not closed, will be transparent to light at certain sun angles. To avoid that, modelers can either :
* ensure that the object is always in the shadow of another objects,
* close their mesh, or,
* double polygons with the normal set to the opposite.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Performance and compatibility considerations ====
Every light on screen is equivalent for the GPU of redrawing the light volume with a shader, without z buffer culling. So each light comes with a cost, that is small taken individually but noticeable when a lot of them are visible. That cost also increase with the size of the light volume.

Beside that, it is wise to allow a model to work with the classical renderer that know nothing about lights and would render light volumes like other geometry. So a good practice is to complement each light animation with a select animation checking :
*if Rembrandt is enabled
*if the user selected quality match with the purpose of the light
*if the light should be on or off according to the other parameters of the simulation (position of the sun, position of switch, presence of power, ...)

A quality property is created to reflect the user preferences about quality vs performance concerning lights, and a proper slider is added to the shader dialog.
[[File:Fgfs-shaders-with-light.jpg|thumb|Light quality slider]]
The propery to use is :
/sim/rendering/shaders/lights
The quality slider sets its range from 0 (no lights) to 4 (all lights on). Simple airport lamp post appears at 1. Few bridge lamps at 2, all simple bridge lamp at 3 and more involved one at 4.

Example:
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSource</name>
<object-name>LightVolume</object-name>
...
</animation>

<animation>
<type>select</type>

<object-name>LightSource</object-name>
<condition>
<and>

<property>/sim/rendering/rembrandt/enabled</property>

<greater-than>
<property>/sim/rendering/shaders/lights</property>
<value>3.0</value>
</greater-than>

<greater-than>
<property>/sim/time/sun-angle-rad</property>
<value>1.57</value>
</greater-than>
</and>
</condition>
</animation>

</syntaxhighlight>

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)
*Fix fog on clouds

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix shadow matrices in multi-screen
*Fix Lights in multi screen (seems same problem than shadows. Forgot a 1/w factor ?)
*Implement lightfield shader
*Add some kind of fog to lights
*Restore depth partitioning using depth ranges
*Move effect of cloud coverage from water shader to sunlight shader
*Allow light masks using textures
*Use stencil double-sided operations to limit the depth of light volumes. Use depth clamp to ensure front faces are always rendered even if the camera is enclosed in the light volume.

=== Long term ideas (unsorted) ===
*Use a separate list of light volumes to avoid traversing the scenegraph again (with transparency problems). Find out how we can detect unloading of models.
*implement volumetric effects by extending lights to arbitrary shaders. Should enable to implement heat haze effect and real wake waves.
*implement strength of glow (in the emissive buffer alpha channel) - use a Poisson-disk distribution to implement variable size blur
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
* {{cite web |url=http://publications.dice.se/publications.asp?show_category=yes&which_category=Rendering |author=DICE |title=Publications on rendering |accessdate=15 August 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

Fr/Projet Rembrandt

2012-09-01T22:14:07Z

Fredb: /* Pipeline configurable */

{{WIP}}
== Pourquoi ce nom? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] était un peintre hollandais du XVIIème siècle, maître incontesté du [http://en.wikipedia.org/wiki/Chiaroscuro "clair-obscur"].

Ce projet a pour but de changer la manière de rendre les éclairages et les [[Shadows|ombres]] qu'utilise [[FlightGear]], et d'essayer d'imiter le style de Rembrandt dans FG.

== De quoi s'agit-il ? ==
L'idée maîtresse du projet est d'implémenter le [http://en.wikipedia.org/wiki/Deferred_shading rendu différé] au sein de FlightGear.
Depuis le début, FlightGear utilise un moteur de rendu en avant qui affiche toutes les propriétés d'un objet en une fois (ombre, éclairage, brouillard, ...), ce qui rend difficile le rendu d'un ombrage plus sophistiqué (voir le [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) du fait que tous les paramètres de [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ l'équation lumineuse] doivent être traités en une seule fois.

[[Image:project_rembrandt_1.png|thumb|300px|Vue principale avec le contenu des buffers affichés dans les coins]]
Au contraire, le rendu différé cherche à séparer les opérations en étapes simplifiées et à collecter les résultats intermédiaires dans des buffers cachés, qui peuvent être utilisés par la prochain étape.

;Le premier stade est celui de la Géométrie:
: nous rendons la scène entière dans 4 textures, en utilisant le rendu multi-cibles pour les traiter en un seul passage: un pour le buffer de profondeur, un pour les normales (en bas à gauche de l'image), un pour les couleurs "diffuses" (en bas à gauche) et un pour les couleurs "spéculaires" (en haut à droite).

;Le stade suivant est celui de l'ombre :
: nous rendons encore la scène dans une texture profondeur concernant les éclairages. Il y aura une texture pour chaque lumière diffusant des ombres.

;Ensuite, c'est le stade de l'éclairage, avec plusieurs sous-stades :
:*Sky pass: Le ciel est dessiné d'abord selon la méthode classique.
:*Ambient pass: la mémoire-tampon "diffuse" est modulé avec la couleur ambiante de la scène, et est dessiné comme un quad texturé aligné sur l'écran
:*Sunlight pass: Un second quad aligné à l'écran est dessiné et un shader examine la position pour calculer sa couleur "diffuse" et "specular" en utilisant le "normal" stocké dans le premier stade. La couleur résultante et mélangée avec la passe précédente. Les ombres sont examinées ici par comparaison de la position du pixel avec la position du cache lumière stockée dans la carte shadows

:*stade de la lumière additionnelle: le scene graph sera traversé une autre fois pour afficher les volumes de lumière (cône ou tronc de cône pour les taches de lumière, sphère pour les lumières omni directionnelles), et leur shader ajoutera la lumière composée uniquement par les pixels éclairés.
:*stade brouillard: un nouveau quad aligné sur l'écran est dessiné, et la position du pixel est calculée pour évaluer la quantité de brouillard du pixel. La couleur du brouillard est mélangée avec le résultat du stade précédent.
:*Stade des objets transparents: les objets transparents (et les nuages) sont enfin rendus en utilisant la méthode classique.

:Tous les données de lumière sont accumulées dans une seule mémoire tampon, qui sera utilisée pour le dernier stade en addition au stade Géométrie.

;A la fin, le stade affichage, avec l'option effet post-traitement :
:Les résultats des précédents tampons sont poussés dans le tampon principal pour être affichés, optionnellement modifiés pour montrer Glow, Motion blur, HDR, redout ou blackout, occlusion ambiante de l'espace écran, anti-aliasing, etc...

Dans FG, on termine le pipeline du rendu par l'affichage du [[Menubar|GUI]] et du [[HUD]].

Tous ces phases sont plus précisément décrites dans [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] c'est la base du code actuel, avec quelque addition ou modifications.

== Avertissements ==
Le rendu différé n'affiche pas la transparence. Pour le moment, les nuages doivent être éclairés et ombrés par eux-mêmes. Les surfaces transparentes sont alpha-testées et non mélangées. Elles doivent être amenées dans leur propre bin sur l'image composite.

La partition profondeur ne marche pas non plus, à cause de la mémoire tampon profondeur, qui doit être gardée pour retenir la position de zone de vue, et, pour le moment, z-fighting est tout à fait visible. La partition profondeur sans rangée de profondeur superposée pourrait être la solution, et devrait être essayée.

Le passage de la lumière peut rendre certains MFD illisibles (ce qui utilisent une couleur émissive) car flous. Ils devront être traités comme transparents.

== Mise en œuvre ==
=== Répertoires ===
Le code est la principale branche du [[FlightGear and Git|répertoire officiel]]. Aucune autre localisation n'existe.

=== Rendu des surfaces transparentes ===
[[Image:project_rembrandt_5.png|thumb|300px|Surfaces transparentes desssinées d'après des objets opaques]]
Les surfaces transparentes sont détectées par les plugins OSG loader qui captent la nuance de rendu TRANSPARENT_BIN
In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Consommation de mémoire ===
Pour chaque caméra définie du groupe, il y a une carte d'ombre séparée, de telle sorte que l'utilisation de la mémoire vidéo soit:
* G-tampon et tampon lumière: 20 bytes par pixel. Pour un écran HD (1920x1080) la mémoire requise est de 40 Mb
* Carte d'ombre 3 x taille_carte_ombre x shadow_map_size bytes (si la taille est 8192, la taille totale de la mémoire est 192 Mb
Ne pas compter les textures, liste d'affichage ou tampons vertex pour kes modèles et terrains

3 écrans HD ont besoin de 120 Mb de mémoire pour les tampons (ombre exclue), il vous faut 3x8192x8192x3 = 576 Mb (megabytes) de mémoire pour les seuls shadows.

Si vous voyez des messages d'erreur pendant le démarrage, ou si FlightGear ne démarre pas bien, c'est probablement parceque vous n'avez pas assez de mémoire vidéo libre. Réduisez la taille de la carte d'ombre dans préférences.xml

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

Et mettez 4096 ou 2048 à la place.
Vous pouvez aussi utiliser un paramètre de démarrage: --prop:/sim/rendering/shadows/map-size=2048

=== Pipeline configurable ===

Le rendu Rembrandt utilise un fichier XML pour installer son pipeline pour chaque fenêtre décrite dans le groupe de cameras. Ce fichier mntre la façon dont les tampons intermédiaires sont établis, et comment les différentes étapes du rendu sont agencés. Le schéma général du fichier pipeline est le suivant :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Tampons ====
Un tampon est une texture utilisée comme zone de stockage dans le GPU. Sa taille est habituellement un multiple de la taille de l'écran, mais une taille fixe est prise en charge ( typique pour la carte d'ombres). La description d'un tampon suit :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix fog on clouds
*Fix shadow matrices in multi-screen
*Implement lightfield shader

=== Long term ideas (unsorted) ===
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore depth partitioning using depth ranges
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[en:Project Rembrandt]]

FlightGear Newsletter August 2012

2012-09-01T22:07:35Z

Fredb: /* FlightGear 2.8.0 Released ! */

{{newsletter}}
{{TOC_right|limit=2}}

''We would like to emphasize that the monthly newsletter can not live without the contributions of FlightGear users and developers. Everyone with a wiki account (free to register) can edit the newsletter and every contribution is welcome. So if you know about any FlightGear related news or projects such as for example updated scenery or aircraft, please do feel invited to add such news to the newsletter.''

== FlightGear 2.8.0 Released ! ==

The FlightGear team is very excited to announce the v2.8.0 Release is now available! I will be brief and simply post the link to the official release announcement.

http://www.flightgear.org/news/flightgear-v2-8-0-released/

Some of the notable changes and improvements include:

Major improvements from v2.6.0 include improved AI aircraft, up to date with all the recent JSBSim flight dynamics model improvements, support for region specific textures, better random 3d buildings, better random object placement, 3d airport signage, in-sim toggling between summer and winter texture sets. The core FlightGear code now supports a flexible 2d rendering system which can be used to accurately model glass cockpit displays and other complex instrumentation. There are many new and improved aircraft to explore. A new automated system is now available for scenery submissions that automatically get rolled into the scenery distribution to be enjoyed by everyone. Version 2.8 has improved atmospheric light scatter modeling, better terrain haze, and improved 3d clouds.

A very exciting new addition is “Project Rembrandt”. This is still considered “experimental” and not enabled by default. But when you turn it on you will get real-time shadows, support for multiple light sources (landing lights, instrument lights, directional ground spotlights, even the rotating beacons illuminate the surrounding correctly.) In addition, Project Rembrandt offers a variety of other visual effects such as night vision, film grain effects, depth of field effects, lens (i.e. fisheye) distortion — all in real time.

For the full change log, please see [[Changelog 2.8.0]].

Enjoy!!!

== Development news ==
A view of FlightGear core and fgdata activity in [[Git]] since the 2.6.0 release. Each committer on fg and fg-data is shown here, as well as the part of fg where he's been working on. A big thank to all involved for making FG such a nice fireworks!
{{#ev:youtube|ibO6ruzQ5gM}} {{#ev:youtube|U0Dsgx9-Pbk}}

=== Project Rembrandt ===
Fred's been working on some fancy new effects, especially useful for our film-making enthusiasts.

The [[Project Rembrandt|Rembrandt renderer]] has a lot of potentiality when it comes to post processing effects. Indeed, the interesting property of a deferred shader is that all stages of rendering are kept into memory. Post-processing is as simple as using a filter on a camera.

So currently the effects added in git are :
* Night vision including amplification grain and restrained field of view,
* Cinema effect including :
** Vignetting,
** Color shift, with Sepia as default value,
** Radial distortion (barrel and pincushion distortion, with scale compensation),
** Lateral chromatic aberration (purple fringing),
** Film wear simulation.

{|
| [[File:Fgfs-rembrandt-old-timer.jpg|thumb|400px|Rembrandt old film style]]
| [[File:Fgfs-rembrandt-distortion.jpg|thumb|400px|Rembrandt ordinary lens simulation]]
| [[File:Fgfs-rembrandt-night-vision-2.png|thumb|400px|Rembrandt night vision simulation]]
|}
This won't make it for the 2.8 release but it's likely that an add-on will be available as it is contained in the data.

See also :
{|
| {{#ev:youtube|tLZzlLjvDRQ}}
| {{#ev:youtube|_Dm-Xx5tfiE}}
| {{#ev:youtube|8jBvVyiq-D0}}
|}

=== Unified Weather GUI ===
As most readers will be aware, FG supports two weather models - Basic Weather which represents METAR data directly, and Local Weather which models weather more realistically, with more variation. In 2.8.0 these were configured separately using different UIs. Stuart Buchanan and Thorsten Renk have been working on unifying the user interfaces to provide a more user-friendly and consistent interface. This work is now available in git.

=== Joystick Configuration GUI ===

[[File:Joystick Configuration Dialog.jpg|thumb|The new Joystick Configuration Dialog, available in git.]]

A further usability enhancement has been made by Stuart Buchanan to allow configuration of joysticks through a GUI in-sim. Instead of configuring a joystick using XML, users can now change joystick bindings within the simulator - no XML knowledge required. The updated joystick configuration takes effect immediately. This work is also available in git, and should make it much easier for new users (or those with joysticks unknown to FG) to get flying.

=== Canvas News ===


As of 08/2012, the new [[Canvas]] 2D drawing system in FlightGear HEAD has now also support for:
* using raster images (vector images were already supported previously)
* nested canvases, where a canvas may contain images created by another canvas texture [[Howto:Using raster images and nested canvases]]
* creating GUI windows (not just widgets)
* handling GUI events using osgGA
* multiplatfom copy/paste support using two new Nasal functions: [[Howto:Clipboard access using Nasal]]

Support for nested canvases will make it possible to also load canvas textures into other canvases, so that you can, for example, easily load an instrument into a GUI dialog, or even use GUI widgets in MFD instruments, which is a feature based on real avionics, i.e. as used in modern airliners like the A320 or the 787.

Nested canvases also make it possible for people to easily implement GUI tools using the Canvas system, for example a GUI instrument or panel editor can be entirely implemented in scripting space now, without touching any C++ code. Similarly, GUI widgets could also be created in a WYSIWYG-fashion, too - so that even dialog editor or full GUI builders can be built using the Canvas system.

In addition, Tom has now implemented support for window stacking. Check out the demo video below, which demonstrates how "nested" canvases and window stacking works:

{{#ev:youtube|llVaasTEf44}}

Also, Tom is currently working on an improved event handling system with support for event-propagation based on the Javascript/DOM model. The event handling systems requires close interaction between the core system and Nasal code. Event propagation and bubbling will be supported like in the W3C DOM standard. We need eg. to be able to mark an event as handled and maybe also other things which change the behavior of the propagation.

In addition, Tom has started work on some building blocks to provide basic inheritance support for canvas properties, so that styling (CSS/XSL) will be much easier.

Missing features, and features currently under development, are listed at [[Canvas Widgets#Missing_.2F_Todo|Missing Canvas Features]]

=== FlightGear Run-Levels ===

Recently, there was talk on the FlightGear developers mailing about implementing a standalone [[FGCanvas]] client, analogous to [[FGPanel]] - in order to be able to render [[Canvas]] instruments separately, in different processes and also on different computers.

However, obviously this new startup mode would not need many of the default subsystems, such as the FDM, sound, terrain etc. This requirement is shared with the [[Flightgear On Android]] project. But, many of these default subsystems are initialized in a hard coded fashion so that they cannot currently be made optional even though they may not be stricly required.
Also, for a very long time, it's never been possible to fully reset FlightGear properly, this makes many things very difficult or even impossible, including the [[FGCanvas]] idea.

Hooray and TheTom have now started collecting ideas on how this could be changed, by making subsystem initialization more explicit using listeners, so that SGSubsystems can be dynamically allocated and freed based on setting signal properties, so that recursive run levels can be implemented. For more information, please see [[FlightGear Run Levels]]. Feedback welcome!

=== Mailing list digest ===

(by far the easiest option to populate the newsletter with contents is copying/pasting stuff from the forum and the mailing list or the git logs)

=== Forum digest ===

=== '''Flightgear On Android''' ===

A new developer has achieved a way to run Flightgear on Android phones. [[File:Flightgear Android Launcher.png|thumb|FOA]]

The project codename will be FOA;he has done a lot of Android and C++ coding. [[File:FOA.jpg|thumb|FOA ingame]]

His phone is a Samsung Galaxy Nexus and is running Android 4.04 (custom ROM but stock ROM should be OK). If your ready to test with us, please read the technical info then send a email to lacbrc@gmail.com!

TECHNICAL INFO:
It should have at least a 1GHz processor and decent graphics chip. Android 2.3 is a
must but a rooted device is not needed but can improve performance.
Not much in this, as we haven't finished it to a release yet.
He tested it on a bluetooth keyboard and a wired one through usb host mode. Hope you are interested in it!

The first ALPHA release is fresh out of its release, and you can read more here:
http://www.flightgear.org/forums/viewtopic.php?f=18&t=16719&p=163745#p163745

The FlightGear for Android Team

Here is a few links:

Forums:

http://www.flightgear.org/forums/viewtopic.php?f=18&t=16719

Website page:

http://horizonflightgearhangar.yolasite.com/fg-on-android.php

Wiki page:

http://wiki.flightgear.org/Flightgear_On_Android

=== Git digest ===

=== Getting involved as a programmer ===

Please see [[Howto:Start core development]]

=== Translators required ===
{|
|[[File:en.gif]]
|The FlightGear Wiki still needs help for translating it into various languages. If you are interested in making the FlightGear Wiki multi-language then start at [[Help:Translate]].
|-
|[[File:de.gif]]
|Das FlightGear Wiki benötigt immer noch Hilfe bei der Übersetzung in verschiedene Sprachen. Wenn Du Interesse daran hast, das FlightGear Wiki Mehrsprachig zu machen, dann fang doch mit [[:de:Help:Übersetzen|Help:Übersetzen]] an.
|-
|[[File:nl.gif]]
|De FlightGear Wiki kan nog steed hulp gebruiken bij het vertalen van artikelen. Als je interesse hebt om de wiki meertalig te maken, raden we je aan om een kijkje te nemen bij [[:nl:Help:Vertalen|Help:Vertalen]].
|}

== Interview with a contributor (NAME) ==
''In each edition we have an interview with a contributor. Suggestions for possible questions are available on [[interview questions]], you are invited to come up with new questions and interview ideas obviously! Anyone is free to write an interview (with him-/herself or others) for next month's newsletter! If you'd like to help interview a contributor or get interviewed, please do consider adding yourself to the [[list of interview volunteers]]! To keep this going and less awkward, we are currently trying to come up with the convention that former interviewees become next month's interviewers.''

* How long have you been involved in FlightGear?
* What are your major interests in FlightGear?
* What project are you working on right now?
* What do you plan on doing in the future?
* Are you happy with the way the FlightGear project is going?
* What do you enjoy most about developing for FlightGear?
* Are there any "hidden features" you have worked on in FlightGear that new users may miss?
* What advice can you give to new developers who want to get started on their first aircraft/new feature/Nasal script?

More questions are being collected here: [[Interview questions]].

Stay tuned for next month's interview, featuring FlightGear contributor XXXXXXXX

== Snapshot releases ==
Every now and then, easy-to-install development snapshots are created (usually, twice montlhy). These snapshos depict a recent state of the development version of FlightGear. By using them users can test out features that will be included in the upcoming release. Testers are encouraged to file bugs at [http://code.google.com/p/flightgear-bugs/ the issue tracker].

The snapshot can be download via the links at the bottom of this page: http://www.flightgear.org/download/. Updates and feedback can be found [http://flightgear.org/forums/viewtopic.php?f=28&t=10488&p=144233&hilit=snapshot#p144233 at the forum].

== Nasal for newbies ==

== New software tools and projects ==

== FlightGear addons and mods ==

== In the hangar ==

All the way back in May 2011, we addopted a new status-rating system for aircraft. So far, only a few have actually been rated, as can be seen in the list 'hockenberry' set up at [https://docs.google.com/spreadsheet/ccc?key=0ApzphjA4w05ndF94Y2F0bzJTbHQ5QTJXZXJRcUVRbWc&hl=en_US Google Docs]. If you're an aircraft developer and your aircraft is/are not on the list, please consider rating their status. All you'll need to know/do is described at [[Formalizing Aircraft Status]]. If you'd just like to get started contributing to FlightGear, this would also seem like an excellent way to get started.

=== New hangers ===

=== New aircraft ===

At the request by a new user of FG, Helijah has created and made available the Yakovlev Yak 18T in his hangar and GIT.
[[File:Yak18t-splash.png|thumb|Yakovlev Yak 18T]]

=== Updated aircraft ===

==== [[ATC-FS]] ====
The Air Traffic Control tool ATC-FS got an update. The old radar screen was replaced by a radar screen, which can display fixes, navaids and ILS localizers. This new radar is resizable, so you can use your big screen more efficient.

The next version is already being worked on, it will reimplement ATC-FS using the new [[Canvas]] system (scheduled for FG 3.0) so that a fully custom drawn radar screen can be created in scripting space, including custom GUI widgets (e.g. for flight strips) - without depending on any hard coded instruments or dialogs any longer.

This new design will eventually even make it possible to reuse ATC-FS for other purposes, or even include it in an AWACS aircraft that is multiplayer-enabled.

[[Image:ATC-FS.jpg|1000px]]

==== [[Mil_Mi6]] ====
The big old russian helicopter Mi-6 from the Mil design bureau has received a new update. More than a year after the last update a new structure of the Models-subfolder allows new instruments to be added much easier. Also the electrical system has been revisited and new instruments have beed added to the engineer panels. Please keep in mind that this still is a very early development version and therefore a lot of work still needs to be done. It is available from its [https://gitorious.org/mil-mi-6 gitorious-page.]

=== Liveries ===

== Scenery corner ==
=== 3D models import webform ===
The webform to import 3D models directly into the DB is in production. Now you can use this tool to send your models to the database used by Terrasync. If everything is fine and your model respects the specifications written in the foreword, then it will quickly be shipped to all FG users worldwide through [[Terrasync]]. Yet another unique feature amongst the flight simulation world held by FlightGear. Thanks to Clément, Julien, Martin, for their help & support, and all of FG users who showed interest and support in those tools.

Already 100 3D models have been imported with this form in Shanghai, Berlin, Stuttgart, France and in the UK! See the statistics [http://scenemodels.flightgear.org/stats.php here]. Thanks Gijs for the new nice layout.

With all the forms now working on the scenemodels website, remember you can:
* add a single object (if you only have one!!). Check [http://scenemodels.flightgear.org/submission/index.php here]!
* update an object (for instance, a lot of power pylons are in fact communication pylones). It's [http://scenemodels.flightgear.org/submission/shared/index_update.php here]!
* delete an object (if it does not exist anymore in real life). It's [http://scenemodels.flightgear.org/submission/shared/index_delete.php here]!
* run a mass object positions import (if you added a lot of objets in one place). It's [http://scenemodels.flightgear.org/submission/shared/index_mass_import.php here]!
* add 3D models you created to have them imported into FG scenery via Terrasync. It's [http://scenemodels.flightgear.org/submission/static/index.php here]: they can be specific to a city (for instance: the Eiffel tower), that's what is called 'static' in the FG world, or 'shared' (a new windturbine model, for instance).

Please read carefully the various forewords on the site:
* all data imported has to comply with the GNU/GPL license ;
* you must use the elevation format used by Terrasync to give your objects/models elevation, and [http://scenemodels.flightgear.org/contribute.php#offset an offset] if necessary, or they could be floating/sunk for most of users.

We hope to see a lot of scenes not yet available via Terrasync merge into scenemodels. Nothing is now easier, especially when used in conjunction with our Mapserver!
Remember that this is the strength of FG compared to other sims: everything can be in Terrasync and no need to download patches or sceneries here and there.

Moreover, an Ac3D viewer using Hangar by Juan Mellado is also available for general public for those whose browser supports WebGL (such as Firefox)!

=== Scenemodels cleanup and update ===
A big cleanup has been done on our scenemodels database. A few thousands of duplicate objects have been removed (they were caused by mass insertion of navigation data, essentially). Nothing urgent but this makes the data in Terrasync lighter and more coherent. A script has also been launched to update the country associated with each object. While not perfect, now a lot more objects have a country!

=== Airports ===

== Aircraft of the month ==
== Airport of the month ==
== Screenshot of the month ==

== Suggested flights ==

The US state of Idaho is known for its remote airstrips in mountainous terrain. The [http://idahoaviation.com/idahoAirstripNetwork.php Idaho Airstrip Network] has details on many of them, including airstrip diagrams.

At this time of year, it's important to take into account the temperature and altitude of the airstrips, as they combine to sap the power and lift from GA aircraft. AVWeb recently [http://www.avweb.com/avwebflash/news/stinson_crash_cockpit_video_ntsb_faa_207184-1.html highlighted] the (non-fatal) crash of a [http://en.wikipedia.org/wiki/Stinson_108 Stinson 108], which included a video taken by one of the passengers. Though the NTSB has only produced a preliminary report, it was notable that the aircraft was taking off from a grass airstrip at high altitude on a hot day and appears from the video to struggle to get airborne. The Stinson is broadly similar in performance to the c172p in our base package, and the airstrip (Bruce Meadows Airport, Idaho - U63) exists in our scenery. Those interested can try taking off using the c172p with 4 people on board (2400lbs max gross weight) and high temperatures. McCall Municipal Airport (KMYL) is 44nm away, and a good destination.

== Aircraft reviews ==

== Wiki updates ==
===New articles===
<DynamicArticleList>
type=new
count=10
</DynamicArticleList>
===New aircraft articles===
<DynamicArticleList>
type=new
count=10
categoryRoot=Aircraft
</DynamicArticleList>
===Most popular newsletters===
<DynamicArticleList>
type=hot
count=5
categoryRoot=FlightGear Newsletter
</DynamicArticleList>

== Community news ==
=== FlightGear on YouTube ===
Lukea made a short promo for the release of FlightGear 2.8
{{#ev:youtube|X2Gq1l8A7iU}}
=== New tutorials and screencasts ===
{{#ev:youtube|VwYOFL9B2CE}}
{{#ev:youtube|wyfqeiHbosU}}

=== Forum news ===
=== Multiplayer ===
=== Virtual airlines ===
=== FlightGear events ===

== Useful links ==
== And finally ... ==
=== Did you know? ===
Did you know that the FlightGear project dates back to a discussion among a group of net citizens in 1996 resulting in a proposal written by David Murr? The original proposal is still available
from the FlightGear website and can be found [http://www.flightgear.org/proposal-3.0.1 on the FlightGear website]. Read it, you'll learn a lot about FlightGears' roots which still lead the project spirit.

=== Contributing ===
One of the regular thoughts expressed on the FlightGear forums is "I'd like to contribute but I don't know how to program, and I don't have the time". Unfortunately, there is a common mis-conception that contributing requires programming and lots of free time. In fact, there are a huge range of ways to contribute to the project without needing to write code or spending days working on something.

For ideas on starting to contribute to FlightGear, you may want to check out: [[Volunteer]].

=== Call for volunteers ===
* The [http://horizonflightgearhangar.yolasite.com/fg-on-android.php FOA] Team is looking for testers.
* The [[Target4Today]] project is looking for volunteers to help improving FlightGear's combat support
* The [[OpenRadar]] project is looking for a new maintainer.
* The [[FGFSPM]] (FlightGear Package Manager) is looking for a new maintainer.

=== Did you know ===

[[Category:FlightGear Newsletter|2012 08]]

FlightGear Newsletter August 2012

2012-08-16T16:18:42Z

Fredb: Avoid duplicated section about Rembrandt

{{newsletter}}
{{TOC_right|limit=2}}

''We would like to emphasize that the monthly newsletter can not live without the contributions of FlightGear users and developers. Everyone with a wiki account (free to register) can edit the newsletter and every contribution is welcome. So if you know about any FlightGear related news or projects such as for example updated scenery or aircraft, please do feel invited to add such news to the newsletter.''

== Development news ==
A view of FlightGear core and fgdata activity in [[Git]] since the 2.6.0 release. A big thank to all involved for making FG such a nice fireworks!
{{#ev:youtube|ibO6ruzQ5gM}} {{#ev:youtube|U0Dsgx9-Pbk}}

=== Project Rembrandt ===
Fred's been working on some fancy new effects, especially useful for our film-making enthusiasts.

The [[Project Rembrandt|Rembrandt renderer]] has a lot of potentiality when it comes to post processing effects. Indeed, the interesting property of a deferred shader is that all stages of rendering are kept into memory. Post-processing is as simple as using a filter on a camera.

So currently the effects added in git are :
* Night vision including amplification grain and restrained field of view,
* Cinema effect including :
** Vignetting,
** Color shift, with Sepia as default value,
** Radial distortion (barrel and pincushion distortion, with scale compensation),
** Lateral chromatic aberration (purple fringing),
** Film wear simulation.

{|
| [[File:Fgfs-rembrandt-old-timer.jpg|thumb|400px|Rembrandt old film style]]
| [[File:Fgfs-rembrandt-distortion.jpg|thumb|400px|Rembrandt ordinary lens simulation]]
| [[File:Fgfs-rembrandt-night-vision-2.png|thumb|400px|Rembrandt night vision simulation]]
|}
This won't make it for the 2.8 release but it's likely that an add-on will be available as it is contained in the data.

See also :
{|
| {{#ev:youtube|tLZzlLjvDRQ}}
| {{#ev:youtube|_Dm-Xx5tfiE}}
| {{#ev:youtube|8jBvVyiq-D0}}
|}

=== Unified Weather GUI ===
As most readers will be aware, FG supports two weather models - Basic Weather which represents METAR data directly, and Local Weather which models weather more realistically, with more variation. In 2.8.0 these were configured separately using different UIs. Stuart Buchanan and Thorsten Renk have been working on unifying the user interfaces to provide a more user-friendly and consistent interface. This work is now available in git.

=== Canvas News ===


As of 08/2012, the new [[Canvas]] 2D drawing system in FlightGear HEAD has now also support for:
* using raster images (vector images were already supported previously)
* nested canvases, where a canvas may contain images created by another canvas texture [[Howto:Using raster images and nested canvases]]
* creating GUI windows (not just widgets)
* handling GUI events using osgGA
* multiplatfom copy/paste support using two new Nasal functions: [[Howto:Clipboard access using Nasal]]

Support for nested canvases will make it possible to also load canvas textures into other canvases, so that you can, for example, easily load an instrument into a GUI dialog, or even use GUI widgets in MFD instruments, which is a feature based on real avionics, i.e. as used in modern airliners like the A320 or the 787.

Nested canvases also make it possible for people to easily implement GUI tools using the Canvas system, for example a GUI instrument or panel editor can be entirely implemented in scripting space now, without touching any C++ code. Similarly, GUI widgets could also be created in a WYSIWYG-fashion, too - so that even dialog editor or full GUI builders can be built using the Canvas system.

In addition, Tom has now implemented support for window stacking. Check out the demo video below, which demonstrates how "nested" canvases and window stacking works:

{{#ev:youtube|llVaasTEf44}}

Missing features, and features currently under development, are listed at [[Canvas Widgets#Missing_.2F_Todo|Missing Canvas Features]]

=== Mailing list digest ===

(by far the easiest option to populate the newsletter with contents is copying/pasting stuff from the forum and the mailing list or the git logs)

=== Forum digest ===

=== '''Flightgear On Android''' ===

A new developer has achieved a way to run Flightgear on Android phones. [[File:Flightgear Android Launcher.png|thumb|FOA]]

The project codename will be FOA;he has done a lot of Android and C++ coding. [[File:FOA.jpg|thumb|FOA ingame]]

His phone is a Samsung Galaxy Nexus and is running Android 4.04 (custom ROM but stock ROM should be OK). If your ready to test with us, please read the technical info then send a email to lacbrc@gmail.com!

TECHNICAL INFO:
It should have at least a 1GHz processor and decent graphics chip. Android 2.3 is a
must but a rooted device is not needed but can improve performance.
Not much in this, as we haven't finished it to a release yet.
He tested it on a bluetooth keyboard and a wired one through usb host mode. Hope you are interested in it!

The first ALPHA release is fresh out of its release, and you can read more here:
http://www.flightgear.org/forums/viewtopic.php?f=18&t=16719&p=163745#p163745

The FlightGear for Android Team

Here is a few links:

Forums:

http://www.flightgear.org/forums/viewtopic.php?f=18&t=16719

Website page:

http://horizonflightgearhangar.yolasite.com/fg-on-android.php

Wiki page:

http://wiki.flightgear.org/Flightgear_On_Android

=== Git digest ===

=== Getting involved as a programmer ===

Please see [[Howto:Start core development]]

== Interview with a contributor (NAME) ==
''In each edition we have an interview with a contributor. Suggestions for possible questions are available on [[interview questions]], you are invited to come up with new questions and interview ideas obviously! Anyone is free to write an interview (with him-/herself or others) for next month's newsletter! If you'd like to help interview a contributor or get interviewed, please do consider adding yourself to the [[list of interview volunteers]]! To keep this going and less awkward, we are currently trying to come up with the convention that former interviewees become next month's interviewers.''

* How long have you been involved in FlightGear?
* What are your major interests in FlightGear?
* What project are you working on right now?
* What do you plan on doing in the future?
* Are you happy with the way the FlightGear project is going?
* What do you enjoy most about developing for FlightGear?
* Are there any "hidden features" you have worked on in FlightGear that new users may miss?
* What advice can you give to new developers who want to get started on their first aircraft/new feature/Nasal script?

More questions are being collected here: [[Interview questions]].

Stay tuned for next month's interview, featuring FlightGear contributor XXXXXXXX

== Snapshot releases ==
Every now and then, easy-to-install development snapshots are created (usually, twice montlhy). These snapshos depict a recent state of the development version of FlightGear. By using them users can test out features that will be included in the upcoming release. Testers are encouraged to file bugs at [http://code.google.com/p/flightgear-bugs/ the issue tracker].

The snapshot can be download via the links at the bottom of this page: http://www.flightgear.org/download/. Updates and feedback can be found [http://flightgear.org/forums/viewtopic.php?f=28&t=10488&p=144233&hilit=snapshot#p144233 at the forum].

== Nasal for newbies ==

== New software tools and projects ==

== FlightGear addons and mods ==

== In the hangar ==

All the way back in May 2011, we addopted a new status-rating system for aircraft. So far, only a few have actually been rated, as can be seen in the list 'hockenberry' set up at [https://docs.google.com/spreadsheet/ccc?key=0ApzphjA4w05ndF94Y2F0bzJTbHQ5QTJXZXJRcUVRbWc&hl=en_US Google Docs]. If you're an aircraft developer and your aircraft is/are not on the list, please consider rating their status. All you'll need to know/do is described at [[Formalizing Aircraft Status]]. If you'd just like to get started contributing to FlightGear, this would also seem like an excellent way to get started.

=== New hangers ===

=== New aircraft ===

At the request by a new user of FG, Helijah has created and made available the Yakovlev Yak 18T in his hangar and GIT.
[[File:Yak18t-splash.png|thumb|Yakovlev Yak 18T]]

=== Updated aircraft ===

==== [[ATC-FS]] ====
The Air Traffic Control tool ATC-FS got an update. The old radar screen was replaced by a radar screen, which can display fixes, navaids and ILS localizers. This new radar is resizable, so you can use your big screen more efficient.

The next version is already being worked on, it will reimplement ATC-FS using the new [[Canvas]] system (scheduled for FG 3.0) so that a fully custom drawn radar screen can be created in scripting space, including custom GUI widgets (e.g. for flight strips) - without depending on any hard coded instruments or dialogs any longer.

This new design will eventually even make it possible to reuse ATC-FS for other purposes, or even include it in an AWACS aircraft that is multiplayer-enabled.

[[Image:ATC-FS.jpg|1000px]]

==== [[Mil_Mi6]] ====
The big old russian helicopter Mi-6 from the Mil design bureau has received a new update. More than a year after the last update a new structure of the Models-subfolder allows new instruments to be added much easier. Also the electrical system has been revisited and new instruments have beed added to the engineer panels. Please keep in mind that this still is a very early development version and therefore a lot of work still needs to be done. It is available from its [https://gitorious.org/mil-mi-6 gitorious-page.]

=== Liveries ===

== Scenery corner ==
=== 3D models import webform ===
The webform to import 3D models directly into the DB is in production. Now you can use this tool to send your models to the database used by Terrasync. If everything is fine and your model respects the specifications written in the foreword, then it will quickly be shipped to all FG users worldwide through [[Terrasync]]. Yet another unique feature amongst the flight simulation world held by FlightGear. Thanks to Clément, Julien, Martin, for their help&support.

With all the forms now working on the scenemodels website, remember you can:
* add a single object (if you only have one!!). Check here: http://scenemodels.flightgear.org/submission/index.php
* update an object (for instance, a lot of power pylons are in fact communication pylones). It's here: http://scenemodels.flightgear.org/submission/shared/index_update.php
* delete an object (if it does not exist anymore in real life). It's here: http://scenemodels.flightgear.org/submission/shared/index_delete.php
* run a mass object positions import (if you added a lot of objets in one place). It's here: http://scenemodels.flightgear.org/submission/shared/index_mass_import.php
* add 3D models you created to have them imported into FG scenery via Terrasync. It's here: http://scenemodels.flightgear.org/submission/static/index.php They can be specific to a city (for instance : the Eiffel tower), or shared (a new windturbine model, for instance).

Please read carefully the various forewords on the site:
* all data imported has to comply with the GNU/GPL license ;
* you must use the elevation format used by Terrasync to give your objects/models elevation, or they could be floating/sunk for most of users.

We hope to see a lot of scenes not yet available via Terrasync merge into scenemodels. Nothing is now easier, especially when used in conjunction with our Mapserver!
Remember that this is the strength of FG compared to other sims: everything can be in Terrasync and no need to download patches or sceneries here and there.

=== Scenemodels cleanup and update ===
A big cleanup has been done on our scenemodels database. A few thousands of duplicate objects have been removed (they were caused by mass insertion of navigation data, essentially). Nothing urgent but this makes the data in Terrasync lighter and more coherent. A script has also been launched to update the country associated with each object. While not perfect, now a lot more objects have a country!

=== Airports ===

== Aircraft of the month ==
== Airport of the month ==
== Screenshot of the month ==

== Suggested flights ==

The US state of Idaho is known for its remote airstrips in mountainous terrain. The [http://idahoaviation.com/idahoAirstripNetwork.php Idaho Airstrip Network] has details on many of them, including airstrip diagrams.

At this time of year, it's important to take into account the temperature and altitude of the airstrips, as they combine to sap the power and lift from GA aircraft. AVWeb recently [http://www.avweb.com/avwebflash/news/stinson_crash_cockpit_video_ntsb_faa_207184-1.html highlighted] the (non-fatal) crash of a [http://en.wikipedia.org/wiki/Stinson_108 Stinson 108], which included a video taken by one of the passengers. Though the NTSB has only produced a preliminary report, it was notable that the aircraft was taking off from a grass airstrip at high altitude on a hot day and appears from the video to struggle to get airborne. The Stinson is broadly similar in performance to the c172p in our base package, and the airstrip (Bruce Meadows Airport, Idaho - U63) exists in our scenery. Those interested can try taking off using the c172p with 4 people on board (2400lbs max gross weight) and high temperatures. McCall Municipal Airport (KMYL) is 44nm away, and a good destination.

== Aircraft reviews ==

== Wiki updates ==
===New articles===
<DynamicArticleList>
type=new
count=10
</DynamicArticleList>
===New aircraft articles===
<DynamicArticleList>
type=new
count=10
categoryRoot=Aircraft
</DynamicArticleList>
===Most popular newsletters===
<DynamicArticleList>
type=hot
count=5
categoryRoot=FlightGear Newsletter
</DynamicArticleList>

== Community news ==
=== FlightGear on YouTube ===

=== New tutorials and screencasts ===
=== Forum news ===
=== Multiplayer ===
=== Virtual airlines ===
=== FlightGear events ===

== Useful links ==
== And finally ... ==
=== Did you know? ===
Did you know that the FlightGear project dates back to a discussion among a group of net citizens in 1996 resulting in a proposal written by David Murr? The original proposal is still available
from the FlightGear website and can be found [http://www.flightgear.org/proposal-3.0.1 on the FlightGear website]. Read it, you'll learn a lot about FlightGears' roots which still lead the project spirit.

=== Contributing ===
One of the regular thoughts expressed on the FlightGear forums is "I'd like to contribute but I don't know how to program, and I don't have the time". Unfortunately, there is a common mis-conception that contributing requires programming and lots of free time. In fact, there are a huge range of ways to contribute to the project without needing to write code or spending days working on something.

For ideas on starting to contribute to FlightGear, you may want to check out: [[Volunteer]].

=== Call for volunteers ===
* The [http://horizonflightgearhangar.yolasite.com/fg-on-android.php FOA] Team is looking for testers.
* The [[Target4Today]] project is looking for volunteers to help improving FlightGear's combat support
* The [[OpenRadar]] project is looking for a new maintainer.
* The [[FGFSPM]] (FlightGear Package Manager) is looking for a new maintainer.

=== Did you know ===

[[Category:FlightGear Newsletter|2012 08]]

FlightGear Newsletter August 2012

2012-08-15T17:05:15Z

Fredb: /* Project Rembrandt */

{{newsletter}}
{{TOC_right|limit=2}}

''We would like to emphasize that the monthly newsletter can not live without the contributions of FlightGear users and developers. Everyone with a wiki account (free to register) can edit the newsletter and every contribution is welcome. So if you know about any FlightGear related news or projects such as for example updated scenery or aircraft, please do feel invited to add such news to the newsletter.''

== Development news ==

=== Project Rembrandt ===
Fred's been working on some fancy new effects, especially useful for our film-making enthusiasts:

{{#ev:youtube|tLZzlLjvDRQ}}

{{#ev:youtube|_Dm-Xx5tfiE}}

{{#ev:youtube|8jBvVyiq-D0}}

=== Unified Weather GUI ===
As most readers will be aware, FG supports two weather models - Basic Weather which represents METAR data directly, and Local Weather which models weather more realistically, with more variation. In 2.8.0 these were configured separately using different UIs. Stuart Buchanan and Thorsten Renk have been working on unifying the user interfaces to provide a more user-friendly and consistent interface. This work is now available in git.

=== Canvas News ===


As of 08/2012, the new [[Canvas]] 2D drawing system in FlightGear HEAD has now also support for:
* using raster images (vector images were already supported previously)
* nested canvases, where a canvas may contain images created by another canvas texture
* creating GUI windows (not just widgets)
* handling GUI events using osgGA
* multiplatfom copy/paste support using two new Nasal functions: [[Howto:Clipboard access using Nasal]]

Support for nested canvases will make it possible to also load canvas textures into other canvases, so that you can, for example, easily load an instrument into a GUI dialog, or even use GUI widgets in MFD instruments, which is a feature based on real avionics, i.e. as used in modern airliners like the A320 or the 787.

Nested canvases also make it possible for people to easily implement GUI tools using the Canvas system, for example a GUI instrument or panel editor can be entirely implemented in scripting space now, without touching any C++ code. Similarly, GUI widgets could also be created in a WYSIWYG-fashion, too - so that even dialog editor or full GUI builders can be built using the Canvas system.

In addition, Tom has now implemented support for window stacking. Check out the demo video below, which demonstrates how "nested" canvases and window stacking works:

{{#ev:youtube|llVaasTEf44}}

Missing features, and features currently under development, are listed at [[Canvas Widgets#Missing_.2F_Todo|Missing Canvas Features]]

=== Mailing list digest ===

(by far the easiest option to populate the newsletter with contents is copying/pasting stuff from the forum and the mailing list or the git logs)

=== Forum digest ===

=== '''Flightgear On Android''' ===

A new developer has achieved a way to run Flightgear on Android phones. [[File:Flightgear Android Launcher.png|thumb|FOA]]

The project codename will be FOA;he has done a lot of Android and C++ coding. [[File:FOA.jpg|thumb|FOA ingame]]

His phone is a Samsung Galaxy Nexus and is running Android 4.04 (custom ROM but stock ROM should be OK). If your ready to test with us, please read the technical info then send a email to lacbrc@gmail.com!

TECHNICAL INFO:
It should have at least a 1GHz processor and decent graphics chip. Android 2.3 is a
must but a rooted device is not needed but can improve performance.
Not much in this, as we haven't finished it to a release yet.
He tested it on a bluetooth keyboard and a wired one through usb host mode. Hope you are interested in it!

The first ALPHA release is fresh out of its release, and you can read more here:
http://www.flightgear.org/forums/viewtopic.php?f=18&t=16719&p=163745#p163745

The FlightGear for Android Team

Here is a few links:

Forums:

http://www.flightgear.org/forums/viewtopic.php?f=18&t=16719

Website page:

http://horizonflightgearhangar.yolasite.com/fg-on-android.php

Wiki page:

http://wiki.flightgear.org/Flightgear_On_Android

=== New effects in Rembrandt ===
The [[Project Rembrandt|Rembrandt renderer]] has a lot of potentiality when it comes to post processing effects. Indeed, the interesting property of a deferred shader is that all stages of rendering are kept into memory. Post-processing is as simple as using a filter on a camera.

So currently the effects added in git are :
* Night vision including amplification grain and restrained field of view,
* Cinema effect including :
** Vignetting,
** Color shift, with Sepia as default value,
** Radial distortion (barrel and pincushion distortion, with scale compensation),
** Lateral chromatic aberration (purple fringing),
** Film wear simulation.

{|
| [[File:Fgfs-rembrandt-old-timer.jpg|thumb|400px|Rembrandt old film style]]
| [[File:Fgfs-rembrandt-distortion.jpg|thumb|400px|Rembrandt ordinary lens simulation]]
| [[File:Fgfs-rembrandt-night-vision-2.png|thumb|400px|Rembrandt night vision simulation]]
|}
This won't make it for the 2.8 release but it's likely that an add-on will be available as it is contained in the data.

=== Git digest ===

=== Getting involved as a programmer ===

Please see [[Howto:Start core development]]

== Interview with a contributor (NAME) ==
''In each edition we have an interview with a contributor. Suggestions for possible questions are available on [[interview questions]], you are invited to come up with new questions and interview ideas obviously! Anyone is free to write an interview (with him-/herself or others) for next month's newsletter! If you'd like to help interview a contributor or get interviewed, please do consider adding yourself to the [[list of interview volunteers]]! To keep this going and less awkward, we are currently trying to come up with the convention that former interviewees become next month's interviewers.''

* How long have you been involved in FlightGear?
* What are your major interests in FlightGear?
* What project are you working on right now?
* What do you plan on doing in the future?
* Are you happy with the way the FlightGear project is going?
* What do you enjoy most about developing for FlightGear?
* Are there any "hidden features" you have worked on in FlightGear that new users may miss?
* What advice can you give to new developers who want to get started on their first aircraft/new feature/Nasal script?

More questions are being collected here: [[Interview questions]].

Stay tuned for next month's interview, featuring FlightGear contributor XXXXXXXX

== Snapshot releases ==
Every now and then, easy-to-install development snapshots are created (usually, twice montlhy). These snapshos depict a recent state of the development version of FlightGear. By using them users can test out features that will be included in the upcoming release. Testers are encouraged to file bugs at [http://code.google.com/p/flightgear-bugs/ the issue tracker].

The snapshot can be download via the links at the bottom of this page: http://www.flightgear.org/download/. Updates and feedback can be found [http://flightgear.org/forums/viewtopic.php?f=28&t=10488&p=144233&hilit=snapshot#p144233 at the forum].

== Nasal for newbies ==

== New software tools and projects ==

== FlightGear addons and mods ==

== In the hangar ==

All the way back in May 2011, we addopted a new status-rating system for aircraft. So far, only a few have actually been rated, as can be seen in the list 'hockenberry' set up at [https://docs.google.com/spreadsheet/ccc?key=0ApzphjA4w05ndF94Y2F0bzJTbHQ5QTJXZXJRcUVRbWc&hl=en_US Google Docs]. If you're an aircraft developer and your aircraft is/are not on the list, please consider rating their status. All you'll need to know/do is described at [[Formalizing Aircraft Status]]. If you'd just like to get started contributing to FlightGear, this would also seem like an excellent way to get started.

=== New hangers ===

=== New aircraft ===

At the request by a new user of FG, Helijah has created and made available the Yakovlev Yak 18T in his hangar and GIT.
[[File:Yak18t-splash.png|thumb|Yakovlev Yak 18T]]

=== Updated aircraft ===

==== [[ATC-FS]] ====
The Air Traffic Control tool ATC-FS got an update. The old radar screen was replaced by a radar screen, which can display fixes, navaids and ILS localizers. This new radar is resizable, so you can use your big screen more efficient.

The next version is already being worked on, it will reimplement ATC-FS using the new [[Canvas]] system (scheduled for FG 3.0) so that a fully custom drawn radar screen can be created in scripting space, including custom GUI widgets (e.g. for flight strips) - without depending on any hard coded instruments or dialogs any longer.

This new design will eventually even make it possible to reuse ATC-FS for other purposes, or even include it in an AWACS aircraft that is multiplayer-enabled.

[[Image:ATC-FS.jpg|1000px]]

==== [[Mil_Mi6]] ====
The big old russian helicopter Mi-6 from the Mil design bureau has received a new update. More than a year after the last update a new structure of the Models-subfolder allows new instruments to be added much easier. Also the electrical system has been revisited and new instruments have beed added to the engineer panels. Please keep in mind that this still is a very early development version and therefore a lot of work still needs to be done. It is available from its [https://gitorious.org/mil-mi-6 gitorious-page.]

=== Liveries ===

== Scenery corner ==
=== 3D models import webform ===
The webform to import 3D models directly into the DB is in production. Now you can use this tool to send your models to the database used by Terrasync. If everything is fine and your model respects the specifications written in the foreword, then it will quickly be shipped to all FG users worldwide through [[Terrasync]]. Yet another unique feature amongst the flight simulation world held by FlightGear. Thanks to Clément, Julien, Martin, for their help&support.

With all the forms now working on the scenemodels website, remember you can:
* add a single object (if you only have one!!). Check here: http://scenemodels.flightgear.org/submission/index.php
* update an object (for instance, a lot of power pylons are in fact communication pylones). It's here: http://scenemodels.flightgear.org/submission/shared/index_update.php
* delete an object (if it does not exist anymore in real life). It's here: http://scenemodels.flightgear.org/submission/shared/index_delete.php
* run a mass object positions import (if you added a lot of objets in one place). It's here: http://scenemodels.flightgear.org/submission/shared/index_mass_import.php
* add 3D models you created to have them imported into FG scenery via Terrasync. It's here: http://scenemodels.flightgear.org/submission/static/index.php They can be specific to a city (for instance : the Eiffel tower), or shared (a new windturbine model, for instance).

Please read carefully the various forewords on the site:
* all data imported has to comply with the GNU/GPL license ;
* you must use the elevation format used by Terrasync to give your objects/models elevation, or they could be floating/sunk for most of users.

We hope to see a lot of scenes not yet available via Terrasync merge into scenemodels. Nothing is now easier, especially when used in conjunction with our Mapserver!
Remember that this is the strength of FG compared to other sims: everything can be in Terrasync and no need to download patches or sceneries here and there.

=== Scenemodels cleanup and update ===
A big cleanup has been done on our scenemodels database. A few thousands of duplicate objects have been removed (they were caused by mass insertion of navigation data, essentially). Nothing urgent but this makes the data in Terrasync lighter and more coherent. A script has also been launched to update the country associated with each object. While not perfect, now a lot more objects have a country!

=== Airports ===

== Aircraft of the month ==
== Airport of the month ==
== Screenshot of the month ==

== Suggested flights ==

The US state of Idaho is known for its remote airstrips in mountainous terrain. The [http://idahoaviation.com/idahoAirstripNetwork.php Idaho Airstrip Network] has details on many of them, including airstrip diagrams.

At this time of year, it's important to take into account the temperature and altitude of the airstrips, as they combine to sap the power and lift from GA aircraft. AVWeb recently [http://www.avweb.com/avwebflash/news/stinson_crash_cockpit_video_ntsb_faa_207184-1.html highlighted] the (non-fatal) crash of a [http://en.wikipedia.org/wiki/Stinson_108 Stinson 108], which included a video taken by one of the passengers. Though the NTSB has only produced a preliminary report, it was notable that the aircraft was taking off from a grass airstrip at high altitude on a hot day and appears from the video to struggle to get airborne. The Stinson is broadly similar in performance to the c172p in our base package, and the airstrip (Bruce Meadows Airport, Idaho - U63) exists in our scenery. Those interested can try taking off using the c172p with 4 people on board (2400lbs max gross weight) and high temperatures. McCall Municipal Airport (KMYL) is 44nm away, and a good destination.

== Aircraft reviews ==

== Wiki updates ==
===New articles===
<DynamicArticleList>
type=new
count=10
</DynamicArticleList>
===New aircraft articles===
<DynamicArticleList>
type=new
count=10
categoryRoot=Aircraft
</DynamicArticleList>
===Most popular newsletters===
<DynamicArticleList>
type=hot
count=5
categoryRoot=FlightGear Newsletter
</DynamicArticleList>

== Community news ==
=== FlightGear on YouTube ===

=== New tutorials and screencasts ===
=== Forum news ===
=== Multiplayer ===
=== Virtual airlines ===
=== FlightGear events ===

== Useful links ==
== And finally ... ==
=== Did you know? ===
Did you know that the FlightGear project dates back to a discussion among a group of net citizens in 1996 resulting in a proposal written by David Murr? The original proposal is still available
from the FlightGear website and can be found [http://www.flightgear.org/proposal-3.0.1 on the FlightGear website]. Read it, you'll learn a lot about FlightGears' roots which still lead the project spirit.

=== Contributing ===
One of the regular thoughts expressed on the FlightGear forums is "I'd like to contribute but I don't know how to program, and I don't have the time". Unfortunately, there is a common mis-conception that contributing requires programming and lots of free time. In fact, there are a huge range of ways to contribute to the project without needing to write code or spending days working on something.

For ideas on starting to contribute to FlightGear, you may want to check out: [[Volunteer]].

=== Call for volunteers ===
* The [http://horizonflightgearhangar.yolasite.com/fg-on-android.php FOA] Team is looking for testers.
* The [[OpenRadar]] project is looking for a new maintainer.
* The [[FGFSPM]] (FlightGear Package Manager) is looking for a new maintainer.

=== Did you know ===

[[Category:FlightGear Newsletter|2012 08]]

Project Rembrandt

2012-08-15T12:35:02Z

Fredb: /* Appendix */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

'''Beware: <Effect> only works on real objects, not on groups of objects or animations.'''

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Making sure that all geometry will cast shadow ===

To limit the amount of geometry rendered in the shadow map, and also to reduce artifacts (shadow acne), only faces not facing the sun are casting shadows. The test is made using the normal orientation. That means that double-sided polygons, or mesh that are not closed, will be transparent to light at certain sun angles. To avoid that, modelers can either :
* ensure that the object is always in the shadow of another objects,
* close their mesh, or,
* double polygons with the normal set to the opposite.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Performance and compatibility considerations ====
Every light on screen is equivalent for the GPU of redrawing the light volume with a shader, without z buffer culling. So each light comes with a cost, that is small taken individually but noticeable when a lot of them are visible. That cost also increase with the size of the light volume.

Beside that, it is wise to allow a model to work with the classical renderer that know nothing about lights and would render light volumes like other geometry. So a good practice is to complement each light animation with a select animation checking :
*if Rembrandt is enabled
*if the user selected quality match with the purpose of the light
*if the light should be on or off according to the other parameters of the simulation (position of the sun, position of switch, presence of power, ...)

A quality property is created to reflect the user preferences about quality vs performance concerning lights, and a proper slider is added to the shader dialog.
[[File:Fgfs-shaders-with-light.jpg|thumb|Light quality slider]]
The propery to use is :
/sim/rendering/shaders/lights
The quality slider sets its range from 0 (no lights) to 4 (all lights on). Simple airport lamp post appears at 1. Few bridge lamps at 2, all simple bridge lamp at 3 and more involved one at 4.

Example:
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSource</name>
<object-name>LightVolume</object-name>
...
</animation>

<animation>
<type>select</type>

<object-name>LightSource</object-name>
<condition>
<and>

<property>/sim/rendering/rembrandt/enabled</property>

<greater-than>
<property>/sim/rendering/shaders/lights</property>
<value>3.0</value>
</greater-than>

<greater-than>
<property>/sim/time/sun-angle-rad</property>
<value>1.57</value>
</greater-than>
</and>
</condition>
</animation>

</syntaxhighlight>

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)
*Fix fog on clouds

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix shadow matrices in multi-screen
*Fix Lights in multi screen (seems same problem than shadows. Forgot a 1/w factor ?)
*Implement lightfield shader
*Add some kind of fog to lights
*Restore depth partitioning using depth ranges
*Move effect of cloud coverage from water shader to sunlight shader
*Allow light masks using textures
*Use stencil double-sided operations to limit the depth of light volumes. Use depth clamp to ensure front faces are always rendered even if the camera is enclosed in the light volume.

=== Long term ideas (unsorted) ===
*Use a separate list of light volumes to avoid traversing the scenegraph again (with transparency problems). Find out how we can detect unloading of models.
*implement volumetric effects by extending lights to arbitrary shaders. Should enable to implement heat haze effect and real wake waves.
*implement strength of glow (in the emissive buffer alpha channel) - use a Poisson-disk distribution to implement variable size blur
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
* {{cite web |url=http://publications.dice.se/publications.asp?show_category=yes&which_category=Rendering |author=DICE |title=Publications on rendering |accessdate=15 August 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

Project Rembrandt

2012-08-15T12:30:16Z

Fredb: /* Making sure that all geometry will cast shadow */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

'''Beware: <Effect> only works on real objects, not on groups of objects or animations.'''

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Making sure that all geometry will cast shadow ===

To limit the amount of geometry rendered in the shadow map, and also to reduce artifacts (shadow acne), only faces not facing the sun are casting shadows. The test is made using the normal orientation. That means that double-sided polygons, or mesh that are not closed, will be transparent to light at certain sun angles. To avoid that, modelers can either :
* ensure that the object is always in the shadow of another objects,
* close their mesh, or,
* double polygons with the normal set to the opposite.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Performance and compatibility considerations ====
Every light on screen is equivalent for the GPU of redrawing the light volume with a shader, without z buffer culling. So each light comes with a cost, that is small taken individually but noticeable when a lot of them are visible. That cost also increase with the size of the light volume.

Beside that, it is wise to allow a model to work with the classical renderer that know nothing about lights and would render light volumes like other geometry. So a good practice is to complement each light animation with a select animation checking :
*if Rembrandt is enabled
*if the user selected quality match with the purpose of the light
*if the light should be on or off according to the other parameters of the simulation (position of the sun, position of switch, presence of power, ...)

A quality property is created to reflect the user preferences about quality vs performance concerning lights, and a proper slider is added to the shader dialog.
[[File:Fgfs-shaders-with-light.jpg|thumb|Light quality slider]]
The propery to use is :
/sim/rendering/shaders/lights
The quality slider sets its range from 0 (no lights) to 4 (all lights on). Simple airport lamp post appears at 1. Few bridge lamps at 2, all simple bridge lamp at 3 and more involved one at 4.

Example:
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSource</name>
<object-name>LightVolume</object-name>
...
</animation>

<animation>
<type>select</type>

<object-name>LightSource</object-name>
<condition>
<and>

<property>/sim/rendering/rembrandt/enabled</property>

<greater-than>
<property>/sim/rendering/shaders/lights</property>
<value>3.0</value>
</greater-than>

<greater-than>
<property>/sim/time/sun-angle-rad</property>
<value>1.57</value>
</greater-than>
</and>
</condition>
</animation>

</syntaxhighlight>

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)
*Fix fog on clouds

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix shadow matrices in multi-screen
*Fix Lights in multi screen (seems same problem than shadows. Forgot a 1/w factor ?)
*Implement lightfield shader
*Add some kind of fog to lights
*Restore depth partitioning using depth ranges
*Move effect of cloud coverage from water shader to sunlight shader
*Allow light masks using textures
*Use stencil double-sided operations to limit the depth of light volumes. Use depth clamp to ensure front faces are always rendered even if the camera is enclosed in the light volume.

=== Long term ideas (unsorted) ===
*Use a separate list of light volumes to avoid traversing the scenegraph again (with transparency problems). Find out how we can detect unloading of models.
*implement volumetric effects by extending lights to arbitrary shaders. Should enable to implement heat haze effect and real wake waves.
*implement strength of glow (in the emissive buffer alpha channel) - use a Poisson-disk distribution to implement variable size blur
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

Project Rembrandt

2012-08-15T12:23:48Z

Fredb: /* Registering all translucent surfaces */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

'''Beware: <Effect> only works on real objects, not on groups of objects or animations.'''

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Making sure that all geometry will cast shadow ===

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Performance and compatibility considerations ====
Every light on screen is equivalent for the GPU of redrawing the light volume with a shader, without z buffer culling. So each light comes with a cost, that is small taken individually but noticeable when a lot of them are visible. That cost also increase with the size of the light volume.

Beside that, it is wise to allow a model to work with the classical renderer that know nothing about lights and would render light volumes like other geometry. So a good practice is to complement each light animation with a select animation checking :
*if Rembrandt is enabled
*if the user selected quality match with the purpose of the light
*if the light should be on or off according to the other parameters of the simulation (position of the sun, position of switch, presence of power, ...)

A quality property is created to reflect the user preferences about quality vs performance concerning lights, and a proper slider is added to the shader dialog.
[[File:Fgfs-shaders-with-light.jpg|thumb|Light quality slider]]
The propery to use is :
/sim/rendering/shaders/lights
The quality slider sets its range from 0 (no lights) to 4 (all lights on). Simple airport lamp post appears at 1. Few bridge lamps at 2, all simple bridge lamp at 3 and more involved one at 4.

Example:
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSource</name>
<object-name>LightVolume</object-name>
...
</animation>

<animation>
<type>select</type>

<object-name>LightSource</object-name>
<condition>
<and>

<property>/sim/rendering/rembrandt/enabled</property>

<greater-than>
<property>/sim/rendering/shaders/lights</property>
<value>3.0</value>
</greater-than>

<greater-than>
<property>/sim/time/sun-angle-rad</property>
<value>1.57</value>
</greater-than>
</and>
</condition>
</animation>

</syntaxhighlight>

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)
*Fix fog on clouds

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix shadow matrices in multi-screen
*Fix Lights in multi screen (seems same problem than shadows. Forgot a 1/w factor ?)
*Implement lightfield shader
*Add some kind of fog to lights
*Restore depth partitioning using depth ranges
*Move effect of cloud coverage from water shader to sunlight shader
*Allow light masks using textures
*Use stencil double-sided operations to limit the depth of light volumes. Use depth clamp to ensure front faces are always rendered even if the camera is enclosed in the light volume.

=== Long term ideas (unsorted) ===
*Use a separate list of light volumes to avoid traversing the scenegraph again (with transparency problems). Find out how we can detect unloading of models.
*implement volumetric effects by extending lights to arbitrary shaders. Should enable to implement heat haze effect and real wake waves.
*implement strength of glow (in the emissive buffer alpha channel) - use a Poisson-disk distribution to implement variable size blur
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

Project Rembrandt

2012-08-14T15:56:33Z

Fredb: /* Near term tasks */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

'''Beware: <Effect> only works on real objects, not on groups of objects or animations.'''

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Performance and compatibility considerations ====
Every light on screen is equivalent for the GPU of redrawing the light volume with a shader, without z buffer culling. So each light comes with a cost, that is small taken individually but noticeable when a lot of them are visible. That cost also increase with the size of the light volume.

Beside that, it is wise to allow a model to work with the classical renderer that know nothing about lights and would render light volumes like other geometry. So a good practice is to complement each light animation with a select animation checking :
*if Rembrandt is enabled
*if the user selected quality match with the purpose of the light
*if the light should be on or off according to the other parameters of the simulation (position of the sun, position of switch, presence of power, ...)

A quality property is created to reflect the user preferences about quality vs performance concerning lights, and a proper slider is added to the shader dialog.
[[File:Fgfs-shaders-with-light.jpg|thumb|Light quality slider]]
The propery to use is :
/sim/rendering/shaders/lights
The quality slider sets its range from 0 (no lights) to 4 (all lights on). Simple airport lamp post appears at 1. Few bridge lamps at 2, all simple bridge lamp at 3 and more involved one at 4.

Example:
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSource</name>
<object-name>LightVolume</object-name>
...
</animation>

<animation>
<type>select</type>

<object-name>LightSource</object-name>
<condition>
<and>

<property>/sim/rendering/rembrandt/enabled</property>

<greater-than>
<property>/sim/rendering/shaders/lights</property>
<value>3.0</value>
</greater-than>

<greater-than>
<property>/sim/time/sun-angle-rad</property>
<value>1.57</value>
</greater-than>
</and>
</condition>
</animation>

</syntaxhighlight>

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)
*Fix fog on clouds

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix shadow matrices in multi-screen
*Fix Lights in multi screen (seems same problem than shadows. Forgot a 1/w factor ?)
*Implement lightfield shader
*Add some kind of fog to lights
*Restore depth partitioning using depth ranges
*Move effect of cloud coverage from water shader to sunlight shader
*Allow light masks using textures
*Use stencil double-sided operations to limit the depth of light volumes. Use depth clamp to ensure front faces are always rendered even if the camera is enclosed in the light volume.

=== Long term ideas (unsorted) ===
*Use a separate list of light volumes to avoid traversing the scenegraph again (with transparency problems). Find out how we can detect unloading of models.
*implement volumetric effects by extending lights to arbitrary shaders. Should enable to implement heat haze effect and real wake waves.
*implement strength of glow (in the emissive buffer alpha channel) - use a Poisson-disk distribution to implement variable size blur
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

Project Rembrandt

2012-08-14T09:48:16Z

Fredb: /* Long term ideas (unsorted) */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

'''Beware: <Effect> only works on real objects, not on groups of objects or animations.'''

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Performance and compatibility considerations ====
Every light on screen is equivalent for the GPU of redrawing the light volume with a shader, without z buffer culling. So each light comes with a cost, that is small taken individually but noticeable when a lot of them are visible. That cost also increase with the size of the light volume.

Beside that, it is wise to allow a model to work with the classical renderer that know nothing about lights and would render light volumes like other geometry. So a good practice is to complement each light animation with a select animation checking :
*if Rembrandt is enabled
*if the user selected quality match with the purpose of the light
*if the light should be on or off according to the other parameters of the simulation (position of the sun, position of switch, presence of power, ...)

A quality property is created to reflect the user preferences about quality vs performance concerning lights, and a proper slider is added to the shader dialog.
[[File:Fgfs-shaders-with-light.jpg|thumb|Light quality slider]]
The propery to use is :
/sim/rendering/shaders/lights
The quality slider sets its range from 0 (no lights) to 4 (all lights on). Simple airport lamp post appears at 1. Few bridge lamps at 2, all simple bridge lamp at 3 and more involved one at 4.

Example:
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSource</name>
<object-name>LightVolume</object-name>
...
</animation>

<animation>
<type>select</type>

<object-name>LightSource</object-name>
<condition>
<and>

<property>/sim/rendering/rembrandt/enabled</property>

<greater-than>
<property>/sim/rendering/shaders/lights</property>
<value>3.0</value>
</greater-than>

<greater-than>
<property>/sim/time/sun-angle-rad</property>
<value>1.57</value>
</greater-than>
</and>
</condition>
</animation>

</syntaxhighlight>

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)
*Fix fog on clouds

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix shadow matrices in multi-screen
*Fix Lights in multi screen (seems same problem than shadows. Forgot a 1/w factor ?)
*Implement lightfield shader
*Add some kind of fog to lights
*Restore depth partitioning using depth ranges
*Move effect of cloud coverage from water shader to sunlight shader
*Allow light masks using textures
*Render light volumes with back face culling and depth clamp instead of front face culling, and don't light pixels in front of the light volume.

=== Long term ideas (unsorted) ===
*Use a separate list of light volumes to avoid traversing the scenegraph again (with transparency problems). Find out how we can detect unloading of models.
*implement volumetric effects by extending lights to arbitrary shaders. Should enable to implement heat haze effect and real wake waves.
*implement strength of glow (in the emissive buffer alpha channel) - use a Poisson-disk distribution to implement variable size blur
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

Changelog 2.8.0

2012-08-14T09:43:20Z

Fredb: /* Some of the major changes include: */

{{Draft|changelog|}}

==FlightGear 2.8.0 ChangeLog==

The FlightGear development team is happy to announce the v2.8.0 release of FlightGear, the free, open-source flight simulator. This new version contains many exciting new features, enhancements and bugfixes.

Founded in 1997, FlightGear is developed by a worldwide group of volunteers, brought together by a shared ambition to create
the most realistic flight simulator possible that is free to use, modify and distribute. FlightGear is used all over the world
by desktop flight simulator enthusiasts, for research in universities and for interactive exhibits in museums.

FlightGear features more than 400 aircraft, a worldwide scenery database, a multi-player environment, detailed sky modelling, a flexible and open aircraft modelling system, varied networking options, multiple display support, a powerful scripting language and an open architecture. Best of all, being open-source, the simulator is owned by the community and everyone is encouraged to contribute.

Download FlightGear v2.8.0 for free from [http://www.flightgear.org FlightGear.org]

FlightGear - Fly Free!

=== Some of the major changes include: ===

'''Aircraft operations'''

'''AI system'''

'''AI Traffic'''
* Aircraft models and textures have been improved.

'''Flight dynamics'''
* FlightGear has been synced with the JSBSim project.

'''Environment'''
* Region-specific terrain textures are used for Europe and Hawaii. Now towns in Europe look different from towns in the USA.
* Cities and towns now look more populated due to random 3D buildings, completed with lighting at night.
* Scenery looks more realistic due to improved placement of random objects, buildings and trees.
* Airport signs with full apt.dat 850 syntax support. Models are rendered in 3D, with support for double-sided signs.
* You can now select between summer and winter scenery in-sim.

'''Instruments & HUDs'''
* There's a new fully scriptable 2D rendering system supported for creating instruments, the so called [[Canvas]] which allows you to create fully dynamic cockpit instruments and HUDs, even by loading and drawing vector images. This can be used to create CDUs/MCDUs for FMC, complex glass cockpit avionics (MFDs, EICAS etc) and any other "glass" instruments that could previously only be created in C++ space. All of this has now become possible just by using [[Nasal]] scripting, without having to rebuild FlightGear.

'''Interface'''
* Support for menu translations.
* Navigation and route-manager data is now available through a Nasal s
'''Highlighted new and improved aircraft'''
* [[Airwave Xtreme 150]]: complete new (JSBSim) flight dynamics, new model, advanced pilot animations.
* [[Cessna 337G Skymaster]]

'''Project infrastructure'''
* Various improvements to our scenery database have made it easier than ever to add, delete or update objects and 3D models into the FlightGear world.

'''Visual effects'''
* A new, experimental renderer supporting shadows via deferred rendering, named after the famous painter Rembrandt, is included for testing purposes.
* An improved simulation of atmospheric light scattering with terrain haze for the classical renderer.
* Easier selection of material files.

'''Other'''
* Additional joysticks and rudder pedals are supported out-of-the-box:
** InterLink Elite
** Qware USB
** Saitek Cyborg X (F.L.Y. 5)
** Saitek Pro Flight Trim Wheel
** Saitek Pro Flight Yoke
** Speedlink Defender
* A French partial translation of The FlightGear Manual is now available.

'''Bug fixes'''
* See [http://code.google.com/p/flightgear-bugs/issues/list?can=1&q=Milestone%3D2.8.0 our bugtracker] for an extensive list of the bugs fixed in this release.

[[Category:FlightGear changelogs]]

Project Rembrandt

2012-08-14T09:37:56Z

Fredb: /* Near term tasks */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

'''Beware: <Effect> only works on real objects, not on groups of objects or animations.'''

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Performance and compatibility considerations ====
Every light on screen is equivalent for the GPU of redrawing the light volume with a shader, without z buffer culling. So each light comes with a cost, that is small taken individually but noticeable when a lot of them are visible. That cost also increase with the size of the light volume.

Beside that, it is wise to allow a model to work with the classical renderer that know nothing about lights and would render light volumes like other geometry. So a good practice is to complement each light animation with a select animation checking :
*if Rembrandt is enabled
*if the user selected quality match with the purpose of the light
*if the light should be on or off according to the other parameters of the simulation (position of the sun, position of switch, presence of power, ...)

A quality property is created to reflect the user preferences about quality vs performance concerning lights, and a proper slider is added to the shader dialog.
[[File:Fgfs-shaders-with-light.jpg|thumb|Light quality slider]]
The propery to use is :
/sim/rendering/shaders/lights
The quality slider sets its range from 0 (no lights) to 4 (all lights on). Simple airport lamp post appears at 1. Few bridge lamps at 2, all simple bridge lamp at 3 and more involved one at 4.

Example:
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSource</name>
<object-name>LightVolume</object-name>
...
</animation>

<animation>
<type>select</type>

<object-name>LightSource</object-name>
<condition>
<and>

<property>/sim/rendering/rembrandt/enabled</property>

<greater-than>
<property>/sim/rendering/shaders/lights</property>
<value>3.0</value>
</greater-than>

<greater-than>
<property>/sim/time/sun-angle-rad</property>
<value>1.57</value>
</greater-than>
</and>
</condition>
</animation>

</syntaxhighlight>

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)
*Fix fog on clouds

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix shadow matrices in multi-screen
*Fix Lights in multi screen (seems same problem than shadows. Forgot a 1/w factor ?)
*Implement lightfield shader
*Add some kind of fog to lights
*Restore depth partitioning using depth ranges
*Move effect of cloud coverage from water shader to sunlight shader
*Allow light masks using textures
*Render light volumes with back face culling and depth clamp instead of front face culling, and don't light pixels in front of the light volume.

=== Long term ideas (unsorted) ===
*implement volumetric effects by extending lights to arbitrary shaders. Should enable to implement heat haze effect and real wake waves.
*implement strength of glow (in the emissive buffer alpha channel) - use a Poisson-disk distribution to implement variable size blur
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

Project Rembrandt

2012-08-14T09:23:11Z

Fredb: /* Near term tasks */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

'''Beware: <Effect> only works on real objects, not on groups of objects or animations.'''

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Performance and compatibility considerations ====
Every light on screen is equivalent for the GPU of redrawing the light volume with a shader, without z buffer culling. So each light comes with a cost, that is small taken individually but noticeable when a lot of them are visible. That cost also increase with the size of the light volume.

Beside that, it is wise to allow a model to work with the classical renderer that know nothing about lights and would render light volumes like other geometry. So a good practice is to complement each light animation with a select animation checking :
*if Rembrandt is enabled
*if the user selected quality match with the purpose of the light
*if the light should be on or off according to the other parameters of the simulation (position of the sun, position of switch, presence of power, ...)

A quality property is created to reflect the user preferences about quality vs performance concerning lights, and a proper slider is added to the shader dialog.
[[File:Fgfs-shaders-with-light.jpg|thumb|Light quality slider]]
The propery to use is :
/sim/rendering/shaders/lights
The quality slider sets its range from 0 (no lights) to 4 (all lights on). Simple airport lamp post appears at 1. Few bridge lamps at 2, all simple bridge lamp at 3 and more involved one at 4.

Example:
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSource</name>
<object-name>LightVolume</object-name>
...
</animation>

<animation>
<type>select</type>

<object-name>LightSource</object-name>
<condition>
<and>

<property>/sim/rendering/rembrandt/enabled</property>

<greater-than>
<property>/sim/rendering/shaders/lights</property>
<value>3.0</value>
</greater-than>

<greater-than>
<property>/sim/time/sun-angle-rad</property>
<value>1.57</value>
</greater-than>
</and>
</condition>
</animation>

</syntaxhighlight>

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)
*Fix fog on clouds

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix shadow matrices in multi-screen
*Fix Lights in multi screen (seems same problem than shadows. Forgot a 1/w factor ?)
*Implement lightfield shader
*Add some kind of fog to lights
*Restore depth partitioning using depth ranges
*Move effect of cloud coverage from water shader to sunlight shader
*Allow light masks using textures
*Render light volumes with back face culling and depth clamp instead of front face culling, and don't light pixels in from of the light volume.

=== Long term ideas (unsorted) ===
*implement volumetric effects by extending lights to arbitrary shaders. Should enable to implement heat haze effect and real wake waves.
*implement strength of glow (in the emissive buffer alpha channel) - use a Poisson-disk distribution to implement variable size blur
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

Project Rembrandt

2012-08-12T18:40:56Z

Fredb: /* Long term ideas (unsorted) */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

'''Beware: <Effect> only works on real objects, not on groups of objects or animations.'''

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Performance and compatibility considerations ====
Every light on screen is equivalent for the GPU of redrawing the light volume with a shader, without z buffer culling. So each light comes with a cost, that is small taken individually but noticeable when a lot of them are visible. That cost also increase with the size of the light volume.

Beside that, it is wise to allow a model to work with the classical renderer that know nothing about lights and would render light volumes like other geometry. So a good practice is to complement each light animation with a select animation checking :
*if Rembrandt is enabled
*if the user selected quality match with the purpose of the light
*if the light should be on or off according to the other parameters of the simulation (position of the sun, position of switch, presence of power, ...)

A quality property is created to reflect the user preferences about quality vs performance concerning lights, and a proper slider is added to the shader dialog.
[[File:Fgfs-shaders-with-light.jpg|thumb|Light quality slider]]
The propery to use is :
/sim/rendering/shaders/lights
The quality slider sets its range from 0 (no lights) to 4 (all lights on). Simple airport lamp post appears at 1. Few bridge lamps at 2, all simple bridge lamp at 3 and more involved one at 4.

Example:
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSource</name>
<object-name>LightVolume</object-name>
...
</animation>

<animation>
<type>select</type>

<object-name>LightSource</object-name>
<condition>
<and>

<property>/sim/rendering/rembrandt/enabled</property>

<greater-than>
<property>/sim/rendering/shaders/lights</property>
<value>3.0</value>
</greater-than>

<greater-than>
<property>/sim/time/sun-angle-rad</property>
<value>1.57</value>
</greater-than>
</and>
</condition>
</animation>

</syntaxhighlight>

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)
*Fix fog on clouds

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix shadow matrices in multi-screen
*Fix Lights in multi screen (seems same problem than shadows. Forgot a 1/w factor ?)
*Implement lightfield shader
*Add some kind of fog to lights
*Restore depth partitioning using depth ranges
*Move effect of cloud coverage from water shader to sunlight shader
*Allow light masks using textures

=== Long term ideas (unsorted) ===
*implement volumetric effects by extending lights to arbitrary shaders. Should enable to implement heat haze effect and real wake waves.
*implement strength of glow (in the emissive buffer alpha channel) - use a Poisson-disk distribution to implement variable size blur
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

FlightGear Newsletter August 2012

2012-08-05T21:58:41Z

Fredb: /* New effects in Rembrandt */

FlightGear Newsletter August 2012

2012-08-05T18:14:51Z

Fredb: /* New effects in Rembrandt */

File:Fgfs-rembrandt-night-vision-2.png

2012-08-05T17:27:48Z

Fredb:

=={{int:filedesc}}==
{{Information
|description={{en|1=Rembrandt postprocessing effect simulating night vision device}}
|date=2012-08-05
|source={{own}}
|author=[[User:Fredb|Fredb]]
|permission=
|other_versions=
|other_fields=
}}

=={{int:license-header}}==
{{self|cc-by-sa-3.0}}

File:Fgfs-rembrandt-old-timer.jpg

2012-08-05T17:27:47Z

Fredb:

=={{int:filedesc}}==
{{Information
|description={{en|1=Rembrandt postprocessing effect simulating very old film}}
|date=2012-08-05
|source={{own}}
|author=[[User:Fredb|Fredb]]
|permission=
|other_versions=
|other_fields=
}}

=={{int:license-header}}==
{{self|cc-by-sa-3.0}}

File:Fgfs-rembrandt-distortion.jpg

2012-08-05T17:27:44Z

Fredb:

=={{int:filedesc}}==
{{Information
|description={{en|1=Rembrandt postprocessing effect simulating ordinary lens distortion (pincushion, vignette and chromatic aberration)}}
|date=2012-08-05
|source={{own}}
|author=[[User:Fredb|Fredb]]
|permission=
|other_versions=
|other_fields=
}}

=={{int:license-header}}==
{{self|cc-by-sa-3.0}}

Project Rembrandt

2012-07-29T17:35:49Z

Fredb: /* Near term tasks */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

'''Beware: <Effect> only works on real objects, not on groups of objects or animations.'''

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Performance and compatibility considerations ====
Every light on screen is equivalent for the GPU of redrawing the light volume with a shader, without z buffer culling. So each light comes with a cost, that is small taken individually but noticeable when a lot of them are visible. That cost also increase with the size of the light volume.

Beside that, it is wise to allow a model to work with the classical renderer that know nothing about lights and would render light volumes like other geometry. So a good practice is to complement each light animation with a select animation checking :
*if Rembrandt is enabled
*if the user selected quality match with the purpose of the light
*if the light should be on or off according to the other parameters of the simulation (position of the sun, position of switch, presence of power, ...)

A quality property is created to reflect the user preferences about quality vs performance concerning lights, and a proper slider is added to the shader dialog.
[[File:Fgfs-shaders-with-light.jpg|thumb|Light quality slider]]
The propery to use is :
/sim/rendering/shaders/lights
The quality slider sets its range from 0 (no lights) to 4 (all lights on). Simple airport lamp post appears at 1. Few bridge lamps at 2, all simple bridge lamp at 3 and more involved one at 4.

Example:
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSource</name>
<object-name>LightVolume</object-name>
...
</animation>

<animation>
<type>select</type>

<object-name>LightSource</object-name>
<condition>
<and>

<property>/sim/rendering/rembrandt/enabled</property>

<greater-than>
<property>/sim/rendering/shaders/lights</property>
<value>3.0</value>
</greater-than>

<greater-than>
<property>/sim/time/sun-angle-rad</property>
<value>1.57</value>
</greater-than>
</and>
</condition>
</animation>

</syntaxhighlight>

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)
*Fix fog on clouds

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix shadow matrices in multi-screen
*Fix Lights in multi screen (seems same problem than shadows. Forgot a 1/w factor ?)
*Implement lightfield shader
*Add some kind of fog to lights
*Restore depth partitioning using depth ranges
*Move effect of cloud coverage from water shader to sunlight shader
*Allow light masks using textures

=== Long term ideas (unsorted) ===
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

Project Rembrandt

2012-07-29T17:33:56Z

Fredb: /* Registering all translucent surfaces */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

'''Beware: <Effect> only works on real objects, not on groups of objects or animations.'''

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Performance and compatibility considerations ====
Every light on screen is equivalent for the GPU of redrawing the light volume with a shader, without z buffer culling. So each light comes with a cost, that is small taken individually but noticeable when a lot of them are visible. That cost also increase with the size of the light volume.

Beside that, it is wise to allow a model to work with the classical renderer that know nothing about lights and would render light volumes like other geometry. So a good practice is to complement each light animation with a select animation checking :
*if Rembrandt is enabled
*if the user selected quality match with the purpose of the light
*if the light should be on or off according to the other parameters of the simulation (position of the sun, position of switch, presence of power, ...)

A quality property is created to reflect the user preferences about quality vs performance concerning lights, and a proper slider is added to the shader dialog.
[[File:Fgfs-shaders-with-light.jpg|thumb|Light quality slider]]
The propery to use is :
/sim/rendering/shaders/lights
The quality slider sets its range from 0 (no lights) to 4 (all lights on). Simple airport lamp post appears at 1. Few bridge lamps at 2, all simple bridge lamp at 3 and more involved one at 4.

Example:
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSource</name>
<object-name>LightVolume</object-name>
...
</animation>

<animation>
<type>select</type>

<object-name>LightSource</object-name>
<condition>
<and>

<property>/sim/rendering/rembrandt/enabled</property>

<greater-than>
<property>/sim/rendering/shaders/lights</property>
<value>3.0</value>
</greater-than>

<greater-than>
<property>/sim/time/sun-angle-rad</property>
<value>1.57</value>
</greater-than>
</and>
</condition>
</animation>

</syntaxhighlight>

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)
*Fix fog on clouds

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix shadow matrices in multi-screen
*Fix Lights in multi screen (seems same problem than shadows. Forgot a 1/w factor ?)
*Implement lightfield shader
*Add some kind of fog to lights
*Restore depth partitioning using depth ranges
*Move effect of cloud coverage from water shader to sunlight shader

=== Long term ideas (unsorted) ===
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

Project Rembrandt

2012-07-22T08:05:15Z

Fredb: /* Adding lights to a model */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Performance and compatibility considerations ====
Every light on screen is equivalent for the GPU of redrawing the light volume with a shader, without z buffer culling. So each light comes with a cost, that is small taken individually but noticeable when a lot of them are visible. That cost also increase with the size of the light volume.

Beside that, it is wise to allow a model to work with the classical renderer that know nothing about lights and would render light volumes like other geometry. So a good practice is to complement each light animation with a select animation checking :
*if Rembrandt is enabled
*if the user selected quality match with the purpose of the light
*if the light should be on or off according to the other parameters of the simulation (position of the sun, position of switch, presence of power, ...)

A quality property is created to reflect the user preferences about quality vs performance concerning lights, and a proper slider is added to the shader dialog.
[[File:Fgfs-shaders-with-light.jpg|thumb|Light quality slider]]
The propery to use is :
/sim/rendering/shaders/lights
The quality slider sets its range from 0 (no lights) to 4 (all lights on). Simple airport lamp post appears at 1. Few bridge lamps at 2, all simple bridge lamp at 3 and more involved one at 4.

Example:
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSource</name>
<object-name>LightVolume</object-name>
...
</animation>

<animation>
<type>select</type>

<object-name>LightSource</object-name>
<condition>
<and>

<property>/sim/rendering/rembrandt/enabled</property>

<greater-than>
<property>/sim/rendering/shaders/lights</property>
<value>3.0</value>
</greater-than>

<greater-than>
<property>/sim/time/sun-angle-rad</property>
<value>1.57</value>
</greater-than>
</and>
</condition>
</animation>

</syntaxhighlight>

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)
*Fix fog on clouds

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix shadow matrices in multi-screen
*Fix Lights in multi screen (seems same problem than shadows. Forgot a 1/w factor ?)
*Implement lightfield shader
*Add some kind of fog to lights
*Restore depth partitioning using depth ranges
*Move effect of cloud coverage from water shader to sunlight shader

=== Long term ideas (unsorted) ===
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

File:Fgfs-shaders-with-light.jpg

2012-07-22T07:57:59Z

Fredb:

=={{int:filedesc}}==
{{Information
|description={{en|1=Shader selection box with light slider}}
|date=2012-07-22
|source={{own}}
|author=[[User:Fredb|Fredb]]
|permission=
|other_versions=
|other_fields=
}}

=={{int:license-header}}==
{{self|cc-by-sa-3.0}}

Changelog 2.8.0

2012-07-17T18:49:29Z

Fredb: /* Some of the major changes include: */

{{Draft|changelog|}}

==FlightGear 2.8.0 ChangeLog==

The [[FlightGear]] development team is happy to announce the v2.8.0 release of FlightGear, the free, open-source flight simulator.
This new version contains many exciting new features, enhancements and bugfixes.

Founded in 1997, FlightGear is developed by a worldwide group of volunteers, brought together by a shared ambition to create
the most realistic flight simulator possible that is free to use, modify and distribute. FlightGear is used all over the world
by desktop flight simulator enthusiasts, for research in universities and for interactive exhibits in museums.

FlightGear features more than 400 aircraft, a worldwide scenery database, a worldwide multi-player environment, detailed sky modelling, a flexible and open aircraft modelling system, varied networking options, multiple display support, a powerful scripting language and an open architecture. Best of all, being open-source, the simulator is owned by the community and everyone is encouraged to contribute.

Download FlightGear v2.8.0 for free from [http://www.flightgear.org FlightGear.org]

FlightGear - Fly Free!

=== Some of the major changes include: ===

'''Aircraft operations'''

'''AI system'''

'''AI Traffic'''
* Improved aircraft models and textures.

'''Flight dynamics'''
* Synced with the JSBSim project.

'''Environment'''
* Random objects are only placed within certain areas of a terrain texture.
* Airport signs are 3D models now.
* Random 3D buildings for urban areas, complete with lighting at night.

'''Interface'''
* Support for menu translations.

'''Highlighted new and improved aircraft'''
* [[Airwave Xtreme 150]]: complete new (JSBSim) flight dynamics, new model, advanced pilot animations.
* [[Cessna 337G Skymaster]]

'''Project infrastructure'''
* Various improvements to our scenery database have made it easier than ever to add, delete or update objects to the FlightGear world.

'''Visual effects'''
* A new, experimental renderer, named after the famous painter Rembrandt, is included for testing purposes.
* Easy selection of material files.

'''Other'''
* Additional joysticks and rudder pedals are supported out-of-the-box:
** InterLink Elite
** Qware USB
** Saitek Cyborg X (F.L.Y. 5)
** Saitek Pro Flight Trim Wheel
** Saitek Pro Flight Yoke
** Speedlink Defender

'''Bug fixes'''
* See [http://code.google.com/p/flightgear-bugs/issues/list?can=1&q=Milestone%3D2.8.0 our bugtracker] for an extensive list of the bugs fixed in this release.

[[Category:FlightGear changelogs]]

Project Rembrandt

2012-07-14T18:38:10Z

Fredb: /* What is it ? */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)
*Fix fog on clouds

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix shadow matrices in multi-screen
*Fix Lights in multi screen (seems same problem than shadows. Forgot a 1/w factor ?)
*Implement lightfield shader
*Add some kind of fog to lights
*Restore depth partitioning using depth ranges
*Move effect of cloud coverage from water shader to sunlight shader

=== Long term ideas (unsorted) ===
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

Fr/Projet Rembrandt

2012-07-14T18:36:54Z

Fredb: /* De quoi s'agit-il ? */

{{WIP}}
== Pourquoi ce nom? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] était un peintre hollandais du XVIIème siècle, maître incontesté du [http://en.wikipedia.org/wiki/Chiaroscuro "clair-obscur"].

Ce projet a pour but de changer la manière de rendre les éclairages et les [[Shadows|ombres]] qu'utilise [[FlightGear]], et d'essayer d'imiter le style de Rembrandt dans FG.

== De quoi s'agit-il ? ==
L'idée maîtresse du projet est d'implémenter le [http://en.wikipedia.org/wiki/Deferred_shading rendu différé] au sein de FlightGear.
Depuis le début, FlightGear utilise un moteur de rendu en avant qui affiche toutes les propriétés d'un objet en une fois (ombre, éclairage, brouillard, ...), ce qui rend difficile le rendu d'un ombrage plus sophistiqué (voir le [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) du fait que tous les paramètres de [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ l'équation lumineuse] doivent être traités en une seule fois.

[[Image:project_rembrandt_1.png|thumb|300px|Vue principale avec le contenu des buffers affichés dans les coins]]
Au contraire, le rendu différé cherche à séparer les opérations en étapes simplifiées et à collecter les résultats intermédiaires dans des buffers cachés, qui peuvent être utilisés par la prochain étape.

;Le premier stade est celui de la Géométrie:
: nous rendons la scène entière dans 4 textures, en utilisant le rendu multi-cibles pour les traiter en un seul passage: un pour le buffer de profondeur, un pour les normales (en bas à gauche de l'image), un pour les couleurs "diffuses" (en bas à gauche) et un pour les couleurs "spéculaires" (en haut à droite).

;Le stade suivant est celui de l'ombre :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix fog on clouds
*Fix shadow matrices in multi-screen
*Implement lightfield shader

=== Long term ideas (unsorted) ===
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore depth partitioning using depth ranges
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

Fr/Projet Rembrandt

2012-07-14T18:34:01Z

Fredb: /* De quoi s'agit-il ? */

{{WIP}}
== Pourquoi ce nom? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] était un peintre hollandais du XVIIème siècle, maître incontesté du [http://en.wikipedia.org/wiki/Chiaroscuro "clair-obscur"].

Ce projet a pour but de changer la manière de rendre les éclairages et les [[Shadows|ombres]] qu'utilise [[FlightGear]], et d'essayer d'imiter le style de Rembrandt dans FG.

== De quoi s'agit-il ? ==
L'idée maîtresse du projet est d'implémenter le [http://en.wikipedia.org/wiki/Deferred_shading rendu différé] au sein de FlightGear.
Depuis le début, FlightGear utilise un moteur de rendu en avant qui affiche toutes les propriétés d'un objet en une fois (ombre, éclairage, brouillard, ...), ce qui rend difficile le rendu d'un ombrage plus sophistiqué (voir le [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) du fait que tous les paramètres de [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ l'équation lumineuse] doivent être traités en une seule fois.

[[Image:project_rembrandt_1.png|thumb|300px|Vue principale avec le contenu des buffers affichés dans les coins]]
Au contraire, le rendu différé cherche à séparer les opérations en étapes simplifiées et à collecter les résultats intermédiaires dans des buffers cachés, qui peuvent être utilisés par la prochain étape.

;Le premier stade est celui de la Géométrie:
: nous rendons la scène entière en 4 textures, en utilisant des objets multi-rendu pour les traiter en un seul passage: un pour le buffer de profondeur, un pour les normales (en bas à gauche de l'image), un pour les couleurs "diffuses" (en bas à gauche) et un pour les couleurs "spéculaires" (en haut à droite).

;Le stade suivant est celui de l'ombre :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix fog on clouds
*Fix shadow matrices in multi-screen
*Implement lightfield shader

=== Long term ideas (unsorted) ===
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore depth partitioning using depth ranges
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

Fr/Projet Rembrandt

2012-07-14T18:24:56Z

Fredb: /* Pourquoi ce nom? */

{{WIP}}
== Pourquoi ce nom? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] était un peintre hollandais du XVIIème siècle, maître incontesté du [http://en.wikipedia.org/wiki/Chiaroscuro "clair-obscur"].

Ce projet a pour but de changer la manière de rendre les éclairages et les [[Shadows|ombres]] qu'utilise [[FlightGear]], et d'essayer d'imiter le style de Rembrandt dans FG.

== De quoi s'agit-il ? ==
L'idée maîtresse du projet est d'obtenir un rendu différé de l'ombre [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] au sein de FlightGear.
Au début, FlightGear essayait de rendre toutes les propriétés d'un objet en un tout (ombre, éclairage, brouillard, ...) ce qui rendait difficile la mise au point des ombres plus sophistiquées
(voir le [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) du fait que tous les aspects de l'équation 3danimation/rendu sont traités d'un bloc [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Vue principale avec le contenu des buffers affichés dans les coins]]
Au contraire, le rendu différé cherche à séparer les opérations en stades simplifiés et a collecter les résultats intermédiaires dans des buffers cachés, qui peuvent être utilisés par le prochain stade.

;Le premier stade est celui de la Géométrie:
: nous rendons la scène entière en 4 textures, en utilisant des objets multi-rendu pour les traiter en un seul passage: un pour le buffer profondeur, un pour les normals (en bas à gauche de l'image), un pour les couleurs "diffuse" (en bas à gauche) et un pour les couleurs "specular" (en haut à droite).

;Le stade suivant est celui de l'ombre :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix fog on clouds
*Fix shadow matrices in multi-screen
*Implement lightfield shader

=== Long term ideas (unsorted) ===
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore depth partitioning using depth ranges
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

Project Rembrandt

2012-07-14T18:19:20Z

Fredb: /* TODO List */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)
*Fix fog on clouds

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix shadow matrices in multi-screen
*Fix Lights in multi screen (seems same problem than shadows. Forgot a 1/w factor ?)
*Implement lightfield shader
*Add some kind of fog to lights
*Restore depth partitioning using depth ranges
*Move effect of cloud coverage from water shader to sunlight shader

=== Long term ideas (unsorted) ===
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

Project Rembrandt

2012-07-14T18:17:19Z

Fredb: /* Near term tasks */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix fog on clouds
*Fix shadow matrices in multi-screen
*Fix Lights in multi screen (seems same problem than shadows. Forgot a 1/w factor ?)
*Implement lightfield shader

=== Long term ideas (unsorted) ===
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore depth partitioning using depth ranges
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]
[[fr:Projet Rembrandt]]

Project Rembrandt

2012-07-08T18:00:54Z

Fredb: References

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix fog on clouds
*Fix shadow matrices in multi-screen
*Implement lightfield shader

=== Long term ideas (unsorted) ===
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore depth partitioning using depth ranges
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

== Appendix ==
{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
* {{cite web |url=http://directtovideo.wordpress.com/2009/11/13/deferred-rendering-in-frameranger/ |title=deferred rendering in frameranger. |author=Matt Swoboda |date=13 November 2009 |accessdate=8 July 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]

Project Rembrandt

2012-07-04T09:43:26Z

Fredb: /* Completed tasks */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)
*Global strength of glow or ambient occlusion via slider in rendering dialog
*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix fog on clouds
*Fix shadow matrices in multi-screen
*Implement lightfield shader

=== Long term ideas (unsorted) ===
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore depth partitioning using depth ranges
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]

Project Rembrandt

2012-07-04T09:41:44Z

Fredb: Restructure TODO List

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

== TODO List ==

=== Completed tasks ===
*Fix shadow rendering when using multi threading in OSG
*Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night)
*Honor 'noshadow' animation (done)
*See what happens with glow in fog : unknown landclass creates white patches in the emission buffer - scenery generation problem
*Test multi-screen (mostly done)
*Restore splashscreen
*Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage) (OK - needs model contribution)
*Add spotlights as animations (nearly finished)
*find a solution for ambient and emissive color of material (may need an additional buffer)
*Provide a shader for transparent objects that could render to the emissive buffer too (using MRT) not doable. Light pass can't use MRT
*Use stencil buffer to limit light range(no - done in light shader)
**needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe
*Use effect system instead of hard-coded shaders
*Add new animation to link a light source to a model (need to provide point light animation duplicating spot light) (done)
*Tidy up the architecture (done)

*Fix dim-factor in multiplayer mode (done)
*Design and implement a configurable pipeline (done)
*Document rendering pipeline configuration file format (done)

=== Near term tasks ===
*Convert existing shaders to deferred rendering
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix fog on clouds
*Fix shadow matrices in multi-screen
*Implement lightfield shader

=== Long term ideas (unsorted) ===
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
*Restore depth partitioning using depth ranges
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]

Project Rembrandt

2012-07-04T09:35:11Z

Fredb: /* "Merge in next" TODO List */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. There are two ways to enable it (only one is needed!):
* <code>--enable-rembrandt</code> (when using [[FGRun]], you may add this behind the <tt>FG_EXECUTABLE</tt> on the first page).
* <code>--prop:/sim/rendering/rembrandt/enabled=true</code> (with FGRun this can be added via <tt>Advanced > Properties</tt> on the last page).

The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

== Initial TODO List ==
*<del>Fix shadow rendering when using multi threading in OSG</del>
*<del>Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night</del>)
*<del>Honor 'noshadow' animation</del> (done)
*<del>See what happens with glow in fog</del> : unknown landclass creates white patches in the emission buffer - scenery generation problem
*<del>Test multi-screen</del> (mostly done)
*Fix shadow matrices in multi-screen
*<del>Restore splashscreen</del>
*<del>Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage)</del> (OK - needs model contribution)
*<del>Add spotlights as animations (nearly finished)</del>
*<del>find a solution for ambient and emissive color of material (may need an additional buffer)</del>
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*<del>Provide a shader for transparent objects that could render to the emissive buffer too (using MRT)</del> not doable. Light pass can't use MRT
*<del>Use stencil buffer to limit light range</del>(no - done in light shader)
**<del>needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe</del>
*<del>Use effect system instead of hard-coded shaders</del>
*Convert existing shaders to deferred rendering
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
**<del>Add new animation to link a light source to a model</del> <del>(need to provide point light animation duplicating spot light)</del> (done)
*<del>Tidy up the architecture</del> (done)
*Restore depth partitioning using depth ranges
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

=== Completed tasks ===
*<del>Fix dim-factor in multiplayer mode</del> (done)
*<del>Design and implement a configurable pipeline</del> (done)
*<del>Document rendering pipeline configuration file format</del> (done)

=== Near term tasks ===
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*take care of particles and precipitation
*Fix fog on clouds
*Implement lightfield shader

=== Long term ideas (unsorted) ===

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]

Project Rembrandt

2012-06-30T14:53:57Z

Fredb: /* Initial TODO List */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. Use the <tt>--enable-rembrandt</tt> option to start FlightGear with this new renderer. The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

== Initial TODO List ==
*<del>Fix shadow rendering when using multi threading in OSG</del>
*<del>Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night</del>)
*<del>Honor 'noshadow' animation</del> (done)
*<del>See what happens with glow in fog</del> : unknown landclass creates white patches in the emission buffer - scenery generation problem
*<del>Test multi-screen</del> (mostly done)
*Fix shadow matrices in multi-screen
*<del>Restore splashscreen</del>
*<del>Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage)</del> (OK - needs model contribution)
*<del>Add spotlights as animations (nearly finished)</del>
*<del>find a solution for ambient and emissive color of material (may need an additional buffer)</del>
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*<del>Provide a shader for transparent objects that could render to the emissive buffer too (using MRT)</del> not doable. Light pass can't use MRT
*<del>Use stencil buffer to limit light range</del>(no - done in light shader)
**<del>needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe</del>
*<del>Use effect system instead of hard-coded shaders</del>
*Convert existing shaders to deferred rendering
*Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
**<del>Add new animation to link a light source to a model</del> <del>(need to provide point light animation duplicating spot light)</del> (done)
*<del>Tidy up the architecture</del> (done)
*Restore depth partitioning using depth ranges
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== "Merge in next" TODO List ==
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*<del>Fix dim-factor in multiplayer mode</del> (done)
*take care of particles and precipitation
*<del>Design and implement a configurable pipeline</del> (done)
*Implement lightfield shader
*<del>Document rendering pipeline configuration file format</del> (done)
*Fix fog on clouds

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]

Project Rembrandt

2012-06-30T14:52:36Z

Fredb: /* Initial TODO List */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. Use the <tt>--enable-rembrandt</tt> option to start FlightGear with this new renderer. The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

== Initial TODO List ==
*<del>Fix shadow rendering when using multi threading in OSG</del>
*<del>Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night</del>)
*<del>Honor 'noshadow' animation</del> (done)
*<del>See what happens with glow in fog</del> : unknown landclass creates white patches in the emission buffer - scenery generation problem
*<del>Test multi-screen</del> (mostly done)
*Fix shadow matrices in multi-screen
*<del>Restore splashscreen</del>
*<del>Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage)</del> (OK - needs model contribution)
*<del>Add spotlights as animations (nearly finished)</del>
*<del>find a solution for ambient and emissive color of material (may need an additional buffer)</del>
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*<del>Provide a shader for transparent objects that could render to the emissive buffer too (using MRT)</del> not doable. Light pass can't use MRT
*<del>Use stencil buffer to limit light range</del>(no - done in light shader)
**<del>needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe</del>
*<del>Use effect system instead of hard-coded shaders</del>
*Convert existing shaders to deferred rendering
**Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
**<del>Add new animation to link a light source to a model</del> <del>(need to provide point light animation duplicating spot light)</del> (done)
*<del>Tidy up the architecture</del> (done)
*Restore depth partitioning using depth ranges
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== "Merge in next" TODO List ==
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*<del>Fix dim-factor in multiplayer mode</del> (done)
*take care of particles and precipitation
*<del>Design and implement a configurable pipeline</del> (done)
*Implement lightfield shader
*<del>Document rendering pipeline configuration file format</del> (done)
*Fix fog on clouds

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]

Project Rembrandt

2012-06-26T08:13:02Z

Fredb: /* "Merge in next" TODO List */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. Use the <tt>--enable-rembrandt</tt> option to start FlightGear with this new renderer. The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

== Initial TODO List ==
*<del>Fix shadow rendering when using multi threading in OSG</del>
*<del>Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night</del>)
*Honor <noshadow> animation directive
*<del>See what happens with glow in fog</del> : unknown landclass creates white patches in the emission buffer - scenery generation problem
*<del>Test multi-screen</del> (mostly done)
*Fix shadow matrices in multi-screen
*<del>Restore splashscreen</del>
*<del>Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage)</del> (OK - needs model contribution)
*<del>Add spotlights as animations (nearly finished)</del>
*<del>find a solution for ambient and emissive color of material (may need an additional buffer)</del>
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*<del>Provide a shader for transparent objects that could render to the emissive buffer too (using MRT)</del> not doable. Light pass can't use MRT
*<del>Use stencil buffer to limit light range</del>(no - done in light shader)
**<del>needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe</del>
*<del>Use effect system instead of hard-coded shaders</del>
*Convert existing shaders to deferred rendering
**Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
**<del>Add new animation to link a light source to a model</del> <del>(need to provide point light animation duplicating spot light)</del> (done)
*<del>Tidy up the architecture</del> (done)
*Restore depth partitioning using depth ranges
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== "Merge in next" TODO List ==
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*<del>Fix dim-factor in multiplayer mode</del> (done)
*take care of particles and precipitation
*<del>Design and implement a configurable pipeline</del> (done)
*Implement lightfield shader
*<del>Document rendering pipeline configuration file format</del> (done)
*Fix fog on clouds

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]

Project Rembrandt

2012-06-26T08:12:34Z

Fredb: /* "Merge in next" TODO List */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. Use the <tt>--enable-rembrandt</tt> option to start FlightGear with this new renderer. The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

== Initial TODO List ==
*<del>Fix shadow rendering when using multi threading in OSG</del>
*<del>Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night</del>)
*Honor <noshadow> animation directive
*<del>See what happens with glow in fog</del> : unknown landclass creates white patches in the emission buffer - scenery generation problem
*<del>Test multi-screen</del> (mostly done)
*Fix shadow matrices in multi-screen
*<del>Restore splashscreen</del>
*<del>Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage)</del> (OK - needs model contribution)
*<del>Add spotlights as animations (nearly finished)</del>
*<del>find a solution for ambient and emissive color of material (may need an additional buffer)</del>
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*<del>Provide a shader for transparent objects that could render to the emissive buffer too (using MRT)</del> not doable. Light pass can't use MRT
*<del>Use stencil buffer to limit light range</del>(no - done in light shader)
**<del>needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe</del>
*<del>Use effect system instead of hard-coded shaders</del>
*Convert existing shaders to deferred rendering
**Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
**<del>Add new animation to link a light source to a model</del> <del>(need to provide point light animation duplicating spot light)</del> (done)
*<del>Tidy up the architecture</del> (done)
*Restore depth partitioning using depth ranges
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== "Merge in next" TODO List ==
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*<del>Fix dim-factor in multiplayer mode</del>(done)
*take care of particles and precipitation
*<del>Design and implement a configurable pipeline</del> (done)
*Implement lightfield shader
*<del>Document rendering pipeline configuration file format</del>(done)
*Fix fog on clouds

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]

Project Rembrandt

2012-06-26T08:11:22Z

Fredb: /* "Merge in next" TODO List */

{{WIP}}
== Why this name ? ==
[http://en.wikipedia.org/wiki/Rembrandt Rembrandt] was a dutch painter living in the 17th century, famous as one of the master of [http://en.wikipedia.org/wiki/Chiaroscuro chiaroscuro].

This project is about changing the way [[FlightGear]] renders lights, [[shadows]] and shades, and aims at making Rembrandt painting style possible in FG.

== What is it ? ==
The idea driving the project is to implement [http://en.wikipedia.org/wiki/Deferred_shading deferred rendering] inside FlightGear.
From the beginning FlightGear had a forward renderer that tries to render all properties of an object in one pass (shading, lighting, fog, ...), making it difficult to render more sophisticated shading
(see the [[FlightGear Newsletter December 2011#Shaders|'Uber-shader']]) because one has to take into account all aspects of the [http://www.wired.com/magazine/2010/07/st_equation_3danimation/ rendering equation].

[[Image:project_rembrandt_1.png|thumb|300px|Main view with the content of buffers displayed at corners]]
On the contrary, deferred rendering is about separating operations in simplified stages and collecting the intermediary results in hidden buffers that can be used by the next stage.

;First stage is the Geometry Stage :
: we render all the scene into 4 textures, using multi render targets, to do it in one pass: one for the depth buffer, one for the normals (lower left of the image), one for the diffuse colors (lower right) and one for the specular colors (upper right).

;Next stage is the Shadow Stage :
: we render the scene again into a depth texture from the point of view of the lights. There will be one texture for every light casting shadows.

;Then comes the Lighting Stage, with several substages :
:*Sky pass: the sky is first drawn using classical method.
:*Ambient pass: the diffuse buffer is modulated with the ambient color of the scene and is drawn as a screen-aligned textured quad
:*Sunlight pass: a second screen aligned quad is drawn and a shader computes the view position of every pixel to compute its diffuse and specular color, using the normal stored in the first stage. The resulting color is blended with the previous pass. Shadows are computed here by comparing the position of the pixel with the position of the light occluder stored in the shadow map.
:*Additional light pass: the scene graph will be traversed another time to display light volumes (cone or frusta for spot lights, sphere for omni-directional lights) and their shader will add the light contributed by the source only on pixels receiving light.
:*Fog pass: a new screen aligned quad is draw and the position of the pixel is computed to evaluate the amount of fog the pixel has. The fog color is blended with the result of the previous stage.
:*Transparent objects pass: transparent objects (and clouds) are finally rendered using classical method.

:All lighting computations are accumulated in a single buffer that will be used for the last stage, in addition of the one computed by the Geometry stage.

;In the end, the Display Stage, with optional Post-Processing effect :
:The results of the previous buffers are pushed to the main framebuffer to be displayed, optionally modified to show Glow, Motion blur, HDR, redout or blackout, screen-space ambient occlusion, anti-aliasing, etc...

In FG, we end the rendering pipeline by displaying the [[Menubar|GUI]] and the [[HUD]].

All these stages are more precisely described if this [http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf tutorial] that is the basis of the current code, with some addition and modifications.

== Caveats ==
Deferred rendering is not able to display transparency. For the moment, clouds are renderer separately and should be lit and shaded by their own. Transparent surfaces are alpha-tested and not blended. They would have to be drawn in their own bin over the composited image.

It also don't fit with depth partitioning because the depth buffer should be kept to retain the view space position, so for the moment, z-fighting is quite visible. Depth partitioning with non overlapping depth range might be the solution and should be experimented at one point.

The glow pass can make certain MFD (that use emissive color) unreadable because blurred. Should be treated as transparent.

== Implementation ==
=== Repositories ===
The code is in the main branch of the [[FlightGear and Git|official repository]]. Any other location is not maintained anymore.

=== Rendering of transparent surfaces ===
[[Image:project_rembrandt_5.png|thumb|300px|Transparent surfaces drawn after opaque objects]]
Transparent surfaces are detected by OSG loader plugins and their state set receive the TRANSPARENT_BIN rendering hint. In the culling pass, the cull visitor orders transparent surfaces in transparent bin. In a cull callback attached to the Geometry camera, after the scenegraph traversal, the transparent bins are removed from the render stage and saved in a temporary collection. In a cull callback attached to the Lighting camera, after the scenegraph traversal, the transparent bins saved at the previous stage, are added to the render stage of the Lighting camera with a high order num. That way, the transparent surface are drawn on top of the scene lighted from the Gbuffer.

=== Memory consumption ===
For each camera defined in the camera group, there is a separate shadow map, so the video memory usage is :
* G-buffer and Lighting buffer: 20 bytes per pixel. For an HD screen (1920x1080) memory requirement is 40 Mb
* Shadow map: 3 x shadow_map_size x shadow_map_size bytes (if size is 8192, whole map size is 192 Mb
Not counting textures, display list or vertex buffers for models and terrain

3 HD screens require 120 Mb of memory for the buffers (shadow excluded), you're asking 3x8192x8192x3 = 576 Mb (megabytes) of memory for the shadows alone.

If you are seeing error messages during startup or FlightGear doesn't start up properly, it's probably because you don't have enough free video memory. Reduce the size of the shadow map in preferences.xml by locating

<syntaxhighlight lang="xml">
<map-size type="int">8192</map-size>
</syntaxhighlight>

And put 4096 or 2048 instead.
You can also use a startup parameter: --prop:/sim/rendering/shadows/map-size=2048

=== Configurable pipeline ===

The Rembrandt renderer uses an XML file to setup its pipeline for each viewport described in the camera group. This file describes the way the intermediary buffers are setup and how the different rendering stages are sequenced. The general outline of a pipeline file is as follow :

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="utf-8"?>
<PropertyList>

<buffer>

</buffer>
<buffer>

</buffer>


<stage>

</stage>
<stage>

</stage>
</PropertyList>
</syntaxhighlight>

==== Buffers ====
A buffer is a texture used as a storage area in the GPU. It's size is usually a multiple of the screen size, but fixed size is supported (typical for shadow map). The description of a buffer is described below :

<syntaxhighlight lang="xml">
<buffer>
<name>buffer-name</name>
<internal-format>rgba8</internal-format> 
<source-format>rgba</source-format> 
<source-type>unsigned-byte</source-type> 
<width>screen</width> 
<height>screen</height> 
<scale-factor>1.0</scale-factor>
<wrap-mode>clamp-to-border</wrap-mode> 

<shadow-comparison>true</shadow-comparison>

<condition>

</condition>
</buffer>
</syntaxhighlight>

==== Stages ====
A stage is an unit of rendering to a group of buffer. Most stages are predefined and their type is not free. When a type is not specified, the name is used. Stage types are :
{| class="wikitable"
!align="left"| Stage type
!align="left"| Purpose
|-
|geometry
|The geometry stage initialize most of the buffers and works on the real objects and geometry. Transparent objects are set aside and will be used untouched in the lighting stage. Other opaque geometry is rendered with the standard effects that do the hard work to put sensible data in the buffers.
|-
|shadow
|In this stage, the geometry is rendered in the normal map from the perspective of the sun.
|-
|lighting
|This stage uses the buffers filled by the previous stages to light every pixel of the scene. The result is rendered in another buffer to allow post effects.
|-
|fullscreen
|Stages of this type are used to alter the whole scene or transform data from a particular buffer.
|-
|display
|Final rendering of the scene to the screen or the texture defined in the camera group.
|}

A stage description is outlined below :
<syntaxhighlight lang="xml">
<stage>
<name>stage-name</name>
<type>stage-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 
<needs-du-dv>true</needs-du-dv> 
<scale-factor>0.25</scale-factor> 


<condition>

</condition>

<attachment>

</attachment>
<attachment>

</attachment>


<pass>

</pass>
<pass>

</pass>
</stage>
</syntaxhighlight>
Stages render in buffers (except for the display stage). Attachments describe which buffers are affected by each stage.

===== Attachments =====
Attachment describe bindings between buffer and attachment point :
<syntaxhighlight lang="xml">
<attachment>
<component>color0</component> 
<buffer>buffer-name</buffer>

<condition>

</condition>
</attachment>
</syntaxhighlight>

===== Passes =====
Passes are only available in the <tt>lighting</tt> stage. Three kind of stage are allowed :

{| class="wikitable"
!align="left"| Pass type
!align="left"| Purpose
|-
|sky-clouds
|Renders the skydome, sun, moon, planet, stars and clouds
|-
|lights
|Renders additional spot and point lights
|-
|fullscreen
|Fullscreen pass analog to a fullscreen stage except that it renders in the buffers attached to the lighting stage
|}
A pass is defined like below :
<syntaxhighlight lang="xml">
<pass>
<name>pass-name</name>
<type>pass-type</type> 
<order-num>-1</order-num>
<effect>Effects/fullscreen-effect</effect> 


<condition>

</condition>
</pass>
</syntaxhighlight>

A typical lighting stage is a succession of 5 passes :
# <tt>sky-clouds</tt> pass
# <tt>fullscreen</tt> pass for ambient light
# <tt>fullscreen</tt> pass for sun light (and shadows)
# <tt>lights</tt> pass
# <tt>fullscreen</tt> pass for fog

Each effect attached to the fullscreen passes define the way blending is done between the pass and the previous accumulation of render.

== Running Flightgear with Rembrandt ==
[[File:Rembrandt dialog.png|thumb|270px|Rembrandt dialog]]
The Rembrandt renderer is now integrated in the main repository but needs to be enabled to run. Use the <tt>--enable-rembrandt</tt> option to start FlightGear with this new renderer. The <tt>View > Rendering Options > Rembrandt Options</tt> dialog allows you to toggle and adjust the various features that Rembrandt offers.

Rembrandt is quite demanding in GPU resources and may fail to run with the default options. The more frequent symptom is an OSG message in the console :

RenderStage::runCameraSetUp(), FBO setup failed, FBO status= 0x8cd6
Warning: RenderStage::runCameraSetUp(State&) Pbuffer does not support multiple color outputs.

Some card also exhibit messages like this :

glLinkProgram "" FAILED
Program "" infolog:
Fragment info
-------------
0(37) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(36) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(35) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable
0(34) : error C6013: Only arrays of texcoords may be indexed in this profile, and only with a loop index variable

There is a number of additional options that can help to avoid these problems :

{| class="wikitable"
|<tt>--prop:/sim/rendering/rembrandt/use-color-for-depth=true</tt>
|Some old NVidia cards, such as 7600GT, don't give enough resolution for depth and that result in "fog curtains" at few meters from the viewer. One trick is to encode depth in another texture and get the proper value afterward. This option enables that.
|-
|<tt>--prop:/sim/rendering/shadows/enabled=false</tt>
|Disable shadows altogether.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades=1</tt>
|Avoid the "error C6013" message on old cards at the cost of resolution in the cockpit. Set <tt>/sim/rendering/shadows/cascade-far-m[0]</tt> to change the shadow map range. The more the range, the less the resolution (default value is 5 meters)
|-
|<tt>--prop:/sim/rendering/shadows/map-size=<power-of-two></tt>
|Set the shadow map size. Useful values are 1024, 2048, 4096 or 8192. Few cards have the resources to support 16384.
|-
|<tt>--prop:/sim/rendering/shadows/num-cascades</tt>
|Set the shadow map cascade number. Less cascades means less time spent in shadow map generation, but also means lower shadow quality. Integer between 1 and 4.
|-
|<tt>--prop:/sim/rendering/shadows/cascade-far-m[i]</tt>
(1 <= i <= <tt>/sim/rendering/shadows/num-cascades</tt> <= 4)
|Set the shadow map cascade range for each cascade. Default values are 5m, 50m, 500m and 5000m for 4 cascades.
|-
|<tt>--prop:/sim/rendering/rembrandt/no-16bit-buffer=false</tt>
|By default, Rembrandt uses 8 bit buffers for normals (so the property is set to true by default). This may create banding artifacts on specular highlights. If it's unacceptable and the GPU supports it, set to false to have better precision for normals and effects relying on normal direction.
|}

== Guidelines for shader writers ==
=== Predefined uniforms ===
These glsl uniforms don't need to be declared in the effect file.
{| class="wikitable"
!Name
!Type
!Purpose
|-
|<tt>fg_ViewMatrix</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix used to transform the screen position to view direction
|-
|<tt>fg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, view matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_ProjectionMatrixInverse</tt>
|<tt>mat4</tt>
|In fullscreen pass only, projection matrix inverse used to transform the screen position to view direction
|-
|<tt>fg_CameraPositionCart</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in cartesian coordinates
|-
|<tt>fg_CameraPositionGeod</tt>
|<tt>vec3</tt>
|Position of the camera in world space, expressed in geodesic coordinates (longitude in radians, latitude in radians, elevation in meters)
|-
|<tt>fg_SunAmbientColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDiffuseColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunSpecularColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_SunDirection</tt>
|<tt>vec3</tt>
|
|-
|<tt>fg_FogColor</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_FogDensity</tt>
|<tt>float</tt>
|
|-
|<tt>fg_ShadowNumber</tt>
|<tt>int</tt>
|
|-
|<tt>fg_ShadowDistances</tt>
|<tt>vec4</tt>
|
|-
|<tt>fg_DepthInColor</tt>
|<tt>bool</tt>
|Tells if the depth is stored in a depth texture or a color texture
|-
|<tt>fg_Planes</tt>
|<tt>vec3</tt>
|Used to convert the value of the depth buffer to a depth that can be used to compute the eye space position of the fragment
|-
|<tt>fg_BufferSize</tt>
|<tt>vec2</tt>
|Dimensions of the buffer, used to convert gl_FragCoord into the range [0..1][0..1]
|-
|<tt>osg_ViewMatrix</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|-
|<tt>osg_ViewMatrixInverse</tt>
|<tt>mat4</tt>
|Defined by OSG, used only when working on actual geometry
|}
They still have to be declared in the fragment or the vertex shader to be used.

=== Utility functions ===

To ease the maintenance of shaders, several utility functions are available for the fragment shader. These functions are put together in two files : <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt>.

==== gbuffer-encode.frag ====

;<tt>void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth)</tt>
:Used to encode all the values of the G-Buffer in material shaders

==== gbuffer-functions.frag ====

;<tt>vec2 normal_encode(vec3 n)</tt>
:Used to compress normals into the G-Buffer in material shaders. Normally called from <tt>encode_gbuffer()</tt>

;<tt>vec3 normal_decode(vec2 enc)</tt>
:Reconstruct normals from the G-Buffer. Used in fullscreen shaders and light shaders

;<tt>vec3 float_to_color(in float f)</tt>
:Encode float values in the range [0..1] in the 24 bits of a color. This function is used by <tt>encode_gbuffer()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>float color_to_float(vec3 color)</tt>
:Decode float values in the range [0..1] from the 24 bits of a color. This function is used by <tt>position()</tt> if the <tt>/sim/rendering/use-color-for_depth</tt> is true, for old cards that don't provide depth information with enough resolution inside fullscreen or light shaders.

;<tt>vec3 position( vec3 viewDir, float depth )</tt>
:Reconstruct eye space position from the view direction and the depth read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec3 depthColor )</tt>
:Reconstruct eye space position from the view direction and the depth encoded in a color read from the depth buffer

;<tt>vec3 position( vec3 viewDir, vec2 coords, sampler2D depth_tex )</tt>
:Reconstruct eye space position from the view direction and the depth buffer (real depth or color, according to the value of <tt>/sim/rendering/use-color-for_depth</tt>) at a given fragment on screen, given by <tt>coords</tt>

==== Usage ====

For material shaders, it is necessary to provide both <tt>gbuffer-functions.frag</tt> and <tt>gbuffer-encode.frag</tt> in the effect file, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/ubershader.vert</vertex-shader>
<fragment-shader>Shaders/ubershader-gbuffer.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-encode.frag</fragment-shader>
</program>
</syntaxhighlight>

For fullscreen passes shaders, only <tt>gbuffer-functions.frag</tt> should be provided, like this :

<syntaxhighlight lang="xml">
<program>
<vertex-shader>Shaders/sunlight.vert</vertex-shader>
<fragment-shader>Shaders/sunlight.frag</fragment-shader>
<fragment-shader>Shaders/gbuffer-functions.frag</fragment-shader>
</program>
</syntaxhighlight>

In the main function of the shader, the functions referenced need to be declared first. With no #include files, the whole function prototype needs to be typed :
<syntaxhighlight lang="glsl">
void encode_gbuffer(vec3 normal, vec3 color, int mId, float specular, float shininess, float emission, float depth);

main() {
vec3 normal;
vec3 color;
int mId;
float specular;
float shininess;
float emission;
float depth;

// Do shader computations

encode_gbuffer(normal, color, mId, specular, shininess, emission, depth);
}
</syntaxhighlight>

=== Geometry Stage ===
The Geometry Stage is there to fill the G-buffer. '''Shading doesn't occur at this stage, so light or fog computation should not be part of the shader'''. The required operation in the Fragment Shader is to fill every individual buffer with sensible value :

{| class="wikitable" style="text-align: center;"
|depth (gl_FragDepth)||GL_DEPTH_COMPONENT32||colspan="4"|Fragment depth
|-
|gl_FragData[0]||GL_RG16||colspan="2"|normal.x * 0.5 + 0.5||colspan="2"|normal.y * 0.5 + 0.5
|-
|gl_FragData[1]||GL_RGBA8||diffuse.r||diffuse.g||diffuse.b||material id * 1/255.0
|-
|gl_FragData[2]||GL_RGBA8||specular.l||specular.s||emission.l||pixel valid if != 0
|}

This is the default layout expected by the sunlight shader. material Id can be used to detect a different layout



=== Additional light pass ===
There would be a single shader for each light type used. The plan is to create lights like animations in the model XML file. The light shader will retrieve scene geometry by combining screen space position converted in view space ray by the inverse of the projection matrix (an helper function should be provided), and the fragment depth at that screen position read from the depth buffer. With the help of the fragment normal, the diffuse and specular color and the properties of the light the shader implements, it will be possible to add to the lighting buffer the contribution of the light rendered.

=== Fog Pass ===
Using the fragment depth, it will be possible to compute any fog distribution. For the moment, the simple fog equation is implemented.

=== Bloom Pass ===
This is a two-pass effect that blurs the lighting buffer in a small texture. This texture is then added to the lighting buffer at the display stage.

=== Required Effects ===
Several pass are implemented using the effect system. For this purpose, some effects are referenced in the core code using reserved names. these effects are:
{| class="wikitable" cellpadding="3"
!Name
!Kind
!Purpose
|-
|<tt>Effects/ssao</tt>
|Works on a full screen quad
|Compute ambient occlusion from the normal buffer and the depth buffer
|-
|<tt>Effects/ambient</tt>
|Works on a full screen quad
|Copies the diffuse color buffer multiplied by the ambient light to the lighting buffer. Ambient Occlusion can also affect ambient light.
|-
|<tt>Effects/light-spot</tt>
|Works on real geometry of the light volume
|Computes the light contribution of a spot light defined in a <tt>light</tt> animation having a <tt>light-type</tt> of '''<tt>spot</tt>'''
|-
|<tt>Effects/fog</tt>
|Works on a full screen quad
|Computes the fog from the G-buffer and the lighting parameters
|-
|<tt>Effects/display</tt>
|Works on a full screen quad
|Renders the composite final image from the G-buffer and the lighting buffer
|}

== Guidelines for modelers ==

=== Porting aircraft ===
* Rembrandt computes shadows => no more fake shadows in the model
* Rembrandt computes ambient occlusion => no ambient occlusion baked into textures
* Rembrandt has light => static lightmap are not needed, emissive color to see models at night is not needed and would interfere
* Rembrandt has glow => incorrectly used emissive colors may blur displays and make some text unreadable. Light size may have to be adjusted
* Rembrandt has strict needs with shaders => shaders need to be adjusted to comply with the new framework otherwise the view will be plain wrong
* Rembrandt can't do transparent surfaces => transparent surface need to be properly registered to render them with the classical path

=== Registering all translucent surfaces ===

Every model is, by default, rendered using the <tt>Effects/model-default</tt> effect. This effect initialize the G-buffer, ignoring transparent surfaces, by doing alpha testing and rendering all the geometry in the default bin. It is not possible to redirect rendering to transparent bins when the associated texture has alpha channel because most models use a single texture atlas and even opaque parts are rendered with texture with alpha channel.

If a model needs to have transparent or translucent surfaces, these surface objects need to be assigned a different effect that sets explicitly the render bin to "DepthSortedBin", or sets the rendering hint to "transparent". This tells the renderer to render this object using forward rendering, so lighting and fog need to be enabled, and if a shader program is used, they should be computed in the classical way. The <tt>Effects/model-transparent</tt> can be used to register simple transparent/translucent surfaces. You assign this effect to an object (or multiple objects) like:
<effect>
<inherits-from>Effects/model-transparent</inherits-from>
<object-name>TheObject</object-name>
</effect>

If opaque surface need to have special effect, for example to apply bump mapping, this effect should use the "RenderBin" bin, or the rendering hint set to "opaque", and the G-buffer needs to be initialized correctly in the Geometry stage.

=== Adding lights to a model ===

There are two things to consider: the appearance of the light source and the illuminated area. For the appearance of the light source (what you see when you look at the bulb), you need a model with an emissive material that will produce the glow effect and that is visible at night.

For the effect of the source on its environment (the lit area), we must have in the 3D model (the .ac file) a volume that includes the effect (Light Volume). It can be a large cone for spotlights or a sphere for point light. It's important that the light volume is closed, convex and it's normals are oriented outward.

The light volume must be part of the geometry of the model and be referenced in the animation file. No need to add a color or an effect to this volume. Light calculation is only done on the fragments covered by the light volume, but has no influence on the color or the attenuation of the light.

All available animations are possible on the light volume, except '''material''' and '''texture'''. It is not possible to change color of lights for the moment, except switching to another animation. Axis and position are in object space and are transformed by the subsequent animations.

==== Spotlights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>spot</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<direction>
<x>-0.9988</x>
<y>0.0349</y>
<z>-0.0349</z>
</direction>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<exponent>30.0</exponent>
<cutoff>39</cutoff>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''spot'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a cone with an apex at <tt>position</tt>, along <tt>direction</tt> axis if <tt>cutoff</tt> is lesser than 90 degrees, or a sphere centered at <tt>position</tt> if <tt>cutoff</tt> is greater than 90 degrees )
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>direction</tt>
|In object space, direction to the center of the spot
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>exponent</tt>
|Attenuation is multiplied by <tt>pow( dot( lightDir, <direction> ), <exponent> )</tt>, lightDir being vector from light position to point, in camera space.
|-
|<tt>cutoff</tt>
|Point is lit by this source if <tt>dot( lightDir, <direction> ) > <cutoff></tt> , lightDir being vector from light position to point, in camera space.
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

==== Point lights ====
{|cellpadding=10|
|valign=top|
<syntaxhighlight lang="xml">
<animation>
<type>light</type>
<light-type>point</light-type>
<name>LightSrcRight</name>
<object-name>LightRight</object-name>
<nopreview/>
<position>
<x>0.169</x>
<y>0.570</y>
<z>0.713</z>
</position>
<ambient>
<r>0.03</r>
<g>0.03</g>
0.03
<a>1.0</a>
</ambient>
<diffuse>
<r>0.7</r>
<g>0.7</g>
0.6
<a>1.0</a>
</diffuse>
<specular>
<r>0.7</r>
<g>0.7</g>
0.7
<a>1.0</a>
</specular>
<dim-factor>
<property>dimming/property</property>

<expression />
<interpolation />
<factor>1</factor>
<offset>0</offset>
<min>0</min>
<max>1</max>

</dim-factor>
<attenuation>
<c>1.0</c>
<l>0.002</l>
<q>0.00005</q>
</attenuation>
<near-m>3.5</near-m>
<far-m>39</far-m>
</animation>
</syntaxhighlight>
|valign=top|
{| class="wikitable" cellpadding="3"
!Name
!Purpose
|-
|<tt>type</tt>
|Install the <tt>'''light'''</tt> animation
|-
|<tt>light-type</tt>
|This is a <tt>'''point'''</tt> light
|-
|<tt>name</tt>
|Name given to this animation
|-
|<tt>object-name</tt>
|Name of the light volume in the 3d model (typically a sphere centered on <tt>position</tt>, with a radius of <tt>far-m</tt>)
|-
|<tt>nopreview</tt>
|Hide light volume in fgrun 3d preview
|-
|<tt>position</tt>
|In object space, position of the light
|-
|<tt>ambient</tt>
|Ambient color of the light
|-
|<tt>diffuse</tt>
|Diffuse color of the light
|-
|<tt>specular</tt>
|Specular color of the light
|-
|<tt>dim-factor</tt>
|Group of parameters to control a factor that is applied to ambient, diffuse and specular at the same time
|-
|<tt>attenuation</tt>
|Three element vector. <c> element is the constant factor, <l> element is the linear factor and <q> element is the quadratic factor. 
Attenuation of color at distance d is [[File:Spotlight_attenuation.png]]
|-
|<tt>near-m</tt>
|Minimum distance of influence, from position, in meters
|-
|<tt>far-m</tt>
|Maximum distance of influence, from position, in meters
|}
|}

== Initial TODO List ==
*<del>Fix shadow rendering when using multi threading in OSG</del>
*<del>Implement Cascaded Shadow Map (need to be optimized - frustum calculation and night</del>)
*Honor <noshadow> animation directive
*<del>See what happens with glow in fog</del> : unknown landclass creates white patches in the emission buffer - scenery generation problem
*<del>Test multi-screen</del> (mostly done)
*Fix shadow matrices in multi-screen
*<del>Restore splashscreen</del>
*<del>Draw transparent objects with forward rendering (may need to capture the transparent bin from the geometry stage and move it in the display stage)</del> (OK - needs model contribution)
*<del>Add spotlights as animations (nearly finished)</del>
*<del>find a solution for ambient and emissive color of material (may need an additional buffer)</del>
*implement strength of glow (in the emissive buffer alpha channel)
**provide levels 0 to 5 - we are currently at level 5
**level 0 should be ok for MFDs that are currenly unreadable because blurred
*<del>Provide a shader for transparent objects that could render to the emissive buffer too (using MRT)</del> not doable. Light pass can't use MRT
*<del>Use stencil buffer to limit light range</del>(no - done in light shader)
**<del>needed for cockpit light to implement fake shadows and avoid lighting the runway from the cabin through the airframe</del>
*<del>Use effect system instead of hard-coded shaders</del>
*Convert existing shaders to deferred rendering
**Modify shadows to allow multiple casters (limited list)
**Implement a priority list of light sources, based on priority and distance from the viewer
**<del>Add new animation to link a light source to a model</del> <del>(need to provide point light animation duplicating spot light)</del> (done)
*<del>Tidy up the architecture</del> (done)
*Restore depth partitioning using depth ranges
*Restore stereo and other options currently available in CameraGroup
*Implement quality vs performance user control

== "Merge in next" TODO List ==
*Avoid to redraw opaque objects in the light pass. Involve OSG node mask not properly initialized.
*Fix dim-factor in multiplayer mode
*take care of particles and precipitation
*<del>Design and implement a configurable pipeline</del> (done)
*Implement lightfield shader
*Document rendering pipeline configuration file format

== Gallery ==
{{#ev:youtube|PXHhtQb5yzc}} {{#ev:youtube|peEzEapavkg}} {{#ev:youtube|RIetPh8iJXk}} {{#ev:youtube|oaNFrxgQY1c}} {{#ev:youtube|8xGzy12hlis}} {{#ev:youtube|ZyuHBlm3xXU}} {{#ev:youtube|RgH9GZRukOI}} {{#ev:youtube|UQvbHnBkpaM}} {{#ev:youtube|v02phoOqWHE}}{{#ev:youtube|dlSo4sBa7Nk}}

{{Appendix|all|
* {{cite web |url=http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_Deferred_Shading.pdf |title=Deferred Shading |author=Shawn Hargreaves and Mark Harris }}
* {{cite web |url=http://bat710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2007/Deferred_Shading_Tutorial_SBGAMES2005.pdf |title=Deferred Shading Tutorial |author=Fabio Policarpo and Francisco Fonseca }}
* {{cite web |url=http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf |title=Deferred Rendering in Killzone 2 |author=Michal Valient |month=July |year=2007 }}
* {{cite web |url=http://www.anandtech.com/show/5011/nvidiaea-posts-battlefield-3-graphics-tech-talk |title=Battlefield 3 Graphics Tech Talk |author=Johan Andersson |month=October |year=2011 }}
* {{cite web |url=http://www.disney.co.uk/cms_res/blackrockstudio/pdf/Rendering_Techniques_in_SplitSecond.pdf |title=Rendering Techniques in Split/Second |author=Jeremy Moore and David Jefferies |year=2009 }}
* {{cite web |url=http://www.crytek.com/sites/default/files/A_bit_more_deferred_-_CryEngine3.ppt |title="A bit more Deferred" - CryEngine 3 (PPT) |author=Martin Mittring }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html |title=Deferred Shading in S.T.A.L.K.E.R. |author=Oles Shishkovtsov }}
* {{cite web |url=http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html |title=Deferred Shading in Tabula Rasa |author=Rusty Koonce }}
* {{cite web |url=http://aras-p.info/texts/CompactNormalStorage.html |title=Compact Normal Storage for small g-buffers |author=Aras Pranckevičius |date=4 August 2009 |accessdate=12 April 2012 }}
* {{cite web |url=http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/ |title=Graphics Tech: Shadow Maps (part 1) |author=Jonathan Blow |date=3 March 2010 |accessdate=13 April 2012 }}
}}

[[Category:Shader development]]
[[Category:Core development projects]]