Status: August 2010
The current weather system of Flightgear can be called global when Flightgear runs offline: Any menu weather setting, be it windspeed, visibility, precipitation or cloud coverage, affects the weather everywhere. It is not possible to fly to a region where the cloud cover is different, or see precipitation from an aircraft flying in sunshine, or to observe a change in atmospheric pressure by flying somewhere else.
When online, the weather system offers a link with METAR servers, which makes the weather locally-determined global: One can change the weather by flying to a different location where a different METAR is available, but the change happens instantaneously everywhere, i.e. there is no transition of for example crossing a rain front or leaving dense clouds behind to emerge in sunshine.
A local weather system in contrast is one in which continuous changes from the weather at one location to the weather in a different location are simulated, i.e. weather phenomena are tied to a specific location. Needless to say, a local weather system is significantly more complicated to simulate than the present system. Below is a conceptual outline how a local weather system for Flightgear could be created using mainly existing technology and code.
- 1 Scales for weather phenomena
- 2 Connection with the METAR system
- 3 Related Discussions
- 4 Connection with the Multiplayer system
- 5 Development
- 6 Feature requests on the C++ side
- 7 Current development status
- 8 Download
Scales for weather phenomena
Real weather phenomena are determined by processes occurring at vastly different size scales. At the large end of the scale is the system of low and high pressure regions and fronts between cold and warm air masses, which have spatial size scales of O(1000) km. At the other end of the scale are very localized phenomena, for instance a dark tarmac surface heated by sunlight acting as the source point of a thermal updraft resulting in a Cumulus cloud. Both size scales are relevant for the Flightgear experience - an airliner on a longhaul flight is able to observe large-scale weather fronts from above, whereas a glider pilot needs a reasonable simulation of local updrafts and sinks tied to terrain features in a credible way for a realistic flight experience. A local weather system therefore needs to
- simulate weather phenomena at different scales
- ideally work offline as well as online
- allow modification by the user on various scales
- be reasonably fast to run in real-time
The actual dynamics of the atmosphere is a very complex problem which even solved approximately needs hours of CPU time on supercomputing clusters. From this, it is clear that a local weather system cannot be an (even approximately) physically accurate description of processes in the atmosphere, but needs to be a credible mockup of such a description in which heuristic rules replace simulation.
To give an example for what this means, consider the development of Cumulus clouds. Cumulus development is reduced on a hot day under a hazy Cirrus sky. The reason is that the Cirrus sky reflects part of the sunlight, thus the ground is not heated that much by direct sunlight and the convective processes responsible for Cumulus cloud formation are lessened. An accurate simulation would need to start with the energy absorbed from the sunlight on various types of surfaces, then solve the equations governing the heat flux from the ground to the air above, followed by fluid-dynamical equations for the convective currents creating Cumulus clouds. A heuristic rule would simply state that the probability for the placement of a Cumulus cloud into the Flightgear environment is reduced by 50% when a Cirrus cover is present (obviously, the sophistication of the ruleset determines how realistic the weather system will appear).
In the following, the technology for such a local weather system needed at the relevant scales is outlined - first for the situation offline, followed by ideas how it could be connected with the METAR system in online use.
Small scale - effect volumes and average conditions
To simulate local weather at small scale, effect volumes (EV) are a useful concept. The effect volumes define regions in which the normal weather conditions (wind, turbulence, updraft, visibility, precipitation...) are replaced by ones characteristic for the region. As an example, one would define the inside of a cloud as an effect volume in which visibility is reduced as compared to outside, or the precipitation layer underneath a thunderstorm as an EV in which heavy rain is on, visibility is reduced, the temperature is reduced and some turbulence may be active.
The conditions characteristic for the EV are imposed once the aircraft enters the volume, they are restored to the outside values once the aircraft leaves the region again. The Flightgear AI system already has examples of structures which in essence are EVs - thermal and thunderstorm. The first structure contains an EV in which vertical air motion is active, the second one in which turbulence is present. These structures could be generalized to an AI system weather in which essentially all weather parameters inside the volume could be set by XML tags.
On the visual side, parts of the EV must also be tied to visible models. The inside of a cloud can be modelled by reduced visibility, but the outside of the cloud must be a model. Similarly, the inside of a rain front appears as rain generated by the Flightgear system, but the outside must be a 3-d model of a precipitation layer. This means that individual 3-d models for various visible atmospheric phenomena (clouds, precipitation, fog, ...) need to be created (see Howto: Modelling Clouds for a collection of techniques for creating cloud and precipitation models).
Outside the EV, many meteorological parameters may vary in a continuous way, for example visibility may decrease from 5000 to 4000 m when flying 20 km north - but inside the cloud (the EV), it will change in a discontinuous way very suddenly to a low value (say 30 m), to jump back to a large value outside the cloud. Thus, the EVs are used to simulate only discontinuous, local changes in conditions, the larger scale changes in the background need to be taken care of by a different system.
EV's should be implemented such that they can be user-specified, i.e. a user should be able to place any particular combination of weather effects and cloud/precipitation model to a specific location.
Mid scale - weather tiles
Weather needs to be explicitly simulated about as far as one can see from an aircraft - that naturally leads to the concept of weather tiles (analoguous to scenery tiles) which would cover a region of, say, 30x30 km or something of that size. Inside tiles, weather is simulated explicitly in terms of EVs and 3-d models.
As experiments will quickly convince anyone, placing a random set of cloud models into a tile does not create a realistic sky appearance. Instead, some cloud types usually occur together, others do not. The underlying reason is of course that there is large scale dynamics of the weather which determines what happens in a given location. For instance, Cirrus clouds are found at the leading edge of a warmfront - and they are usually followed by lower, layered cloud types like Altostratus, but typically not by a thunderstorm front.
Thus, there need to be different 'types' of tiles, each representing a typical snapshot of a development inside the larger system. Tile themes would be something like 'leading edge of warmfront', 'trailing edge of warmfront', 'cold front', 'dry high-pressure region', 'developed tropical thunderstorms' and so on. This has the added benefit that the user can specify the weather locally simply by chosing an appropriate tile from a menu (he does not need to micromanage the weather by placing individual EV and cloud models).
The type of tile then determines the rules for populating the tile with EV and models. A 'leading edge of a warmfront' tile could for example have the rules
- populate the northern part of the tile randomly with 10 Cirrus clouds between 25.000 and 30.000 ft
- make sure the models are spaced sufficiently far apart
- make sure the models show the same type of texture (do not mix feathery Cirrus textures with amorphous ones)
- populate the southern part of the tile with 20 Cirrocumulus clouds between 20.000 and 25.000 ft
The values of weather parameters outside the EVs would be set for the tile center and interpolated between neighbouring tiles.
Technically, each tile can be populated by a Nasal script specific for the tile type whenever the tile is needed (probably when it becomes visible). As an example for the development state of the system (March 2010), here is the view for a weather tile representing an approaching cold front with a Stratocumulus layer and occasional Cumulonimbus activity.
Large scale - tile placement rules
To create a realistic experience for flights beyond a tile, there need to be matching rules for tiles. Since each tile represents a particular development in a larger weather environment, only certain types of tiles match. For instance, one cannot place a 'high-pressure region' tile next to a 'low pressure region' tile without crossing a front. With a moderately large library of different tile types and a set of placement rules, out of which (in offline mode) a credible set of neighbouring tiles is chosen, a more or less realistic long-distance flight experience can result.
If each tile type contains default values for weather parameters (to be overwritten by METAR info if available), the actual value of any parameter such as atmospheric pressure of visibility can be obtained by interpolation between tile centers, and the parameters will vary even in the absence of METAR info simpy due to the tile placement rules mocking up the presence of real large-scale weather systems.
Tile placement rules can be implemented in a Nasal loop which monitors the local position and initializes a new tile as soon as it is needed.
Connection with the METAR system
Connecting the above concepts with real METAR info is not straightforward. The METAR info should conceptually influence three different things:
- selection of tile type
- placement of EVs and models inside the tile
- continuously varying weather parameters
Selection of tile type
If METAR info is to determine tile type selection, one would need to find an algorithm which gets the METAR from nearby stations and tries to extract a more global weather pattern from that info. For instances, pressure differences and wind direction could be used to locate high and low pressure regions, temperature differences (corrected for elevation) could help to find cold and warm air masses, and based on the most likely weather pattern given these bits of information, the initial tile type as well as the most likely matching neighbouring tiles could be chosen.
Placement of EVs and models inside the tile
At least some aspects of EV and model placement, such as cloud base, precipitation strength or number of clouds to be placed can more or less be inferred directly from the METAR information. However, in general the tile rules may in details conflict with the METAR info, so the weather system outlined here would not always literally reproduce the METAR (for example, it could happen that the METAR reports rain at an airfield, but that the tile generation does not place a raincloud directly above the airfield).
METAR and continuously varying weather parameters
If METAR info is available, this (rather than default tile center info) should be used to set parameters like visibility, pressure or temperature. To avoid discontinuous jumps, the info should be linearly interpolated in time for each nearby position from which a METAR is available. If METAR is updated every 10 minutes, one can use the information at startup for each position for the first 10 minutes of flight, then as soon as the next METAR is available slowly change the value (at each position) within the next 10 minutes to the values just fetched, and continue to do so (in essence creating a 10 minute offset from real to simulated weather).
To interpolate in space, in principle a simple algorithm with inverse distance weighting can be used (there are more accurate but also more complicated solutions): If, say, the pressure p is known at n positions, such that the pressure at position i is p_i and the distance between aircraft and i is d_i, then a good approximation of the local pressure is
p = (sum p_i (1/d_i)) / (sum 1/d_i)
as this will make the pressure at position i equal to p_i and lead to the average of the individual pressures if the distance to all METAR stations is the same.
- Weather algorithms (06/2010)
- A smart cloud altitude algorithm (05/2010)
- Weather dynamics (04/2010)
- New thunderstorm AI scenario alpha release (02/2010)
- Cirrus sky (download link in first post) (02/2010)
- A local weather system (03/2010)
Connection with the Multiplayer system
(should be written by someone who knows what the Multiplayer system can and cannot do)
- Some users have expressed having problems when using the reset/new location facility, the local weather system should probably register listeners to /sim/signals/* and suspend/reset itself accordingly, so that all operations are gracefully interrupted? This may include releasing registered times and listeners, as well as terminating worker threads (once being used) 
We need to add an option to completely disable the local weather system, so that it doesn't run at all, or even better so that it doesn't even get loaded (this is possible by explicit rather than implicit loading using the io.nas module, see next item). This should make it much easier to do troubleshooting, and it would be easier for users to tell if their problems are related to the local weather system or not. 
We should seriously think about moving the local weather system to its own directory, either as a sub directory within $FG_ROOT/Nasal, or possibly even directly in $FG_ROOT/LocalWeather ? The local weather system is becoming one of the most complex systems ever implemented in Nasal, and so it is only good and natural to modularize it even further to keep everything organized (i.e. split code into distinct modules), however the $FG_ROOT/Nasal directory is getting increasingly cluttered due to this ((I am getting the impression that if we had an active Nasal maintainer, that person would rightly complain about this ...because the local weather stuff is not really a part of the "standard library" but instead a completely new system)). It would be better to move everything to a dedicated directory, and only leave a small wrapper script in $FG_ROOT/Nasal that handles loading/unloading of complete systems, loading the corresponding modules explicitly (rather than implicitly due to being located in the $FG_ROOT/Nasal directory) using the helpers available in the io.nas module. The neat thing about using this approach is that it would also be trivially possible to enable/disable the complete local weather system just by setting a property during startup. In addition, it would help keeping things organized. This is especially because the $FG_ROOT/Nasal directory is -by convention- considered to be the "library" directory, but addons like the local weather system (or the bombable addon) are complete systems, which would better live in separate places.(implemented by ThorstenB)
- Dynamically examining the FG/Nasal APIs by running code that may cause exceptions, may also cause errors printed to the console, it would probably be a good idea to expose logging related variables to the property tree, so that the log system can be optionally muted in such cases, i.e. by introducing a "NONE" or "MUTE" logging priority .
- The feature-checking approach used by "compat_layer.nas" should probably be generalized and made available as a standard Nasal module, also for use by other scripts, as it provides a good method for handling backward compatibility 
- we could introduce another hash/namespace in the compat_layer module for Nasal based workarounds that are directly performance critical, so that the C++ implementation is prioritized.
- for some uses, threading may provide a real performance benefit. However, we need to ensure that threading is always OPTIONAL, so that users can easily verify if potential problems are related to threading or not. Worker threads seem to be the easiest and safest way for doing that.
- excessive use of the property tree for storing all sort of state variables is a potential performance bottleneck, it is usually better to directly use Nasal data structures. In addition, it is much easier to make use of worker threads when all data is available in Nasal space, especially given that the property tree (and basically all Nasal extension functions) are not designed to be thread safe.
- to be able to further optimize the system, it might be good to integrate some form of simple benchmark so that users can easily provide the results  (we talked about this in 9/2011, there are possibilities to provide this info automatically, but these would require modifying the core Nasal interpreter, and probably introducing some new instrumentation-specific opcodes, too - that doesn't seem to be a very popular idea at the moment, though).
Feature requests on the C++ side
Intended as a base for discussions on how to implement particular features in the C++ code. Also see Weather algorithms discussion from 06/2010.
As of version 0.81, the local weather system runs on a high-end system, but has problems on slower machines and could use performance boosts from implementing some features which currently are done from Nasal in C++. Guiding principles for that should be performance, accessibility, a clear structure and backward compatibility. The decision if a particular function should be implemented on the C++ level should be investigated with these in mind - usually performance is served better from C++, but Nasal structures remain more accessible. For example, cloud configurations (assembling a vector of coordinates where to place clouds) can usually be done in Nasal in a single frame and thus the performance boost when porting to C++ is marginal. On the other hand, cloud configurations are crated by functions which turn a set of parameters into cloud positions, and it is useful to have quick access to the parameters and functions to tune the system to reproduce a real sky better and better without the need to recompile the code. Thus, cloud configuration computations are an example for structures which should probably not be ported to C++.
Based on an idea by Hooray, version 0.81 of the local weather package has low-level function calls gathered in a separate Nasal file compat_layer.nas. The idea is that C++ counterparts for these are supplied, along with Nasal structures which check if the Flightgear core has the functionality and call the C++ function if the functionality is there while they use current Nasal implementations if no C++ structures are available, thus ensuring backward compatibility to Flightgear 2.0.0. Anyone interested in porting a structure to C++ could then start to implement one of the functions found there. These functions are:
- To check for wave conditions far from the original obstacle doesn't work well in Flightgear, because you can't rely on terrain far from your present location being loaded already. So by necessity one has to make it a more local phenomenon.
- Once I cross the mountaintops, some system switches on turbulence (I suspect ridge lift, since I could not find any other property which would create turbulence). That's bad, because wave lift isn't turbulent at all.
- calls to set the environment (temperature, visibility, turbulence, pressure, dewpoint, thermal lift). It would seem a good idea if the environment subsystem could be structured in such a way as to accept direct input from Nasal via the property tree (and switches its own interpolation routines off) if a particular property is set.
Note: This is currently work in progress. --T3r 11:17, 3 August 2010 (UTC)
- interactions with the terrain - chiefly samling terrain elevation for large vectors of coordinates
Note: This is currently work in progress. --T3r 11:17, 3 August 2010 (UTC)
Another note: There is already a proposal for a related patch  which needs to be reviewed/reworked and hopefully committed at some point. Also, we have talked about other ways for speeding up the whole thing even more: --Hooray 12:09, 3 August 2010 (UTC)
- calls to place cloud models into the scenery, to remove them and to change their position. Compared with the standard Flightgear 3d clouds, placing and moving models from Nasal seems exceedingly slow. However, local weather uses some features for which it is not obvious if they are supported by the way the standard 3d clouds are implemented. These are:
- a 'cloudlet' is not a single texture, but a complete *.ac model. This allows to make multi-layered cloudlets and opens the possibility to make sure that one type of cloud texture is always seen in front of another (for example, a diffuse haze could always be before a well-structured cloud top, such that the top seems to emerge from the haze). This is a *very* useful feature for designing clouds, and is also non-trivial to get via explicit placement rules.
- clouds need to be identified by various means. For movement, the system needs to know all clouds in the visual field, for deletion all clouds with given tile index, for evolution (not implemented yet) all cloudlets which belong to a particular Cumulus cloud. Internally, this is done by storing pointers to the cloud property node in various arrays, such that simply using the right array ensures that the correct cloud is referenced for an operation. Any C++ implementation should preferably allow similar referencing by pointer from Nasal or provide equivalent functionality to replace what the pointer array is for.
For reference, the main performance bottlenecks in v0.81 seem to be:
- rotating cloud models towards the viewer. This scales with the number of vertices, and is currently done by the shader. Performance is needed whenever clouds are in the field of view. Probably performance here can not be improved significantly and this is an upper limit for the number of distinct clouds which can be in the field of view - for denser cloud population, larger textures containing a group of clouds can be used.
- moving clouds with the wind. This scales with the number of cloudlets in the field of view and is done by Nasal. performance is needed whenever dynamical weather is on and clouds are in the field of view. Currently, the Nasal code limits the max. number of clouds in the loop to avoid a dramatic drop in framerate, which means that not all clouds in the visual field may be processed.
- generating a cloud configuration. This is only a siginficant issue for convective clouds, where several thousand terrain coverage calls need to be done. More performance is needed in more detailed terrain (custom France for example). Performance is only needed when a weather tile is generated, so unlike in the previous cases, performance loss occurs over a finite time interval, after which things get back to normal
- sampling terrain elevation (as above, just no terrain coverage, only elevation data is needed)
- placing clouds into the scenery. Performance is reduced while the flightgear core lets clouds appear in the scenery
- deleting clouds. A brief drop in framerate (usually less than a second) occurs when a large amount of information is deleted from the property tree.
New Nasal API requests
For things that should be implemented as new Nasal APIs, rather than being hard coded features in C++ The file $FG_ROOT/Nasal/compat_layer.nas contains compatibility wrappers for functions that are currently implemented in Nasal space, because the corresponding features are not yet supported by the core fgfs/Nasal APIs. This may be a good starting point for implementing new Nasal APIs.
Current development status
The current version 0.7 supports placement of effect volumes for visibility, rain, snow, turbulence and thermal lift, as well as interpolation of visibility, pressure, temperature and dewpoint between weather stations. It has a variety of 40x40 km weather tiles with layered and non-layered clouds and weather effects, thunderstorms, external models of precipitation and (with a Git patch) also support for gliders, i.e. it creates thermals below convective clouds. Automatic weather tile reloading for long-range flights is implemented, both in a simple 'repeat-tile' function and a set of tile selection rules to simulate the long-range change of real weather. 3d clouds are rendered up to 45 km distance. Terrain presampling algorithms automatically adjust the cloud placement over elevated terrain.
A feature gallery of version 0.85:
A feature gallery of version 1.0: