Scripted Compilation on Linux Debian/Ubuntu

From FlightGear wiki
Revision as of 14:44, 30 April 2019 by Rominet (talk | contribs) (Streamline general introduction and add an Introduction section as recommended by the wiki's Manual of Style)
Jump to navigation Jump to search

download_and_compile.sh is a Bash script that takes care of downloading and compiling FlightGear and related software from their source code repositories with just one command execution[1] for both 32-bit and 64-bit Debian-based systems. Pre-existing versions (if any) of the software installed by download_and_compile.sh are not touched at all since the script downloads, builds and installs everything under the directory in which it is launched. You can choose the particular components to download, build and install.

Unless told not to do so, download_and_compile.sh installs packages with apt-get. For this reason, it is primarily useful on Debian-based distributions. However, if one disables package installation (using -pn or --sudo=echo) and installs the corresponding dependencies oneself, it might be useful on other distributions as well.

For hints on using rpm-based distributions such as Redhat, Fedora and CentOS, please see CentOS. Please also see Superbuild.

Introduction

download_and_compile.sh is a Bash script written for Debian-derived distributions (Ubuntu, Devuan, Linux Mint, etc.). Its purpose is to automatically install dependencies using the package manager, then build and install FlightGear-related programs.

By default, download_and_compile.sh installs most dependencies with apt-get run under sudo.[2] Other dependencies, either because they aren't available in the standard APT repositories, or because of non-option arguments passed to download_and_compile.sh, are downloaded and compiled on the fly (this can be the case for PLIB, Simgear and OpenSceneGraph, for instance—it all depends on the arguments passed to the script).

download_and_compile.sh works in the directory it is run from: apart from dependencies installed via the package manager, all programs built by download_and_compile.sh are installed under the install subdirectory of the directory from which the script was run. In other words, installation of programs by download_and_compile.sh is clean, very easy to undo and doesn't interfere with other programs on the system.

It is possible to manage several directory trees with download_and_compile.sh; as far as it is concerned, such directory trees are completely independent from each other. For instance, if you run download_and_compile.sh in dir1 and dir2, the programs installed under dir1 won't “see” those installed under dir2, and vice versa.

Apart from its main purpose, download_and_compile.sh can be used to find hopefully up-to-date build-dependency information for FlightGear and related software. You would do so by inspecting the script at the point where it installs packages.[3]

Update Available

The latest version of download_and_compile.sh can be obtained here, however there are advantages getting it from an FGMeta clone as explained here. Contents should be moved from there to this page; this and the following section are not up-to-date at the time of this writing.

Cut to the Chase: for the impatient

Beware: with the following, you are likely to encounter problems when cloning FGData: read just above.

cd  <your working directory for building FlightGear>
wget -O download_and_compile.sh https://sourceforge.net/p/flightgear/fgmeta/ci/next/tree/download_and_compile.sh?format=raw
chmod +x download_and_compile.sh
mkdir -p stable
mkdir -p next
cd stable
../download_and_compile.sh -j$(nproc) -s
cd ../next
../download_and_compile.sh -j$(nproc) -p n
Note  With -j$(nproc) as above, the compilation will use all cores available on your processor, which can save several hours. If you want to use, say, 4 cores, replace -j$(nproc) with -j4.

Conversion of directory structure from earlier versions of download_and_compile.sh

Earlier versions of the script used a different directory structure. If you used the earlier version, the new script includes a section that will convert the earlier structure to the current set of directories.

Disk Space Requirements

As of April 2019, building FlightGear requires about 12 GiB of disk space. Note that this includes downloaded source code for SimGear and FlightGear, generated build files and the large FGData repository (about 6 GiB for that one).

List of compiled programs

The download_and_compile.sh script is able to download and compile:

(Note that OpenRTI is just an optional dependency for HLA support. For the time being, you should be just fine building without it. Eventually, the idea is for HLA to replace the existing MP system and even increasingly distribute the FlightGear architecture such that more and more components can be more easily run in separate threads or even separate processes, possibly even on different machines. So this is going to be an important feature for professional users, using several computers and screens to create a comprehensive and immersive simulation environment.

At the moment, it's probably safe to say that HLA is only of interest to developers and people willing to play with experimental features.)

Each of the items listed above corresponds to a component in download_and_compile.sh terminology. Components are written in uppercase. The list of supported ones is currently:

  • SIMGEAR
  • FGFS (this corresponds to FlightGear)
  • DATA (this corresponds to FGData)
  • CMAKE
  • PLIB
  • OSG (this corresponds to OpenSceneGraph)
  • FGRUN
  • FGO
  • FGX
  • TERRAGEAR
  • TERRAGEARGUI
  • OPENRADAR
  • ATCPIE
  • OPENRTI

The up-to-date list of components can be obtained by running:

download_and_compile.sh --help

What is the point of knowing this? Because you may pass component names to download_and_compile.sh in order to tell it what you want to download, build and install. By default, only the three essential components SIMGEAR, FGFS and DATA are taken care of, which means that the command:

download_and_compile.sh

is equivalent to:

download_and_compile.sh SIMGEAR FGFS DATA

In case you want to do the same build with just OpenSceneGraph added, you can use:

download_and_compile.sh SIMGEAR FGFS DATA OSG

You get the idea. When several components are passed on the same command line, download_and_compile.sh chooses a reasonable order for processing, so don't worry about that.

When building Next you may see build errors

Keeping in mind that this script compiles sometimes bleeding edge software, it can happen that what was successfully compiling last week, does not compile anymore today. Building the stable version should always work, unless there is a problem with the script.

That said, don't be too afraid of building the development version (called next): this is the one developers use all the time, so kindly asking on the flightgear-devel mailing list in case a problem popped up[4] should allow you to find good advice and get the problem quickly fixed, if it's a new one. Conversely, probably not many people (in 2019) build themselves the “stable” version of FlightGear. People who want the stable version can usually have it from their distribution, this is generally easier. On the other hand, those running download_and_compile.sh typically want to build the latest code that FlightGear developers are working on; this is useful when you want to contribute feedback, code, aircraft or scenery based on recent technology, or just want to enjoy the latest features and bugs. ;-)

Download

You can get download_and_compile.sh from FGMeta. It is contained in the FGMeta repository, which is maintained by the FlightGear developers. Remember to update this script whenever a new FlightGear version is released, so that you'll be able to download the latest stable revision (of course, you can update it more often to benefit from bug fixes, it doesn't have to happen after a release).

Instructions

To run download_and_compile.sh, just save it in a directory called for example: ~/fgfs then execute it (no need to execute it as root).

Here is for example a sequence of commands to get the script from the next branch:

mkdir ~/fgfs
cd ~/fgfs
wget https://sourceforge.net/p/flightgear/fgmeta/ci/next/tree/download_and_compile.sh?format=raw
chmod 755 download_and_compile.sh

You have two options now: build the latest stable FlightGear release or build the current development version (bleeding edge). After building stable or the latest, if you need to build a different version, try the instructions for Avoiding multiple downloads of FGData on Linux (warning: as of April 2019, they are outdated).

Build the latest stable FlightGear release

When executing the script, use the "-s" option to build the latest stable release:

./download_and_compile.sh -s

Build the current FlightGear development version

When executing the script without any options, the latest development version is built.

Warning: The development version of FlightGear changes on an almost daily basis. It provides the latest features, but is not always guaranteed to work reliably. If you're unfamiliar with software testing, you may prefer to use the latest stable release.

./download_and_compile.sh

Once the script has finished running, you will successfully get FlightGear installed in the ~/fgfs directory.

Launching FlightGear

When using download_and_compile.sh, apart from those installed with the package manager, the FlightGear dependencies (which are typically libraries) are not installed system-wide but under the directory from which download_and_compile.sh is run. This makes it possible to easily use, for instance, different OpenSceneGraph, SimGear and FlightGear versions on a single system—e.g., for testing purposes—but also to have separate build trees (optimized/debug). This is also why you either need to set LD_LIBRARY_PATH to run the built programs, or simply use the scripts created by download_and_compile.sh in the directory where it is run, such as run_fgfs.sh and run_fgfs_debug.sh: these scripts automatically set up the required environment variables according to your build settings before firing the desired program (e.g., fgfs) with the arguments you provided.

Therefore, the simplest way to run your newly-built FlightGear is to launch the run_fgfs.sh script that download_and_compile.sh created in the directory from which it was run, for example:

cd ~/fgfs
./run_fgfs.sh --launcher
Note./run_fgfs.sh --launcher starts FlightGear with its built-in launcher. If you just do ./run_fgfs.sh, FlightGear will be started without any launcher, at the default airport and with the default aircraft.

Launching FGRun

Note  As of 2019, FGRun is not maintained anymore. You may want to try the built-in launcher (see above) or FFGo.
Fgrun-page2.jpg

For many users it's more comfortable having FlightGear launched by the graphical utility FGRun which is installed as well in the same folder. You have to launch the run_fgrun.sh command, for example:

cd ~/fgfs
sh run_fgrun.sh

FGRun will save its settings in ~/.fltk/flightgear.org/fgrun.prefs you may want to save copies of the preferences customized for stable and next.

Launching FGo!

Note  As of 2019, FGo! is not maintained anymore. You may want to try the built-in launcher (see above) or FFGo.
Fgo01.jpg

This is a graphical utility written in python, You have to launch the run_fgo.sh command, for example:

cd ~/fgfs
sh run_fgo.sh

Remember that the first time you run it, you have to go to preferences and set the binary and FGData path (Do no set the working directory, you don't need it).

Additional programs

If you wish to get other programs, you need to launch the script with the desired component names as arguments. For instance:

./download_and_compile.sh SIMGEAR FGFS DATA OSG

The available components are (in April 2019):
CMAKE, PLIB, OPENRTI, OSG, SIMGEAR, FGFS, DATA, FGRUN, FGO, FGX, OPENRADAR, ATCPIE, TERRAGEAR, TERRAGEARGUI.

The full, up-to-date list of these components can always be obtained by running:

./download_and_compile.sh --help

Launching FGCOM

NoteFGCom has been integrated into FlightGear long ago, therefore the following is not needed in general.

FGCom is the system used by FlightGear to simulate radio communications between users. Launch it using the run_fgcom.sh command:

cd ~/fgfs
sh run_fgcom.sh

Troubleshooting

Compilation errors

Here we are, no fear, if you wish to use programs from the cvs/svn/git repositories, you might face compilation errors that will prevent you to have a working copy of one or more of the programs provided by this script. What can be the causes that prevent us from successfully compiling? As far as I know those:

  1. Software developers introduce a new functionality with a new piece of code that prevents the compilation under your architecture, this can happen working with cvs/svn/git sources.
  2. The program refuses to compile because of a divergence in the libraries on which it depends. For example FlightGear might not compile because OSG has been modified, while OSG itself compiles fine, FG won't.
  3. One or more repositories are down and you can't get the library you need. (Both from cvs/svn/git or apt-get)

There is a simple solution to the above errors: wait and relaunch the script after some time (hours or days), if software developers repair or synchronize their code with the newly updated libraries (which generally happens eventually), your FlightGear will compile fine as if the previous error never took place.

Sometimes it happens that the script fails to compile only FGRun, FGCOM or atlas, if you then see the run_fgfs.sh file it means that FlightGear installation was successful and you can safely run it.

OpenRTI undefined reference errors

Sometimes due to the way d&c build cleans up projects, linking errors might occur, this is the case with this error "libRTI-NG.so: undefined reference to xxx", sadly at this point either you need to patch the d&c script to clean OpenRTI with rm -f CMakeCache.txt && rm -rf CMakeFiles/, or just start from a clean environment, assuming you are in the parent project directory, you will need to issue this command to clean everything(except FGDATA):

rm -rf build/* install/simgear/ install/openrti/ install/flightgear/share/ install/flightgear/bin/

see this thread for more details (https://forum.flightgear.org/viewtopic.php?f=45&t=26244)

Options

By default, download_and_compile.sh downloads or updates, then compiles, SimGear and FlightGear, and downloads or updates FGData (by nature, FGData can't be compiled). This is what happens when running:

./download_and_compile.sh

To make it download or update and compile something else, you may pass non-option arguments naming the components you want, for instance:

./download_and_compile.sh SIMGEAR FGFS DATA OSG

SIMGEAR, FGFS, DATA and OSG are the component names respectively corresponding to SimGear, FlightGear, FGData and OpenSceneGraph in download_and_compile.sh's terminology.

See the output of download_and_compile.sh --help for more details.

Compiling only one program

Note  The following is not necessarily up-to-date. See the output of download_and_compile.sh --help for the up-to-date list of components.

If you wish to recompile only one of the programs you can launch the script with one of the following parameters:

  • CMAKE (to compile and install only cmake)
  • PLIB (to compile and install only plib)
  • OSG (to compile and install only OpenSceneGraph)
  • SIMGEAR (to compile and install only Simgear)
  • FGFS (to compile and install only FlightGear)
  • DATA (to download / update only data files for FlightGear)
  • FGRUN (to compile and install only Fgrun)
  • TERRAGEAR (to compile and install only terragear!)
  • TERRAGEARGUI (to compile and install only terrageargui!)
  • OPENRADAR (to compile and install only OpenRadar!)
  • FGO (to compile and install only Fgo!)

Compiling the last known-stable versions

Even if download_and_compile.sh fetches code and data from development branches of the source repositories by default (which sometimes causes compilation errors), it is possible to tell the script to download the latest known versions of the software that were compiling successfully by means of the -s option:

./download_and_compile.sh -s

How does it work?

  • For SimGear, FlightGear and FGData, it uses the most recent stable release branch of the corresponding Git repository.
  • For some components, a known-stable version is hardcoded in download_and_compile.sh and used when the -s option is given[5] (for instance, the hardcoded information can be the name of a Git branch).
  • For the other components, the -s option has no effect.
Warning  In a given folder where download_and_compile.sh is run, you should either always use the -s option, or never. Building some components with -s and others without (all within the same base folder) is not supported.

Advanced options

  • Build a release version: -b Release
  • Build a version that should run as fast as a release build, yet has debug information that can be used to post backtraces: -b RelWithDebInfo (this is the default)
  • Build a full debug version for very complete bug reporting: -b Debug
  • Skip download of distro packages (i.e., by default: apt-get install ...): -p n
  • Skip retrieving of software updates (i.e., by default: apt-get update): -d n
  • Skip the configure step (like running CMake or autoconf's ./configure): -r n
  • Skip compilation of programs: -c n.

For example, if you are a developer and wish to quickly recompile and reinstall only your own modifications for FlightGear, you can do this:

./download_and_compile.sh -p n -d n -r n FGFS

Note that this is the same as:

./download_and_compile.sh -pn -dn -rn FGFS

This command will only rebuild modified files and reinstall FlightGear. Note that depending on the kind of changes you made, reconfiguring and thus dropping the -d n option may be necessary, though (this is the case in particular if you added or removed C++ files).

Multicore Acceleration

Using the option -j x (where x is the number of your CPU-Cores you wish to assign to the job) will speed up the whole compilation process considerably.

Disk usage and build time

Keeping compiled programs, source code and data repositories requires some hard disk space: it will take something like 12 GiB of space for just SimGear, FlightGear and FGData (i.e., the SIMGEAR, FGFS and DATA components).

With an Intel Core i7 860 CPU (2.80 GHz) purchased in 2009, compiling SimGear and FlightGear 2019.2 with option -j8 takes about 14 minutes. If you don't have a fast machine and build using only one core, it may require several hours.

Optimus technology

If your computer has a GPU with Optimus technology, you need a dedicated script in order to make FlightGear run with the powerful GPU.

After having installed required tools (Bumblebee) you just need to run this command line in your FlightGear installation directory (where you executed download_and_compile.sh):

sed  's|\./fgfs|optirun ./fgfs|' run_fgfs.sh > run_fgfs_optirun.sh && chmod +x run_fgfs_optirun.sh

Now you can run FlightGear with ./run_fgfs_optirun.sh.

The same can be done for the FGRun launcher:

sed  's|\./fgrun|optirun ./fgrun|' run_fgrun.sh > run_fgrun_optirun.sh && chmod +x run_fgrun_optirun.sh

Remove warning message for DDS files

You can remove the warning message displayed when DDS files are parsed by SimGear by adding the following line just after cd "simgear" line :

	sed -i s/SG_ALERT,\ \"Image/SG_WARN,\ \"Image/g simgear/scene/model/ModelRegistry.cxx

See also

  • Other FlightGear build scripts can be found in FGMeta.
  • Another script for building FlightGear and all its dependencies in an automated fashion can be found here. The page seems a bit oldish, though (as of 2019).

References

  1. Due to technical problems on the SourceForge side, this is currently only true once you have an FGData clone. See here for details.
  2. If you think you already have the dependencies, this installation can be disabled either by using option -pn or by passing option --sudo=echo (the latter results in printing the apt-get command line without running it).
  3. Look for strings such as zlib1g-dev, libglew-dev or qt5-default.
  4. Don't forget in this case to precisely tell what you did and include the compilation_log.txt file written by download_and_compile.sh.
  5. As of April 2019, apart from SIMGEAR, FGFS and DATA, the only component for which download_and_compile.sh “knows” a stable version is OPENRTI.