Scripted Compilation on Linux Debian/Ubuntu: Difference between revisions

From FlightGear wiki
Jump to navigation Jump to search
(Remove section “Remove warning message for DDS files”: the given advice was a blunt batch-replacing of 'SG_ALERT, "Image' with 'SG_WARN, "Image', however such SG_ALERT logs don't exist anymore in the indicated file since 2014)
(Simpler method for changing the protocol or URL of a Git remote (used for FGData here))
Line 264: Line 264:
There we are! You now have a clean, up-to-date [[FGData]] clone in <tt>~/flightgear/dnc-managed/install/flightgear/fgdata</tt> (remember: <tt>~/flightgear/dnc-managed</tt> is the directory from which you ran <tt>download_and_compile.sh</tt>). Note this place: the full path of the <tt>~/flightgear/dnc-managed/install/flightgear/fgdata</tt> directory is your [[$FG_ROOT]].
There we are! You now have a clean, up-to-date [[FGData]] clone in <tt>~/flightgear/dnc-managed/install/flightgear/fgdata</tt> (remember: <tt>~/flightgear/dnc-managed</tt> is the directory from which you ran <tt>download_and_compile.sh</tt>). Note this place: the full path of the <tt>~/flightgear/dnc-managed/install/flightgear/fgdata</tt> directory is your [[$FG_ROOT]].


Now open the <tt>[[$FG_ROOT]]/.git/config</tt> file that lives inside your FGData clone (i.e., <tt>~/flightgear/dnc-managed/install/flightgear/fgdata/.git/config</tt> in our example). You should see a paragraph resembling this:
Now, change the protocol to use for future updates of your FGData clone:<ref name="changing-the-protocol-for-a-git-remote-manual-method">Another way would be to manually change the relevant line starting with <code><nowiki>url = ssh://SFusername@</nowiki></code> for the <code>origin</code> remote in the <tt>.git/config</tt> file that lives inside your repository clone (i.e., <tt>~/flightgear/dnc-managed/install/flightgear/fgdata/.git/config</tt> in our example).</ref>
  [remote "origin"]
(cd install/flightgear/fgdata && \
        url = ssh://''SFusername''@git.code.sf.net/p/flightgear/fgdata
  git remote set-url origin <nowiki>https://git.code.sf.net/p/flightgear/fgdata</nowiki>)
        fetch = +refs/heads/*:refs/remotes/origin/*
(you can check at any time the protocol(s) in use with the command <code>git remote -v</code> run inside a Git repository—in this case, inside the folder <tt>install/flightgear/fgdata</tt>). As a consequence of this change, all future updates of your FGData clone will use the <tt>https</tt> protocol, therefore you won't be prompted anymore for your SourceForge password.
Replace <code>ssh://''SFusername''@</code> with <code>https://</code> and save the file. As a consequence of this change, all future updates of your FGData clone will use the <tt>https</tt> protocol, therefore you won't be prompted anymore for your SourceForge password.


All that remains to do is to run, from the same directory as before (<tt>~/flightgear/dnc-managed</tt> in our example):
All that remains to do is to run, from the same directory as before (<tt>~/flightgear/dnc-managed</tt> in our example):

Revision as of 09:08, 15 May 2019

download_and_compile.sh is a Bash script that takes care of downloading and compiling FlightGear and related software from their source code repositories with just one command execution[1] for both 32-bit and 64-bit Debian-based systems. Pre-existing versions (if any) of the software installed by download_and_compile.sh are not touched at all since the script downloads, builds and installs everything under the directory in which it is launched. You can choose the particular components to download, build and install.

Unless told not to do so, download_and_compile.sh installs packages with apt-get. For this reason, it is primarily useful on Debian-based distributions. However, if one disables package installation (using -pn or --sudo=echo) and installs the corresponding dependencies oneself, it might be useful on other distributions as well.

For hints on using rpm-based distributions such as Redhat, Fedora and CentOS, please see CentOS. Please also see Superbuild.

Introduction

download_and_compile.sh is a Bash script written for Debian-derived distributions (Ubuntu, Devuan, Linux Mint, etc.). Its purpose is to automatically install dependencies using the package manager, then build and install FlightGear-related programs.

By default, download_and_compile.sh installs most dependencies with apt-get run under sudo.[2] Other dependencies, either because they aren't available in the standard APT repositories, or because of non-option arguments passed to download_and_compile.sh, are downloaded and compiled on the fly (this can be the case for PLIB, Simgear and OpenSceneGraph, for instance—it all depends on the arguments passed to the script).

download_and_compile.sh works in the directory it is run from: apart from dependencies installed via the package manager, all programs built by download_and_compile.sh are installed under the install subdirectory of the directory from which the script was run. In other words, installation of programs by download_and_compile.sh is clean, very easy to undo and doesn't interfere with other programs on the system.

It is possible to manage several directory trees with download_and_compile.sh; as far as it is concerned, such directory trees are completely independent from each other. For instance, if you run download_and_compile.sh in dir1 and dir2, the programs installed under dir1 won't “see” those installed under dir2, and vice versa.

Apart from its main purpose, download_and_compile.sh can be used to find hopefully up-to-date build-dependency information for FlightGear and related software. You would do so by inspecting the script at the point where it installs packages.[3]

Disk space requirements and build time

As of April 2019, building FlightGear requires about 12 GiB of disk space. Note that this includes downloaded source code for SimGear and FlightGear, generated build files and the large FGData repository (about 6 GiB for that one).

With an Intel Core i7 860 CPU (2.80 GHz) purchased in 2009, compiling SimGear and FlightGear 2019.2 with option -j8 takes about 14 minutes. If you don't have a fast machine and build using only one core, it may require several hours.

Download

You can get download_and_compile.sh from FGMeta. It is contained in the FGMeta repository, which is maintained by the FlightGear developers. The script can be downloaded from the link given above, however, for easier updates and in order to have the command download_and_compile.sh --version work as intended, it is recommended to get it as explained below.

In case you build stable versions of FlightGear using the -s option of download_and_compile.sh, remember to update the script before trying to build a new version of FlightGear (see Updating download_and_compile.sh below). Of course, you can update it more often in order to benefit from new features or bug fixes; this is especially useful if you are building next, that is, the development branch of FlightGear.

Getting started with download_and_compile.sh

Because of technical problems at SourceForge (cloning FGData using the https protocol usually fails), the initial setup is unfortunately more complex than it used to be. You'll be guided step by step though, so don't be afraid. Besides, once you have a complete clone of the FGData repository, all future operations will be really easy.

We'll first explain how to get download_and_compile.sh in a way that makes it convenient to update and causes the command download_and_compile.sh --version to work as intended (the reported “version” is a Git blob id such as 6a5e4f05e2ccf27115eec58313be027b11266097[4]). Then we'll show how to clone the large FGData repository, and finally give instructions to get FlightGear up and running.

Notations

When a command should be run as an unpriviledged user, it will be preceded by a dollar sign:

$ whoami
toto

In contrast, a hash sign (#) means that the command must be run with superuser privileges to achieve the desired effect:

# whoami
root

In order to make instructions easy to understand, two directories (= folders) will be consistently used for the same purpose below:

  • ~/flightgear/fgmeta will contain a clone of the FGMeta repository; therefore, download_and_compile.sh will reside in that directory;
  • ~/flightgear/dnc-managed will be the directory from which we run download_and_compile.sh. In other words, with this setup, a typical sequence of commands could be:
$ cd ~/flightgear/dnc-managed
$ ~/flightgear/fgmeta/download_and_compile.sh SIMGEAR FGFS DATA

These are of course just examples. The aforementioned paths are not hardwired anywhere in the script; you are free to choose the directories you want for these purposes.

Getting download_and_compile.sh the “right way”

There are several ways to obtain download_and_compile.sh. The method described here makes it very easy to update the script and causes the command download_and_compile.sh --version to work as intended.

As explained in Notations, we want to clone the FGMeta repository in ~/flightgear/fgmeta. Let's go:

$ mkdir -p ~/flightgear
$ cd ~/flightgear
$ git clone https://git.code.sf.net/p/flightgear/fgmeta

You now have a fresh FGMeta clone in ~/flightgear/fgmeta and your brand new download_and_compile.sh script is located in that directory. You can already try it to see the available options:

$ ~/flightgear/fgmeta/download_and_compile.sh --help
download_and_compile.sh [OPTION...] [--] [COMPONENT...]
Download and compile components belonging to the FlightGear ecosystem.

Without any COMPONENT listed, or if ALL is specified, recompile all
components listed in the WHATTOBUILDALL variable. Each COMPONENT may
be one of the following words:

  ALL, CMAKE, OSG, PLIB, OPENRTI, SIMGEAR, FGFS, DATA, FGRUN, FGO, FGX,
  OPENRADAR, ATCPIE, TERRAGEAR, TERRAGEARGUI

Available options:
  -h, --help    show this help message and exit
      --version print version and license information, then exit

(...)

Updating download_and_compile.sh

Now that you have download_and_compile.sh from the FGMeta repository, it is very easy to update (this assumes you didn't modify anything yourself inside ~/flightgear/fgmeta!):

$ cd ~/flightgear/fgmeta && git pull

If you want to keep updates as easy as we just shown, it is best not to modify download_and_compile.sh yourself. download_and_compile.sh has plenty of options that usually make it unnecessary to modify the script. Just run download_and_compile.sh --help and learn about the available options when you feel the need to change something. Unless you have special needs that can only be accomodated by modifying download_and_compile.sh, you are invited to skip to the next section.

If you really, really want to modify download_and_compile.sh while keeping updates easy, a good technique is to add your changes to your FGMeta clone in the form of one or more Git commits (no need to push them anywhere, commits can remain in your clone). How to do that is beyond the scope of this document, though; read Git tutorials if you want to learn it (there are plenty on the Internet). Once you have committed your changes to your FGMeta clone, make sure the repository is clean (use git status), then update it with:

$ cd ~/flightgear/fgmeta && git pull --rebase

This will apply your commits on top of the latest commit of the branch that is currently checked out, which so far contained the official version of download_and_compile.sh. In case your changes conflict with the update, Git will tell you and you'll have to resolve the conflict manually (look for “Git resolve conflict” on your favorite search engine)... or start again from a pristine FGMeta clone.

Building FlightGear

For the method described below, you'll need an account at SourceForge (this is unfortunate, but methods that don't require such an account are either unreliable or insecure, as long as FGData can't be cloned using the https protocol). If you don't already have one, go to the registration page and create an account. In all this section, we'll assume that your account name at SourceForge is SFusername.

In what follows, we won't give the full path to download_and_compile.sh when showing commands to be run, but you should prepend it to download_and_compile.sh whenever you see a download_and_compile.sh command. For instance, if you used the same path as in Notations and see the command:

$ download_and_compile.sh --help

what you should actually run is:

$ ~/flightgear/fgmeta/download_and_compile.sh --help

Apart from this harmless command, do not run other download_and_compile.sh commands from an arbitrary directory, in particular don't run them from ~/flightgear/fgmeta. This is because most other download_and_compile.sh commands write to the current directory (download_and_compile.sh --help and download_and_compile.sh --version are safe to run from any directory, though).

Of course, it is always possible to make commands shorter by setting up aliases (see tips at the end of this message), by adding the directory containing download_and_compile.sh to your PATH or by creating a symbolink link pointing to download_and_compile.sh in a directory that is part of your PATH. This is not necessary, though; do it only if you feel the need (when enabled, persistent shell history is often enough for such things).

Note  The following commands should be run from an empty directory[5] in a partition that has enough free space (see Disk space requirements and build time). As explained in Notations, we are going to choose the directory ~/flightgear/dnc-managed for this purpose, in order to express that the whole directory tree is managed by download_and_compile.sh. This is just an example; feel free to choose another directory if you want.

Don't run the commands from a non-dedicated directory, because it will be filled with files and directories created by download_and_compile.sh and the FlightGear, SimGear, etc. build systems. That would be a complete mess! In particular, don't run the commands from the directory containing your FGMeta clone.

Note  As explained in Notations, we assume that your Unix user name (login) is toto. Don't confuse the sudo password prompt (where you need to enter toto's password) with the password prompt for your SourceForge account! The former appears as
[sudo] password for toto:

whereas the latter is just:

Password:
Tip  In case you want to run some other program instead of sudo, this can be done with the --sudo option of download_and_compile.sh. For instance, in order to see the commands that would be run with sudo without actually running them, you can pass --sudo=echo to download_and_compile.sh. Like all other options, --sudo must be given before all arguments that are component names (such as SIMGEAR, FGFS, DATA, etc.).

The package manager used by download_and_compile.sh by default is apt-get. You can use another one if you want, as long as it supports the following calls:

pkg-mgr update
pkg-mgr install pkg1 pkg2 ...

This is the case for aptitude as well as apt. If you want download_and_compile.sh to use aptitude, give it the option --package-manager=aptitude before any of the COMPONENT arguments.

All options of download_and_compile.sh can be seen by running the following command:

$ download_and_compile.sh --help

Now the instructions we promised you. You have chosen a dedicated directory where all the stuff that is downloaded and built by download_and_compile.sh will be stored. This is ~/flightgear/dnc-managed in our example, and should be empty before you run download_and_compile.sh for the first time. However, it is quite correct to start download_and_compile.sh from the same directory for subsequent runs, even when non-empty (otherwise, download_and_compile.sh would automatically reclone the repositories every time you run it; that would be a sheer waste of time and bandwidth).

Ready? Let's go!

$ mkdir -p ~/flightgear/dnc-managed
$ cd ~/flightgear/dnc-managed
$ download_and_compile.sh --git-clone-site-params SourceForge=ssh:SFusername DATA
**********************************************************************
*                                                                    *
* Warning: a typical SimGear + FlightGear + FGData build requires    *
* about 12 GiB of disk space. The compilation part may last from a   *
* few minutes to hours, depending on your computer.                  *
*                                                                    *
* Hint: use the -j option if your CPU has several cores, as in:      *
*                                                                    *
*         download_and_compile.sh -j$(nproc)                         *
*                                                                    *
**********************************************************************
Running 'apt-get update'...
[sudo] password for toto:

(...)

Considering a package alternative: libcurl4-openssl-dev libcurl4-gnutls-dev
Package alternative matched for libcurl4-openssl-dev
Running 'apt-get install build-essential git libcurl4-openssl-dev cmake'...
[sudo] password for toto:

(...)

****************************************
**************** DATA ******************
****************************************
Fetching DATA with 'git clone ssh://SFusername@git.code.sf.net/p/flightgear/fgdata'
Cloning into '.'...
The authenticity of host 'git.code.sf.net (216.105.38.16)' can't be established.
ECDSA key fingerprint is SHA256:FeVkoYYBjuQzb5QVAgm3BkmeN5TTgL2qfmqz9tCPRL4.
Are you sure you want to continue connecting (yes/no)?
Warning: Permanently added 'git.code.sf.net,216.105.38.16' (ECDSA) to the list of known hosts.
Connection closed by 216.105.38.16 port 22
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

The above messages are perfectly normal but deserve a little explanation. Here, ssh asked us to confirm that the fingerprint sent by the remote host is that of the real git.code.sf.net, as opposed to that of some malicious server pretending to be git.code.sf.net. This confirmation only has to be done once, after which it is remembered thanks to ~/.ssh/known_hosts. You should visit the page that gives the host key fingerprint of every publically-accessible SSH server at SourceForge and carefully check that the fingerprint appearing on your terminal is listed on that page for git.code.sf.net, or some matching pattern such as *.code.sf.net.

If the fingerprint that is printed on your terminal is not listed on that page, answer no to the question Are you sure you want to continue connecting (yes/no)? and copy/paste to flightgear-devel (see Mailing lists) the above message from ssh that contains the fingerprint sent to you by the remote host which pretends to be git.code.sf.net. If this happened, you should stop here and wait for answers from readers of flightgear-devel.

From now on, we'll assume that the fingerprint you received was correct, and therefore that you have answered yes to the Are you sure you want to continue connecting (yes/no)? question.

In this example, it took us several minutes to verify the fingerprint of the git.code.sf.net server and confirm it to ssh. Because of this delay, git.code.sf.net hung up on us and closed the connection. This is absolutely not a problem: we can just rerun the download_and_compile.sh command with the same arguments as the first time. Since we answered yes to the Are you sure you want to continue connecting (yes/no)? prompt, the fingerprint of git.code.sf.net's key has been stored in ~/.ssh/known_hosts, therefore we won't get this prompt anymore. But if some server claiming to be git.code.sf.net presents a host key that has a different fingerprint in the future, ssh will print a big fat warning that the server may belong to an attacker trying to impersonate git.code.sf.net. Therefore, this SSH host key verification is very useful to protect us from future attacks (which hopefully won't happen at all).

As said, we just rerun the download_and_compile.sh command with the same arguments:

$ download_and_compile.sh --git-clone-site-params SourceForge=ssh:SFusername DATA
**********************************************************************
*                                                                    *
* Warning: a typical SimGear + FlightGear + FGData build requires    *
* about 12 GiB of disk space. The compilation part may last from a   *
* few minutes to hours, depending on your computer.                  *
*                                                                    *
* Hint: use the -j option if your CPU has several cores, as in:      *
*                                                                    *
*         download_and_compile.sh -j$(nproc)                         *
*                                                                    *
**********************************************************************
Running 'apt-get update'...
[sudo] password for toto:

(...)

Considering a package alternative: libcurl4-openssl-dev libcurl4-gnutls-dev
Package alternative matched for libcurl4-openssl-dev
Running 'apt-get install build-essential git libcurl4-openssl-dev cmake'...
[sudo] password for toto:

(...)

****************************************
**************** DATA ******************
****************************************
Fetching DATA with 'git clone ssh://SFusername@git.code.sf.net/p/flightgear/fgdata'
Cloning into '.'...
Password:

As explained above, the preceding prompt is for your SourceForge password (which you could guess from the git clone ssh://SFusername@git.code.sf.net/p/flightgear/fgdata command).

remote: Enumerating objects: 67011, done.
remote: Counting objects: 100% (67011/67011), done.
remote: Compressing objects: 100% (31342/31342), done.
remote: Total 67011 (delta 38776), reused 59640 (delta 33570)
Receiving objects: 100% (67011/67011), 2.60 GiB | 313.00 KiB/s, done.
Resolving deltas: 100% (38776/38776), done.
Checking out files: 100% (12959/12959), done.
Password:

(It will take a fair amount of time to get there, because this is the complete download of FGData.)
This is again a prompt for your SourceForge password, because download_and_compile.sh wants to run git pull --rebase in the repository (admittedly, it's a bit dumb after a clone operation—please forgive us). In case you were not monitoring the clone operation, you probably saw the password prompt way after git.code.sf.net got bored waiting for you and closed our second connection:

Connection closed by 216.105.38.16 port 22
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

(if not, there should be no error message and you should have a clean FGData clone)
No worries. Just as before, simply rerun the command with the same arguments:

$ download_and_compile.sh --git-clone-site-params SourceForge=ssh:SFusername DATA
**********************************************************************
*                                                                    *
* Warning: a typical SimGear + FlightGear + FGData build requires    *
* about 12 GiB of disk space. The compilation part may last from a   *
* few minutes to hours, depending on your computer.                  *
*                                                                    *
* Hint: use the -j option if your CPU has several cores, as in:      *
*                                                                    *
*         download_and_compile.sh -j$(nproc)                         *
*                                                                    *
**********************************************************************
Running 'apt-get update'...
[sudo] password for toto:

(...)

Considering a package alternative: libcurl4-openssl-dev libcurl4-gnutls-dev
Package alternative matched for libcurl4-openssl-dev
Running 'apt-get install build-essential git libcurl4-openssl-dev cmake'...
[sudo] password for toto:

(...)

****************************************
**************** DATA ******************
****************************************
DATA: the repository already exists
Password:
Already up to date.
Current branch next is up to date.
Already on 'next'
Your branch is up to date with 'origin/next'.
All optional package alternatives have found a matching package.

download_and_compile.sh has finished to work.

There we are! You now have a clean, up-to-date FGData clone in ~/flightgear/dnc-managed/install/flightgear/fgdata (remember: ~/flightgear/dnc-managed is the directory from which you ran download_and_compile.sh). Note this place: the full path of the ~/flightgear/dnc-managed/install/flightgear/fgdata directory is your $FG_ROOT.

Now, change the protocol to use for future updates of your FGData clone:[6]

(cd install/flightgear/fgdata && \
git remote set-url origin https://git.code.sf.net/p/flightgear/fgdata)

(you can check at any time the protocol(s) in use with the command git remote -v run inside a Git repository—in this case, inside the folder install/flightgear/fgdata). As a consequence of this change, all future updates of your FGData clone will use the https protocol, therefore you won't be prompted anymore for your SourceForge password.

All that remains to do is to run, from the same directory as before (~/flightgear/dnc-managed in our example):

$ download_and_compile.sh

or

$ download_and_compile.sh -j$(nproc)

(-j$(nproc) is “only” useful to save time—see the tip below).

When you don't pass any non-option argument to download_and_compile.sh as done here, it takes care of the three base components needed to run FlightGear: SIMGEAR, FGFS and DATA (these are the component names used by download_and_compile.sh, i.e., the final arguments one can optionally give in a download_and_compile.sh command; in normal speech, they correspond to the SimGear, FlightGear and FGData repositories). Therefore, the above command is presently exactly equivalent to:

$ download_and_compile.sh SIMGEAR FGFS DATA

In case you wanted to build another component such as OSG, you could add it to the command, like this:

$ download_and_compile.sh SIMGEAR FGFS DATA OSG

(With current Debian stable, this is not necessary because it has OpenSceneGraph 3.4. But if your distribution only has an older version, you'll certainly need to select the OSG component like this.)

When the command terminates, you should have a script called run_fgfs.sh in the directory ~/flightgear/dnc-managed from which you ran download_and_compile.sh. This will be your script to run FlightGear. For instance, in order to start the built-in launcher, you can run the following commands:[7]

$ cd ~/flightgear/dnc-managed
$ ./run_fgfs.sh --launcher

(You may omit the --launcher option; this would simply start FlightGear without any launcher, at the default airport and with the default aircraft.)
In case you find this tedious to type or have more arguments to pass on a regular basis, you can follow the advice given at the end of this message or use another launcher such as FFGo (but the FlightGear built-in launcher started with run_fgfs.sh --launcher is quite fine, be sure to try it first!).

Tip  You can considerably speed up the build process—literally, save hours—by telling download_and_compile.sh to use several cores simultaneously when compiling. With option -j$(nproc), compilations will use all cores available on your processor; thus, a typical download_and_compile.sh command is download_and_compile.sh -j$(nproc). If you want to use, say, 4 cores, replace -j$(nproc) with -j4.

Updating FlightGear

Just go to the directory from which you you previously ran download_and_compile.sh (~/flightgear/dnc-managed in our example). This is the folder which, if you did a complete run of download_and_compile.sh as shown in the previous section, contains the run_fgfs.sh script and a log file named compilation_log.txt that records what download_and_compile.sh did in its last run. If you wish to update, say, SimGear, FlightGear and FGData, simply execute this:

$ download_and_compile.sh -pn SIMGEAR FGFS DATA

We'll explain the -pn in a minute. SIMGEAR, FGFS and DATA are called components in download_and_compile.sh terminology. A component generally corresponds to a software repository, or something close. In fact, since SIMGEAR, FGFS and DATA are often precisely the components people wish to update, they form the default components set, so that the previous command is equivalent to:

$ download_and_compile.sh -pn

Now about this -pn. It is equivalent to -p n and means “don't install packages from my (Linux) distribution” (y means yes, please install, n means no, don't install). In case you forgot that, simply run:

$ download_and_compile.sh --help

What does it imply to pass -pn? This tells download_and_compile.sh to completely skip the step where it checks for needed packages from your distribution and installs them, by default using apt-get. It thus goes straight to the following steps:

  • update each repository corresponding to one of the selected components (SIMGEAR, FGFS and DATA in our example);
  • compile each selected component that requires compilation;
  • install each selected component in the appropriate place (under ~/flightgear/dnc-managed according to our Notations).

In case you don't have all required dependencies for the selected components, one of them is likely to fail, of course, since by passing -pn to download_and_compile.sh, you forbid it to install these dependencies for you. So, you can also very well update without passing the -pn option, it will simply take a little longer (the time to check if all dependencies of the selected components are available with APT). In fact, this is what you should do if the previous download_and_compile.sh run failed: first update download_and_compile.sh (see above) then run it without -pn[8] in case new dependencies have been recently added and you don't have them on your system yet—this would be a very likely cause for the failure.

Summary

Routine update:

$ download_and_compile.sh -pn COMPONENT...

In case this fails, first update download_and_compile.sh (see above), then run

$ download_and_compile.sh COMPONENT...

where COMPONENT... stands for the space-separated list of selected components, and defaults to SIMGEAR FGFS DATA if you don't specify any.

Examining the history of download_and_compile.sh

Looking at the latest commits that affected download_and_compile.sh is quite easy with your FGMeta clone:

$ cd ~/flightgear/fgmeta
$ git log -- download_and_compile.sh

(then quit by typing q, assuming your $GIT_PAGER is less)

In order to do the same, but also see the patch for each commit:

$ cd ~/flightgear/fgmeta
$ git log -p -- download_and_compile.sh

List of available components

The download_and_compile.sh script is able to download, compile (when applicable) and install the following components:

  • ATCPIE (for the ATC-pie air traffic control simulation program)
  • CMAKE (for the CMake build tool—this can be useful in case CMake is too old in your distribution)
  • DATA (for FGData, the main set of data files used by FlightGear)
  • FGFS (for FlightGear itself)
  • FGO (for the FGo! FlightGear launcher)
  • FGRUN (for the FGRun FlightGear launcher)
  • FGX (for the FGx FlightGear launcher[9])
  • OPENRADAR (for the OpenRadar air traffic control simulation program)
  • OPENRTI (for OpenRTI[10])
  • OSG (for the OpenSceneGraph library)
  • PLIB (for the PLIB library)
  • SIMGEAR (for the SimGear library—foundation for FlightGear and TerraGear)
  • TERRAGEAR (for the TerraGear terrain building toolchain)
  • TERRAGEARGUI (for TerraGear GUI, a graphical interface for TerraGear)

Each of the items listed above is a component in download_and_compile.sh terminology. Components are written in uppercase by convention.

Note  The preceding list might not be up-to-date. The up-to-date list of components supported by download_and_compile.sh can always be obtained by running download_and_compile.sh --help.

What is the point of knowing this? Because you may pass component names to download_and_compile.sh in order to tell it what you want to download, build and install. By default, only the three essential components SIMGEAR, FGFS and DATA are taken care of, which means that the command:

$ download_and_compile.sh

is equivalent to:

$ download_and_compile.sh SIMGEAR FGFS DATA

In case you want to do the same build with just OpenSceneGraph added, you can use:

$ download_and_compile.sh SIMGEAR FGFS DATA OSG

You get the idea. When several components are passed on the same command line, download_and_compile.sh chooses a reasonable order for processing, so don't worry about that.

When building 'next', you may see build errors

Keeping in mind that this script compiles sometimes bleeding edge software, it can happen that what was successfully compiling last week, does not compile anymore today. Building the stable version should always work, unless there is a problem with the script.

That said, don't be too afraid of building the development version (called next): this is the one developers use all the time, so kindly asking on the flightgear-devel mailing list in case a problem popped up[11] should allow you to find good advice and get the problem quickly fixed, if it's a new one. Conversely, probably not many people (in 2019) build themselves the “stable” version of FlightGear. People who want the stable version can usually have it from their distribution, this is generally easier. On the other hand, those running download_and_compile.sh typically want to build the latest code that FlightGear developers are working on; this is useful when you want to contribute feedback, code, aircraft or scenery based on recent technology, or just want to enjoy the latest features and bugs. ;-)

Task-specific instructions

Note  In this section, we assume you've read and followed the advice given in Getting started with download_and_compile.sh.

Selecting the components to build

By default, download_and_compile.sh downloads or updates, then compiles, SimGear and FlightGear, and downloads or updates FGData (by nature, FGData can't be compiled). This is what happens when running:

$ download_and_compile.sh

The preceding command is therefore equivalent to:

$ download_and_compile.sh SIMGEAR FGFS DATA

To make download_and_compile.sh take care of other programs or libraries, use non-option arguments naming the components you want, for instance:

$ download_and_compile.sh SIMGEAR FGFS DATA OSG

SIMGEAR, FGFS, DATA and OSG are the component names respectively corresponding to SimGear, FlightGear, FGData and OpenSceneGraph in download_and_compile.sh's terminology.

A list of available components is provided on this page, but the fully up-to-date list can always be obtained by running download_and_compile.sh --help.

Choosing between stable and development versions

By default, download_and_compile.sh fetches code and data from development branches of the source repositories (which sometimes causes compilation or runtime errors). However, it is possible to tell the script to download the latest ”stable” version of each component, for some suitable definition of “stable”. This is by means of the -s option:

$ download_and_compile.sh -s COMPONENT1 COMPONENT2...

How does it work?

  • For SimGear, FlightGear and FGData, it uses the most recent stable release branch of the corresponding Git repository.
  • For some components, a known-stable version is hardcoded in download_and_compile.sh and used when the -s option is given[12] (for instance, the hardcoded information can be the name of a Git branch).
  • For the other components, the -s option has no effect.
Warning  In a given folder where download_and_compile.sh is run, you should either always use the -s option, or never. Building some components with -s and others without (all within the same base folder) is not supported.

Applying what we just said to the FGFS component (FlightGear) leaves us with two main options:

  • build the latest stable release;
  • build the current development version (bleeding edge), which lives in the next branch of the FlightGear repository.

Building the latest stable FlightGear release

When executing download_and_compile.sh, use the -s option to build the latest stable release:

$ cd ~/flightgear/dnc-managed
$ download_and_compile.sh -s
Note  If you decide to use the -s option in a given directory tree, you should use it for all components in that directory tree (SIMGEAR, FGFS, DATA, etc.). Running download_and_compile.sh in a given directory with the -s option for some components and not for others is not supported.

Building the current FlightGear development version

When executing download_and_compile.sh without any option, the development version of every selected component is built:

$ cd ~/flightgear/dnc-managed
$ download_and_compile.sh

(In this example, the implicitly-selected components are SIMGEAR, FGFS and DATA, as explained above.)

Note  The development version of FlightGear changes on an almost daily basis. It provides the latest features, but is not guaranteed to always work reliably. If you don't want to take the risk of finding new bugs when updating, you may prefer to use the latest stable release.

Passing custom arguments to CMake

Sometimes, when building a program, you may want to enable a feature that is not enabled by default, or disable a feature that is enabled by default. With recent versions of download_and_compile.sh (April 2019 or later), the environment variables SG_CMAKEARGS and FG_CMAKEARGS allow one to do that for SimGear and FlightGear. For instance, in order to link SimGear with the system Expat library, you can do:

$ SG_CMAKEARGS='-DSYSTEM_EXPAT=ON' download_and_compile.sh SIMGEAR

Similarly, disabling HID-based input when building FlightGear can be achieved this way:

$ FG_CMAKEARGS='-DENABLE_HID_INPUT=OFF' download_and_compile.sh FGFS
Note  Such options are typically defined in CMakeLists.txt files, for example here for SimGear and here for FlightGear.

This can be useful for instance to work around bugs in a part of SimGear or FlightGear that you don't need, but causes a build or runtime failure (see here for example). This is often convenient when using the development version of FlightGear, but doesn't mean such bugs shouldn't be reported!

If you have several such options to pass, just use spaces to separate them inside the quotes:

$ FG_CMAKEARGS='-DENABLE_SWIFT=ON -DENABLE_HID_INPUT=OFF' download_and_compile.sh FGFS

Of course, the same thing can also be achieved this way in Bourne-style shells:

$ export FG_CMAKEARGS='-DENABLE_SWIFT=ON -DENABLE_HID_INPUT=OFF'
$ download_and_compile.sh FGFS

It is also possible to set both SG_CMAKEARGS and FG_CMAKEARGS at the same time for a given download_and_compile.sh run:

$ export SG_CMAKEARGS='-DSYSTEM_EXPAT=ON'
$ export FG_CMAKEARGS='-DENABLE_SWIFT=ON -DENABLE_HID_INPUT=OFF'
$ download_and_compile.sh

Launching FlightGear

When using download_and_compile.sh, apart from those installed with the package manager, the FlightGear dependencies (which are typically libraries) are not installed system-wide but under the directory from which download_and_compile.sh was run. This makes it possible to easily use, for instance, different OpenSceneGraph, SimGear and FlightGear versions on a single system—e.g., for testing purposes—but also to have separate build trees (optimized/debug). This is also why you either need to set LD_LIBRARY_PATH to run the built programs, or simply use the scripts created by download_and_compile.sh in the directory where it is run, such as run_fgfs.sh and run_fgfs_debug.sh: these scripts automatically set up the required environment variables according to your build settings before firing the desired program (e.g., fgfs) with the arguments you provided.

Therefore, the simplest way to run a FlightGear program built by download_and_compile.sh is to launch the run_fgfs.sh script that download_and_compile.sh created in the directory from which it was run, for example:

$ cd ~/flightgear/dnc-managed
$ ./run_fgfs.sh --launcher
Note./run_fgfs.sh --launcher starts FlightGear with its built-in launcher. If you just do ./run_fgfs.sh, FlightGear will be started without any launcher, at the default airport and with the default aircraft.

In order to start FlightGear without any launcher, at a given airport (say, Paro airport, whose ICAO code is VQPR) and with a chosen aircraft, you can do:

$ cd ~/flightgear/dnc-managed
$ ./run_fgfs.sh --airport=VQPR --aircraft=dhc6

Actually, the directory change is not needed, we only gave it here for readability. Therefore, the following single command does the same:

$ ~/flightgear/dnc-managed/run_fgfs.sh --airport=VQPR --aircraft=dhc6

Avoiding multiple downloads of FGData

Some people use download_and_compile.sh to maintain several directory trees such as the tree starting at ~/flightgear/dnc-managed in Getting started with download_and_compile.sh (this can be useful if you want to have one tree with programs compiled in Release mode and another tree where they are built in Debug mode, for instance). This can easily be done by running download_and_compile.sh in each of the directories. But since FGData is so large, it may be tempting to share a single instance of this repository among several trees. This is not officially supported, but apparently can be made to work with symbolic links.

Let's show how this can be done on an example. Suppose your master copy of FGData is in ~/flightgear/dnc-managed/install/flightgear/fgdata. Then the following appears to work:

$ mkdir -p ~/flightgear/other-dnc-managed-tree/install/flightgear
$ cd ~/flightgear/other-dnc-managed-tree/install/flightgear
$ ln -s ../../../dnc-managed/install/flightgear/fgdata
$ cd ~/flightgear/other-dnc-managed-tree
$ download_and_compile.sh

The last of these commands will use and update the FGData repository present in ~/flightgear/dnc-managed/install/flightgear/fgdata.

Warning  This can only work simply if all trees that share a given FGData repository are from the same release (e.g., current stable or development). Running a “stable“ FlightGear with FGData from the next branch or the other way round, a development version of FlightGear with FGData from a release branch, doesn't work—and FlightGear should tell you when you start it in such a situation.

That said, people comfortable with Git can check out the correct FGData branch before building or starting FlightGear, for instance:

$ cd /path/to/fgdata && git checkout release/2019.1

or

$ cd /path/to/fgdata && git checkout next

So, this is possible but somewhat acrobatic. You've been warned.

Note: there is a wiki article about this subject, but it is severely outdated as of April 2019.

Additional programs

Note  In this section, we assume you've read and followed the advice given in Getting started with download_and_compile.sh.

If you wish to get other programs (precisely: download, build and install them), you need to launch download_and_compile.sh with the desired component names as arguments. For instance:

$ cd ~/flightgear/dnc-managed
$ download_and_compile.sh SIMGEAR FGFS DATA OSG

See above for the list of available components.

TerraGear

Run download_and_compile.sh with the TERRAGEAR component in order to build and install the TerraGear terrain building toolchain:

$ cd ~/flightgear/dnc-managed
$ download_and_compile.sh TERRAGEAR

This creates the following scripts in ~/flightgear/dnc-managed:

  • run_genapts850.sh
  • run_ogr-decode.sh
  • run_tg-construct.sh

These scripts themselves run the corresponding TerraGear tools, as expected.

TerraGear GUI

TerraGear GUI is a graphical interface for TerraGear written with the Qt toolkit (still Qt 4 in 2019, but it works). In order to install it, run download_and_compile.sh with the TERRAGEARGUI component:

$ cd ~/flightgear/dnc-managed
$ download_and_compile.sh TERRAGEARGUI

This will create a run_terrageargui.sh script in ~/flightgear/dnc-managed, and also a default configuration file ~/.config/TerraGear/TerraGearGUI.conf, unless you already have one. This default configuration file contains paths to the TerraGear and $FG_ROOT directories, assuming you have installed the TERRAGEAR and DATA components.

To run TerraGear GUI:

$ cd ~/flightgear/dnc-managed
$ ./run_terrageargui.sh

FGCom

NoteFGCom has been integrated into FlightGear long ago, therefore the following is not needed in general.

FGCom is the system used by FlightGear to simulate radio communications between users. It is automatically built and installed when you tell download_and_compile.sh to take care of the FGFS component. You can launch the standalone FGCom program by using the run_fgcom.sh script:

$ cd ~/flightgear/dnc-managed
$ ./run_fgcom.sh

FGRun

Note  As of 2019, FGRun has been superseded by the FlightGear built-in launcher. The built-in launcher is the most actively maintained launcher for FlightGear. Other launchers are FFGo and FGx.
Fgrun-page2.jpg

Before FlightGear had its built-in launcher (the one you get with run_fgfs.sh --launcher), many users found comfortable having FlightGear launched by the graphical utility FGRun. This program is built and installed when download_and_compile.sh is run with the FGRUN component. You then have to launch the run_fgrun.sh command, for example:

$ cd ~/flightgear/dnc-managed
$ ./run_fgrun.sh

FGRun will save its settings in ~/.fltk/flightgear.org/fgrun.prefs. You may want to save copies of the preferences customized for stable and next.

FGo!

Note  As of 2019, FGo! is not maintained anymore. You may want to try the built-in launcher (started with run_fgfs.sh --launcher) or FFGo.
Fgo01.jpg

FGo! is a graphical utility written in python. It is downloaded and installed when download_and_compile.sh is run with the FGO component. You then have to launch the run_fgo.sh command, for example:

$ cd ~/flightgear/dnc-managed
$ ./run_fgo.sh

Remember that the first time you run it, you have to go to open the Preferences dialog and set the paths to the fgfs executable and to FGData.

Troubleshooting

Compilation errors

Here we are, no fear, if you wish to use programs from the cvs/svn/git repositories, you might face compilation errors that will prevent you to have a working copy of one or more of the programs provided by this script. What can be the causes that prevent us from successfully compiling? As far as I know those:

  1. Software developers introduce a new functionality with a new piece of code that prevents the compilation under your architecture, this can happen working with cvs/svn/git sources.
  2. The program refuses to compile because of a divergence in the libraries on which it depends. For example FlightGear might not compile because OSG has been modified, while OSG itself compiles fine, FG won't.
  3. One or more repositories are down and you can't get the library you need. (Both from cvs/svn/git or apt-get)

There is a simple solution to the above errors: wait and relaunch the script after some time (hours or days), if software developers repair or synchronize their code with the newly updated libraries (which generally happens eventually), your FlightGear will compile fine as if the previous error never took place.

Sometimes it happens that the script fails to compile only FGRun, FGCom or atlas, if you then see the run_fgfs.sh file it means that FlightGear installation was successful and you can safely run it.

OpenRTI undefined reference errors

Sometimes due to the way d&c build cleans up projects, linking errors might occur, this is the case with this error "libRTI-NG.so: undefined reference to xxx", sadly at this point either you need to patch the d&c script to clean OpenRTI with rm -f CMakeCache.txt && rm -rf CMakeFiles/, or just start from a clean environment, assuming you are in the parent project directory, you will need to issue this command to clean everything (except FGData):

$ rm -rf build/* install/simgear/ install/openrti/ install/flightgear/share/ install/flightgear/bin/

See this thread for more details.

Options

Multicore acceleration

Passing option -j x to download_and_compile.sh (where x is the number of your CPU cores you wish to assign to the job) will considerably speed up the compilation steps.

Advanced options

  • Build a release version: -b Release
  • Build a version that should run as fast as a release build, yet has debug information that can be used to post backtraces: -b RelWithDebInfo (this is the default)
  • Build a full debug version for very complete bug reporting: -b Debug
  • Skip download of distro packages (i.e., by default: apt-get install ...): -p n
  • Skip retrieving of software updates (i.e., by default: apt-get update): -d n
  • Skip the configure step (like running CMake or autoconf's ./configure): -r n
  • Skip compilation of programs: -c n.

For example, if you are a developer and wish to quickly recompile and reinstall only your own modifications for FlightGear, you can do this:

$ download_and_compile.sh -p n -d n -r n FGFS

Note that this is the same as:

$ download_and_compile.sh -pn -dn -rn FGFS

This command will only rebuild modified files and reinstall FlightGear. Note that depending on the kind of changes you made, reconfiguring and thus dropping the -d n option may be necessary, though (this is the case in particular if you added or removed C++ files).

Optimus technology

If your computer has a GPU with Optimus technology, you need a dedicated script in order to make FlightGear run with the powerful GPU.

After having installed required tools (Bumblebee) you just need to run this command line in your FlightGear installation directory (where you executed download_and_compile.sh):

$ sed  's|\./fgfs|optirun ./fgfs|' run_fgfs.sh > run_fgfs_optirun.sh && chmod +x run_fgfs_optirun.sh

Now you can run FlightGear with ./run_fgfs_optirun.sh.

The same can be done for the FGRun launcher:

$ sed  's|\./fgrun|optirun ./fgrun|' run_fgrun.sh > run_fgrun_optirun.sh && chmod +x run_fgrun_optirun.sh

See also

References

  1. Due to technical problems on the SourceForge side, this is currently only true once you have an FGData clone. See here for details.
  2. If you think you already have the dependencies, this installation can be disabled either by using option -pn or by passing option --sudo=echo (the latter results in printing the apt-get command line without running it).
  3. Look for strings such as zlib1g-dev, libglew-dev or qt5-default.
  4. This looks like, but is not a Git commit identifier. This kind of “version number” is admittedly not very pretty, but it doesn't pollute Git commits (the diffs) and is automatically updated by Git every time you update download_and_compile.sh the way we present here; thus, the advantages compensate for the ugliness.
  5. Well, empty before the first time; later, download_and_compile.sh is going to populate it with plenty of FlightGear files and subdirectories, of course.
  6. Another way would be to manually change the relevant line starting with url = ssh://SFusername@ for the origin remote in the .git/config file that lives inside your repository clone (i.e., ~/flightgear/dnc-managed/install/flightgear/fgdata/.git/config in our example).
  7. We give these commands because they are easy to read, but the cd command is not needed if you use the correct path, as in ~/flightgear/dnc-managed/run_fgfs.sh --launcher.
  8. Which is the same as passing -py.
  9. Support for FGx in download_and_compile.sh would probably benefit from a code review.
  10. Note that OpenRTI is just an optional dependency for HLA support. For the time being, you should be just fine building without it. Eventually, the idea is for HLA to replace the existing MP system and even increasingly distribute the FlightGear architecture such that more and more components can be more easily run in separate threads or even separate processes, possibly even on different machines. So this is going to be an important feature for professional users, using several computers and screens to create a comprehensive and immersive simulation environment.

    At the moment, it is probably safe to say that HLA is only of interest to developers and people willing to play with experimental features.

  11. Don't forget in this case to precisely tell what you did and include the compilation_log.txt file written by download_and_compile.sh.
  12. As of May 2019, apart from SIMGEAR, FGFS and DATA, the only components for which download_and_compile.sh “knows” a stable version are TERRAGEAR and OPENRTI.