Hi fellow wiki editors!

To help newly registered users get more familiar with the wiki (and maybe older users too) there is now a {{Welcome to the wiki}} template. Have a look at it and feel free to add it to new users discussion pages (and perhaps your own).

I have tried to keep the template short, but meaningful. /Johan G

Difference between revisions of "Scripted Compilation on Linux Debian/Ubuntu"

From FlightGear wiki
Jump to: navigation, search
(Advanced options: Update)
(Disk usage: Rename section and update)
Line 261: Line 261:
 
Using the option '''-j x''' (where x is the number of your CPU-Cores you wish to assign to the job) will speed up the whole compilation process considerably.
 
Using the option '''-j x''' (where x is the number of your CPU-Cores you wish to assign to the job) will speed up the whole compilation process considerably.
  
== Disk usage ==
+
== Disk usage and build time ==
Having both compiled program, source code, and data from git requires some hard disk space: It will take something like 13 GB of space.
+
 
If you don't have a fast machine, it will require several hours of compilation time.
+
Keeping compiled programs, source code and data repositories requires some hard disk space: it will take something like 12 GB of space for just [[SimGear]], FlightGear and [[FGData]] (i.e., the SIMGEAR, FGFS and DATA components).
 +
 
 +
With an Intel Core i7 860 CPU (2.80 GHz) purchased in 2009, compiling [[SimGear]] and FlightGear 2019.2 with option <code>-j8</code> takes about 14 minutes. If you don't have a fast machine and build using only one core, it may require several hours.
  
 
== Optimus technology ==
 
== Optimus technology ==

Revision as of 02:20, 24 April 2019

download_and_compile.sh is a Bash script that takes care of downloading and compiling FlightGear from the Git repositories with just one command execution[1] for both 32-bit and 64-bit Debian-based systems (Debian, Ubuntu, Devuan, Linux Mint, etc.). Pre-existing installed versions (if any) of FlightGear are not touched at all since the script downloads, builds and installs everything under the directory in which it is launched. You can choose between building all or only specific tools and applications. For RedHat-based systems such as Fedora and CentOS, you may want to check out CentOS.

By default, download_and_compile.sh installs most dependencies with apt-get run under sudo.[2] Other dependencies, either because they aren't available in the standard APT repositories, or because it was explicitly requested using the non-option arguments of download_and_compile.sh, are downloaded and compiled on the fly (this can be the case for PLIB, Simgear and OpenSceneGraph, for instance—depending on the arguments passed to download_and_compile.sh).

For hints on using a RPM-based distribution like CentOS, please see CentOS.

Please also see Superbuild.

Update Available

The latest version of download_and_compile.sh can be obtained here, however there are advantages getting it from an FGMeta clone as explained here. Contents should be moved from there to this page; significant parts of what follows on this page haven't been updated recently.

Cut to the Chase: for the impatient

Beware: with the following, you are likely to encounter problems when cloning FGData: read just above.

cd  <your working directory for building FlightGear>
wget -O download_and_compile.sh https://sourceforge.net/p/flightgear/fgmeta/ci/next/tree/download_and_compile.sh?format=raw
chmod +x download_and_compile.sh
mkdir -p stable
mkdir -p next
cd stable
../download_and_compile.sh -j$(nproc) -s
cd ../next
../download_and_compile.sh -j$(nproc) -p n
Note  With -j$(nproc) as above, the compilation will use all cores available on your processor, which can save several hours. If you want to use, say, 4 cores, replace -j$(nproc) with -j4.

Conversion of directory structure from earlier versions of download_and_compile.sh

Earlier versions of the script used a different directory structure. If you used the earlier version, the new script includes a section that will convert the earlier structure to the current set of directories.

Disk Space Requirements

As of April 2019, building FlightGear requires about 11 Gigabytes of disk space. Note that this includes downloaded source code, build files and the large FGData repository (about 5 Gigabytes for that one).

List of compiled programs

The download_and_compile.sh script is able to download and compile:

(Note that OpenRTI is just an optional dependency for HLA support. For the time being, you should be just fine building without it. Eventually, the idea is for HLA to replace the existing MP system and even increasingly distribute the FlightGear architecture such that more and more components can be more easily run in separate threads or even separate processes, possibly even on different machines. So this is going to be an important feature for professional users, using several computers and screens to create a comprehensive and immersive simulation environment.

At the moment, it's probably safe to say that HLA is only of interest to developers and people willing to play with experimental features.)

Each of the items listed above corresponds to a component in download_and_compile.sh terminology. Components are written in uppercase. The list of supported ones is currently:

  • SIMGEAR
  • FGFS (this corresponds to FlightGear)
  • DATA (this corresponds to FGData)
  • CMAKE
  • PLIB
  • OSG (this corresponds to OpenSceneGraph)
  • FGRUN
  • FGO
  • FGX
  • TERRAGEAR
  • TERRAGEARGUI
  • OPENRADAR
  • ATCPIE
  • OPENRTI

The up-to-date list of components can be obtained by running:

download_and_compile.sh --help

What is the point of knowing this? Because you may pass component names to download_and_compile.sh in order to tell it what you want to download, build and install. By default, only the three essential components SIMGEAR, FGFS and DATA are taken care of, which means that the command:

download_and_compile.sh

is equivalent to:

download_and_compile.sh SIMGEAR FGFS DATA

In case you want to do the same build with just OpenSceneGraph added, you can use:

download_and_compile.sh SIMGEAR FGFS DATA OSG

You get the idea. When several components are passed on the same command line, download_and_compile.sh chooses a reasonable order for processing, so don't worry about that.

When building Next you may see build errors

Keeping in mind that this script compiles sometimes bleeding edge software, it can happen that what was successfully compiling last week, does not compile anymore today. Building the stable version should always work, unless there is a problem with the script.

That said, don't be too afraid of building the development version (called next): this is the one developers use all the time, so kindly asking on the flightgear-devel mailing list in case a problem popped up[3] should allow you to find good advice and get the problem quickly fixed, if it's a new one. Conversely, probably not many people (in 2019) build themselves the “stable” version of FlightGear. People who want the stable version can usually have it from their distribution, this is generally easier. On the other hand, those running download_and_compile.sh typically want to build the latest code that FlightGear developers are working on; this is useful when you want to contribute feedback, code, aircraft or scenery based on recent technology, or just want to enjoy the latest features and bugs. ;-)

Download

You can get download_and_compile.sh from FGMeta. It is contained in the FGMeta repository, which is maintained by the FlightGear developers. Remember to update this script whenever a new FlightGear version is released, so that you'll be able to download the latest stable revision (of course, you can update it more often to benefit from bug fixes, it doesn't have to happen after a release).

Instructions

To run download_and_compile.sh, just save it in a directory called for example: ~/fgfs then execute it (no need to execute it as root).

Here is for example a sequence of commands to get the script from the next branch:

mkdir ~/fgfs
cd ~/fgfs
wget https://sourceforge.net/p/flightgear/fgmeta/ci/next/tree/download_and_compile.sh?format=raw
chmod 755 download_and_compile.sh

You have two options now: build the latest stable FlightGear release or build the current development version (bleeding edge). After building stable or the latest, if you need to build a different version, try the instructions for Avoiding multiple downloads of FGData on Linux (warning: as of April 2019, they are outdated).

Build the latest stable FlightGear release

When executing the script, use the "-s" option to build the latest stable release:

./download_and_compile.sh -s

Build the current FlightGear development version

When executing the script without any options, the latest development version is built.

Warning: The development version of FlightGear changes on an almost daily basis. It provides the latest features, but is not always guaranteed to work reliably. If you're unfamiliar with software testing, you may prefer to use the latest stable release.

./download_and_compile.sh

Once the script has finished running, you will successfully get FlightGear installed in the ~/fgfs directory.

Launching FlightGear

When using download_and_compile.sh, apart from those installed with the package manager, the FlightGear dependencies (which are typically libraries) are not installed system-wide but under the directory from which download_and_compile.sh is run. This makes it possible to easily use, for instance, different OpenSceneGraph, SimGear and FlightGear versions on a single system—e.g., for testing purposes—but also to have separate build trees (optimized/debug). This is also why you either need to set LD_LIBRARY_PATH to run the built programs, or simply use the scripts created by download_and_compile.sh in the directory where it is run, such as run_fgfs.sh and run_fgfs_debug.sh: these scripts automatically set up the required environment variables according to your build settings before firing the desired program (e.g., fgfs) with the arguments you provided.

Therefore, the simplest way to run your newly-built FlightGear is to launch the run_fgfs.sh script that download_and_compile.sh created in the directory from which it was run, for example:

cd ~/fgfs
./run_fgfs.sh --launcher
Note./run_fgfs.sh --launcher starts FlightGear with its built-in launcher. If you just do ./run_fgfs.sh, FlightGear will be started without any launcher, at the default airport and with the default aircraft.

Launching FGRun

Note  As of 2019, FGRun is not maintained anymore. You may want to try the built-in launcher (see above) or FFGo.
Fgrun-page2.jpg

For many users it's more comfortable having FlightGear launched by the graphical utility FGRun which is installed as well in the same folder. You have to launch the run_fgrun.sh command, for example:

cd ~/fgfs
sh run_fgrun.sh

FGRun will save its settings in ~/.fltk/flightgear.org/fgrun.prefs you may want to save copies of the preferences customized for stable and next.

Launching FGo!

Note  As of 2019, FGo! is not maintained anymore. You may want to try the built-in launcher (see above) or FFGo.
Fgo01.jpg

This is a graphical utility written in python, You have to launch the run_fgo.sh command, for example:

cd ~/fgfs
sh run_fgo.sh

Remember that the first time you run it, you have to go to preferences and set the binary and FGData path (Do no set the working directory, you don't need it).

Additional programs

If you wish to get other programs, you need to launch the script with the desired component names as arguments. For instance:

./download_and_compile.sh SIMGEAR FGFS DATA OSG

The available components are (in April 2019):
CMAKE, PLIB, OPENRTI, OSG, SIMGEAR, FGFS, DATA, FGRUN, FGO, FGX, OPENRADAR, ATCPIE, TERRAGEAR, TERRAGEARGUI.

The full, up-to-date list of these components can always be obtained by running:

./download_and_compile.sh --help

Launching FGCOM

NoteFGCom has been integrated into FlightGear long ago, therefore the following is not needed in general.

FGCom is the system used by FlightGear to simulate radio communications between users. Launch it using the run_fgcom.sh command:

cd ~/fgfs
sh run_fgcom.sh

Troubleshooting

Compilation errors

Here we are, no fear, if you wish to use programs from the cvs/svn/git repositories, you might face compilation errors that will prevent you to have a working copy of one or more of the programs provided by this script. What can be the causes that prevent us from successfully compiling? As far as I know those:

  1. Software developers introduce a new functionality with a new piece of code that prevents the compilation under your architecture, this can happen working with cvs/svn/git sources.
  2. The program refuses to compile because of a divergence in the libraries on which it depends. For example FlightGear might not compile because OSG has been modified, while OSG itself compiles fine, FG won't.
  3. One or more repositories are down and you can't get the library you need. (Both from cvs/svn/git or apt-get)

There is a simple solution to the above errors: wait and relaunch the script after some time (hours or days), if software developers repair or synchronize their code with the newly updated libraries (which generally happens eventually), your FlightGear will compile fine as if the previous error never took place.

Sometimes it happens that the script fails to compile only FGRun, FGCOM or atlas, if you then see the run_fgfs.sh file it means that FlightGear installation was successful and you can safely run it.

OpenRTI undefined reference errors

Sometimes due to the way d&c build cleans up projects, linking errors might occur, this is the case with this error "libRTI-NG.so: undefined reference to xxx", sadly at this point either you need to patch the d&c script to clean OpenRTI with rm -f CMakeCache.txt && rm -rf CMakeFiles/, or just start from a clean environment, assuming you are in the parent project directory, you will need to issue this command to clean everything(except FGDATA):

rm -rf build/* install/simgear/ install/openrti/ install/flightgear/share/ install/flightgear/bin/

see this thread for more details (https://forum.flightgear.org/viewtopic.php?f=45&t=26244)

Options

By default, download_and_compile.sh downloads or updates, then compiles, SimGear and FlightGear, and downloads or updates FGData (by nature, FGData can't be compiled). This is what happens when running:

./download_and_compile.sh

To make it download or update and compile something else, you may pass non-option arguments naming the components you want, for instance:

./download_and_compile.sh SIMGEAR FGFS DATA OSG

SIMGEAR, FGFS, DATA and OSG are the component names respectively corresponding to SimGear, FlightGear, FGData and OpenSceneGraph in download_and_compile.sh's terminology.

See the output of download_and_compile.sh --help for more details.

Compiling only one program

Note  The following is not necessarily up-to-date. See the output of download_and_compile.sh --help for the up-to-date list of components.

If you wish to recompile only one of the programs you can launch the script with one of the following parameters:

  • CMAKE (to compile and install only cmake)
  • PLIB (to compile and install only plib)
  • OSG (to compile and install only OpenSceneGraph)
  • SIMGEAR (to compile and install only Simgear)
  • FGFS (to compile and install only FlightGear)
  • DATA (to download / update only data files for FlightGear)
  • FGRUN (to compile and install only Fgrun)
  • TERRAGEAR (to compile and install only terragear!)
  • TERRAGEARGUI (to compile and install only terrageargui!)
  • OPENRADAR (to compile and install only OpenRadar!)
  • FGO (to compile and install only Fgo!)

Compiling the last known-stable versions

Even if download_and_compile.sh fetches code and data from development branches of the source repositories by default (which sometimes causes compilation errors), it is possible to tell the script to download the latest known versions of the software that were compiling successfully by means of the -s option:

./download_and_compile.sh -s

How does it work?

  • For SimGear, FlightGear and FGData, it uses the most recent stable release branch of the corresponding Git repository.
  • For some components, a known-stable version is hardcoded in download_and_compile.sh and used when the -s option is given[4] (for instance, the hardcoded information can be the name of a Git branch).
  • For the other components, the -s option has no effect.
Warning  In a given folder where download_and_compile.sh is run, you should either always use the -s option, or never. Building some components with -s and others without (all within the same base folder) is not supported.

Advanced options

  • Build a release version: -b Release
  • Build a version that should run as fast as a release build, yet has debug information that can be used to post backtraces: -b RelWithDebInfo (this is the default)
  • Build a full debug version for very complete bug reporting: -b Debug
  • Skip download of distro packages (i.e., by default: apt-get install ...): -p n
  • Skip retrieving of software updates (i.e., by default: apt-get update): -d n
  • Skip the configure step (like running CMake or autoconf's ./configure): -r n
  • Skip compilation of programs: -c n.

For example, if you are a developer and wish to quickly recompile and reinstall only your own modifications for FlightGear, you can do this:

./download_and_compile.sh -p n -d n -r n FGFS

Note that this is the same as:

./download_and_compile.sh -pn -dn -rn FGFS

This command will only rebuild modified files and reinstall FlightGear. Note that depending on the kind of changes you made, reconfiguring and thus dropping the -d n option may be necessary, though (this is the case in particular if you added or removed C++ files).

Multicore Acceleration

Using the option -j x (where x is the number of your CPU-Cores you wish to assign to the job) will speed up the whole compilation process considerably.

Disk usage and build time

Keeping compiled programs, source code and data repositories requires some hard disk space: it will take something like 12 GB of space for just SimGear, FlightGear and FGData (i.e., the SIMGEAR, FGFS and DATA components).

With an Intel Core i7 860 CPU (2.80 GHz) purchased in 2009, compiling SimGear and FlightGear 2019.2 with option -j8 takes about 14 minutes. If you don't have a fast machine and build using only one core, it may require several hours.

Optimus technology

If your computer has a GPU with Optimus technology, you need a dedicated script in order to make FG running with the powerful GPU.

After having installed required tools (Bumblebee) you just need to run this command line in your FG installation directory (where you executed ./download_and_compile.sh) :

sed  's|\./fgfs|optirun ./fgfs|' run_fgfs.sh > run_fgfs_optirun.sh && chmod +x run_fgfs_optirun.sh

Now you can run FG with ./run_fgfs_optirun.sh


the same is applied with the FGRun launcher :

sed  's|\./fgrun|optirun ./fgrun|' run_fgrun.sh > run_fgrun_optirun.sh && chmod +x run_fgrun_optirun.sh

Remove warning message for DDS files

You can remove the warning message displayed when DDS files are parsed by SimGear by adding the following line just after cd "simgear" line :

	sed -i s/SG_ALERT,\ \"Image/SG_WARN,\ \"Image/g simgear/scene/model/ModelRegistry.cxx

See also

  • Other FlightGear build scripts can be found in FGMeta.
  • Another script for building FlightGear and all its dependencies in an automated fashion can be found here. The page seems a bit oldish, though (as of 2019).

References

  1. Due to technical problems on the SourceForge side, this is currently only true once you have an FGData clone. See here for details.
  2. This can be changed with options --package-manager and --sudo, or completely turned off with option -pn (see the output of download_and_compile.sh --help for the list of available options).
  3. Don't forget in this case to precisely tell what you did and include the compilation_log.txt file written by download_and_compile.sh.
  4. As of April 2019, apart from SIMGEAR, FGFS and DATA, the only component for which download_and_compile.sh “knows” a stable version is OPENRTI.