Talk:Scripted Compilation on Linux Debian/Ubuntu

Script Overview -- Discussion of why the script was originally developed and where it might go from here

The download_and_compile.sh script was originally developed as an easy way to get the latest flightgear features in the hands of non-developers and to entice new developers to the project with an easy way to get a workable build environment in place. Suggestions have been made to move the script in the direction of a more general purpose tool that might also support or complement work on build servers.

Functional requirements --Specifics of what the script should produce

1. shell script (currently bash) with options and parameters
2. install any pre-requisite development packages
3. download or update sources as needed
4. build the most recent released version
5. build any pre-release version of flightgear
6. build different variations of flightgear such as with a different event handler, with openrti or compilation with the -g (debug) option.
7. download fgdata
8. set of scripts to start flightgear, launchers and utilities

Other requirements:

1. minimal load on network resources
2. support for building with different versions of underlying packages such as boost, OSG, etc
3. logging of build steps with statistics and evaluation of build results (success, failure etc)

Current Script Features

The script has the following major features:

1. logging
2. pre-requsite development package installation
3. option and parameter interpretation
4. build a debug version
5. build with openrti
6. option to update or not update sources from upstream
7. option to re-configure or not
8. for each component
   1. obtain original sources, update, set to correct version
   2. configure
   3. build and Install of components

Callahanp (talk) 17:37, 17 November 2013 (UTC)

I also did some work (and pushed it) to support CMake superbuilds using fgmeta. It works on Mac and Windows, and probably on Linux with some small tweaks, so this might be a way to simplify future versions of the download_and_compile script, since most of the steps (download & build OSG, download and build PLIB) are handled by the super-build. — James Turner

1. use of makefile or cmake as a basis for or as part of the script.

Advantages:

1. Cross Platform.
2. fits with general direction things seem to be moving in.

Disadvantages:

1. Cmake and makefiles can be difficult for the un-initiated.

How about a walkthrough using flightgear builds as an example?

Opinion:

1. This is a great idea, Should it be a separate alternative script? Callahanp 11:34, 31 May 2013 (UTC)

Agreed, the cmake-based approach sounds really sensible, and would also help unify the build infrastructure,i.e. regarding the FlightGear Build Server, to make the setup better reproducable, but also allow CI to be set up locally, i.e. as part of NetBeans or Eclipse. The question is which parts of the script could be reasonably moved into some form of CMake/superbuild script - certain things are highly distro/OS/platform specific. And I'm not sure just how flexible cmake is here ?

Proposed features -- New features of the script that have yet to be implemented

• No more multiple downloads of fgdata for each separate build. You download only one, and if you need more than one version, it is created locally with copy and git commands. All variant builds use a copy of fgdata from a parent directory
• No more multiple download and builds of plib and osg. You download and build plib-1.8.5 and osg-3.0.1 once. Their sources are kept in a separate directory.
• A single copy of flightgear, simgear and openrti sources that can be used to build whatever versions are needed, one at a time
• Support for additional versions of osg such as 3.1.9
• Add support for both stable and latest osg.
• support for personal and team clones of components to be built
• installing components in a system location
• build while downloading
• download fgdata as a restartable bundle (FGDATA is being broken up so this may be less of an issue from 3.0 forward.
• download all scenery (or maybe specific regions?)
• SUSE, Redhat/yum & CENTOS support
• Distribution builds
• use installed versions of libraries for plib, osg, boost instead of rebuilding the same version
• network setup tests for fgcom, multiplayer and terragear
• torrents
• When updating the sources, its possible to automatically detect if a reconfiguration is needed or not.
• You might want to control which components are rebuilt and which are simply recompiled and re-linked
• You might be in the process of updating a file in one of the components. How would the build need to change to avoid overwriting your change.
• We currently use the main flightgear git archives for sources. Its possible to use personal or team clones for one or more components instead?
• Do we need better reporting of the results of the build?
• what can we do with cmake to reduce the bash code used to make decisions about what steps need to be done?
• how much of the scripts logic can we delegate to cmake?

git usage

create a feature branch

git branch -f script_prototype # creates a new branch
git checkout -f script_prototype
gedit download_and_compile.sh
# make changes
git commit -a
git push --set-upstream origin script_prototype

work on a branch

Making a change on branch: feature_logging

git branch -f feature_logging
gedit download_and_compile.sh
# make feature_logging
git commit -a
git push

More Details

Clear description of what is being built in the log file

When running the download_and_compile script, it would be nice if the version of FlightGear being installed is displayed early on (e.g. so that it appears at the top of the compilation_log.txt file). Is it easy to add this to the script? Or is there another obvious way to find the version? In the download_and_compile script the last stable revision is stated. But is this line always correct?

when connection is down

Great Job!!. Is there any way to dont download all the scenery DATA? it takes a long time, or even better, a way to resume the DATA download without lose what was downloaded before?.

fgbundle via rtorrent

fgbundle could be supported via rtorrent

Redhat/yum

Maybe integrate the instructions from CentOS once the scripts are merged ? Why wait. give it a shot. using next as a base. Be sure to let us know you're doing it so we can coordinate it with other changes.

Dependencies

Provide a separate step to download and install necessary tools and libs [1]

Building while downloading

for better use of bandwidth, it would make sense to also download while compiling (i.e. when building plib and osg, download simgear, when building simgear, download flightgear - and the fgdata checkout/update can also be initiated early). It might make sense to turn the shell script into a conventional makefile that supports dependency resolution and parallel steps, so that network-bound tasks like downloading (fgdata!) can be interleaved with the build process. The run_FOO scripts could then become conventional make targets.

• should we add a flightgear test run to the script?

Callahanp (talk) 11:39, 8 August 2013 (UTC)

The "test run" idea is not really new and very popular, and IIRC it was once a part of the old autotools system - meanwhile, a number of users have suggested some form of "testing framework" (via some custom/restricted startup mode built into FG), see FlightGear Headless and FlightGear Benchmark for a summary of the various discussions here. Basically, we are half-way there already, we really only need to make the GUI window optional and some additional subsystems that are currently enabled by default. Some core developers have been working on better regression testing support by making subsystems increasinly optional, see FlightGear Run Levels. Thus, a scripted test would already be possible to a certain degree, but may require additional customizations for certain scenarios. However, we shouldn't really be using any bash-magic here, because scripted test runs of FG would ideally be as platform-independent as FG itself, so that they could be run via CMake/CTest and/or the build server.--Hooray (talk) 00:17, 9 August 2013 (UTC)