Talk:Scripted Compilation on Linux Debian/Ubuntu: Difference between revisions

Jump to navigation Jump to search

Talk:Scripted Compilation on Linux Debian/Ubuntu (view source)

Revision as of 21:43, 7 July 2013

1,430 bytes removed ,  7 July 2013
no edit summary
No edit summary
No edit summary
Line 1: Line 1:


=== Development ===
=== Development ===
The main fgmeta repository script is currently at version 1.9.4 and it DOES NOT WORK without changes.  We need to test what's on Next in a variety of scenaria.


Development will be done in branches on https://www.gitorious.org/+fg-download-and-compile/fg/fg-download-and-compile-fgmeta
Development will be done in branches on https://www.gitorious.org/+fg-download-and-compile/fg/fg-download-and-compile-fgmeta


This is being set up starting with version 1.9.4, soon to be replaced by 1.9.10 and 1.9.11
<pre>
work is nearly complete on 1.9.12
mkdir ~/fgfs
cd ~/fgfs
wget http://www.gitorious.org/+fg-download-and-compile/fg/fg-download-and-compile-fgmeta/blobs/next/download_and_compile.sh
chmod 755 download_and_compile.sh
</pre>
:
:For those who can write bash scripts and want to help with development and testing, register as a  user on gitorious.org.  Then log in and send a message on gitorious to users fbrisa or pac1.  You can also contact me by registering on this wiki and sending a message throught the wiki.  The privileged few have my actual email address. 
:
 
At this point only the next branch of the repository is being used.  the version of the script there contains a number of new features, not all of which are complete and tested.
 
The biggest change is the addition of a directory management option.  This option allows efficient use of your disk space, your network connection, and gitorious.org bandwidth usage is kept to a minimum.
 
:-D 0 will cause the script to build flightgear in the same way as it always has.  Each directory you build in gets a complete set of sources, build and install objects, plus a copy of fgdata.
 
: -D 1 will cause the script to convert to using a single set of sources for different builds and to download fgdata only once.
 
to use -D 1,  simply download the script into a directory the usual way, but before building, create a sub-directory to contain each variety of build. Even if you are doing just one type of build, there is an advantage to using -D 1. For example, if you decide to completely rebuild, you can just remove the sub-directory and recreate it. The sources and fgdata are in the parent directory and remain in place
 
Once the -D 1 option is used, it will remain in effect as long as the fgdata and fgsources directories are present.
 
<pre>
mkdir ~/fgfs
cd ~/fgfs
wget http://www.gitorious.org/+fg-download-and-compile/fg/fg-download-and-compile-fgmeta/blobs/next/download_and_compile.sh
chmod 755 download_and_compile.sh
mkdir -p next
cd next
../download_and_compile.sh -D 1 all
</pre>
 
When the script completes, you will have
 
~/fgfs/download_and_compile.sh
~/fgfs/fgdata_2.11.0/fgdata
~/fgfs/fgsrc
~/fgfs/othersrc
~/fgfs/next/build etc
~/fgfs/next/install etc
 
Additional Changes:


There may be several feature branches as well to support some of the proposals below
# plib-1.8.5 and OpenSceneGraph-3.0.1 are not rebuilt after a successful build.
# you can build next, master, 2.10.0, 2.8.0 etc
# additional script options have been added.
# the window title shows what's being built and what stage things are at
# log file is now named differently and includes a timestamp


The main fgmeta repository is currently at version 1.9.4


To start, clone the team clone repository on gitorious.
To start working on the script, clone the team clone repository on gitorious.


I'll be posting other git instructions as I uncover them.
I'll be posting other git instructions as I uncover them.
Line 19: Line 64:
  git checkout -f 1.9.4 # for example
  git checkout -f 1.9.4 # for example


There are branches for versions 1.9.4 Through 1.9.12
On the team repository, there are branches for versions 1.9.4 Through 1.9.12 and master. ignore them.
These need to be populated with the correct versions (I don't have 1.9.5 through 1.9.9)
use next.
 
 
== Implementation Plan ==


#  Script and cmake debugging features has been included in 1.9-10 as options w, x and v
Additional requested features:
#  Create a team clone on [http://www.gitorious/org/fg/fgmeta specifically for work on this script.
#  Expanded help and labels for script sections is implemented in 1.9-10 and 1.9-11
#  Support for additional releases and tags is undergoing testing.  It will be relaased as 1.9.12
#  Instructions to allow the script to use a single copy of fgdata are working but not fully tested. This needs to be implemented in the script see [[Avoiding Multiple Downloads of fgdata]]


=== Proposals ===
#  Change the way directories are handled has been discussed, this has been prototyped and will be implemented in a future release
#  Change the way the log file is produced.
#  Add support for both stable and latest osg.  
#  Add support for both stable and latest osg.  
#  build while downloading
#  build while downloading
#  download fgdata as a restartable bundle
#  download fgdata as a restartable bundle
#  download all scenery (or maybe specific regions?)
#  download all scenery (or maybe specific regions?)
#  Add version of flightgear to be built at the top of the compiliation log
#  add version of flightgear being built to the window label
#  SUSE, Redhat/yum & CENTOS support
#  SUSE, Redhat/yum & CENTOS support
#  network setup tests for fgcom, multiplayer and terragear
#  network setup tests for fgcom, multiplayer and terragear
Line 55: Line 88:


# Cmake and makefiles can be difficult for the un-initiated.
# Cmake and makefiles can be difficult for the un-initiated.
How about a walkthrough using flightgear builds as an example?


Opinion:
Opinion:
Line 63: Line 98:


When running the download_and_compile script, it would be nice if the version of FlightGear being installed is displayed early on (e.g. so that it appears at the top of the compilation_log.txt file). Is it easy to add this to the script? Or is there another obvious way to find the version? In the download_and_compile script the last stable revision is stated. But is this line always correct?
When running the download_and_compile script, it would be nice if the version of FlightGear being installed is displayed early on (e.g. so that it appears at the top of the compilation_log.txt file). Is it easy to add this to the script? Or is there another obvious way to find the version? In the download_and_compile script the last stable revision is stated. But is this line always correct?
This is improved in the developemnt version.  Let us know if this is enough.


== when connection is down ==
== when connection is down ==
Line 72: Line 109:


== Redhat/yum ==
== Redhat/yum ==
Maybe integrate the instructions from [[CentOS]] once the scripts are meged ?
Maybe integrate the instructions from [[CentOS]] once the scripts are merged ?
Why wait.  give it a shot. using next as a base.  Be sure to let us know you're doing it so we can coordinate it with other changes.


== Dependencies ==
== Dependencies ==
Line 79: Line 117:
== Building while downloading ==
== Building while downloading ==
for better use of bandwidth, it would make sense to also download while compiling (i.e. when building plib and osg, download simgear, when building simgear, download flightgear - and the fgdata checkout/update can also be initiated early). It might make sense to turn the shell script into a conventional makefile that supports dependency resolution and parallel steps, so that network-bound tasks like downloading (fgdata!) can be interleaved with the build process. The run_FOO scripts could then become conventional make targets.
for better use of bandwidth, it would make sense to also download while compiling (i.e. when building plib and osg, download simgear, when building simgear, download flightgear - and the fgdata checkout/update can also be initiated early). It might make sense to turn the shell script into a conventional makefile that supports dependency resolution and parallel steps, so that network-bound tasks like downloading (fgdata!) can be interleaved with the build process. The run_FOO scripts could then become conventional make targets.
== Details of Implementation Plan Items ==
=== Clarify the use of and current position in the hierarchy of directories ===
As part of its work, the script must reposition itself in directories and refer to files in other directories. It does this at numerous points when building each of the 12 flightgear components.
The common method is to position to $CBD, the current build directory and then move to sub folders as needed, usually on level at a time. The script use cd .. to move back out of the sub directories.
Relevant directories include:
* $CBD - The "Current Build Directory"
* a directory one level above the git top level for a component
* the directory contaiing the top level of the git archive
* the top level of sources for the component (usually the same as the git archive
* a build directory for all components
* an install directory for all components
* a build directory for a specific component (12 of them)
* an install directory for a specific component (12 of them)
* a directory for fgdata
I'm proposing that we change this by adding to the  Script Section: Create Required Directories.
This section currently just ensures that there is an install directory.  I'd expand this to define variable names for each directory in the list above and use the variable names for directory navigation in the sections that build and install the components. 
=== Ensure that only one download of fgdata, plib and osg is done and then used for all builds.  ===
Provide a standard place to put one or more versions of fgdata, and a single place for plib and osg etc.  This would be immplemented using symbolic links as described in [[Avoiding Multiple Downloads of fgdata]]
=== Allow builds for all branches and recent versions ===
Currently the script can build next, or a single released version.  Changes to  allow this via the B (Branch) and R (revision) options are being tested for script version 1.9.12. 
=== Keep the build and install folders and the log files separate ===
For each branch, version and set of options affecting what is built keep a separate set of build and install folders.
Since the script is now capable of building several branches and versions, each with a selectable set of options that affect the build, it makes sense to be able to keep these various builds for usage and testing.
In the scheme for assigning variables to represent all directories, the build and install top levels can be changed from just build and install to names that describe specific builds and installs.  For example: build_master_ei to hold the build directories for openrti, simgear and fgfs when building from the master branch with options e & i.
=== Label log files by changing their file names to indicate what was built and when. ===
This is straightforward enough, just delay logging until all parameters are processed.
But there's an additional thing that might be considered.  After the log file is determined, start a sub-process with output to the log file, pass it the same parameters as were passed in,  then exit  This would allow us to remove constructs like |tee $LOGFile and >> $LOGFILE.  When x and v are chosen, it provides a log of all bash steps. (nice for debugging the script)
Callahanp
982

edits

Retrieved from "https://wiki.flightgear.org/Special:MobileDiff/61299"

Navigation menu