Talk:Scripted Compilation on Linux Debian/Ubuntu: Difference between revisions

Jump to navigation Jump to search

Talk:Scripted Compilation on Linux Debian/Ubuntu (view source)

Revision as of 23:06, 1 September 2014

13,900 bytes removed ,  1 September 2014
removed references to stuff that is not going to happen.
(removed references to stuff that is not going to happen.)
Line 2: Line 2:
== Current Status of development ==
== Current Status of development ==


team clone: fg-download-and-compile-fgmeta.git is being removed until there is an actual team. ;-P
* FGMeta master is currently an old script version that is not maintained
* fgmeta next is currently script version and will build 3.2.0 if -s is specified


* FGMeta master is currently script version 1.9.14
* fgmeta next is currently script version 1.9.13 and will have 1.9.15 soon.
* Team clone branches
** master is a copy of what is on fgmeta
1.9.15 will contain one or more of the following:
*** version number comparison.  is version 3.10.0 higher than 3.9.0 or lower?  this is needed to allow version to be specified as "3.9.0 or higher".
*** cleanup task:  for some reason, the local .git repostiories and sources are placed in directories such as simgear/simgear, fgfs/flightgear, openrti/openrti.  Why?  Why not just simgear, flightgear and openrti?
*** cleanup task:  the script uses cd to move up and down the source/build/install directories  as an alternative, define variable values containing full paths and cd only to the specific directory needed to do a particular task. plib and osg have gotten this treatment already.
*** cleanup task:  lowercase all variables that are not used as environment variables.  This is the current accepted practice for bash scripts.
*** cleanup task:  use a "Here Document" for help
*** cleanup task:  get rid of all that my_slash stuff in the fgx section
*** cleanup task:  in the component build sections use cd to move between directories using full path names where needed.  remove relative cd commands. remove necessity to return to current build directory at the end of each component.
*** Default versions of OSG for each version of flightgear
** feature_modify_fltk. produce a script to modify the paths in ~/fltk/flightgear/org/fgrun.prefs to point to the current directory
** feature_symlinks
*** use installed OSG or plib
*** allow use of specified boost or fltk library version
*** allow use of a specified version of just about anything.
*** Allow a single downloaded set of sources and fgdata to support multiple builds.  There are two ways to do this:  Keep a single .git and checkout directory for each component, using them one at a time or keep a single .git and checkout version per branch.  The second option allows work on more than one branch at a time.
*** Eclipse integration
*** Netbeans integration
*** Codeblocks integration
*** Your favorite IDE integration
** feature_fgmeta_cmake
*** an implementation that makes use of the fgmeta cmake super build
People on #bash have had a look at the script a few months ago and made several good suggestions that have made their way into the maintenance list for the script above.


=== Script Overview  -- Discussion of why the script was originally developed and where it might go from here ===
=== Script Overview  -- Discussion of why the script was originally developed and where it might go from here ===
Line 72: Line 42:
## configure
## configure
## build and Install of components
## build and Install of components
=== Script implementation notes -- How the script implements features ===
The script currently uses bash associative arrays for determining the version to download and build.
once selected, the version information is stored in bash variables
bash variables are used to
# hold a list of components to build
# specify component container, source, build and install directories.
# determine if
## an update of sources from upstream repositories is desired
## reconfigure is desired
## the build should produce a debug version
##  set flags for debug, build options for use of openrti
The script will build everything in the directory it is run from.
The script does not need to be in the directory you are building in.  If you build separate versions, the current version of the script will download everything for each version.  to avoid this, you can set up symbolic links to sources and to the correct version of fgdata before running the script.  The script has built in support for this, but you have to use a specific directory structure to take advantage of it:
The following directory structure will support building multiple versions and variations on builds for flightgear with minimal use of network resources
* <your working directory for building flightgear>
** fgdata-<version>
*** fgdata
** fgdata-<2nd version>
*** fgdata
** fgsrc
*** fgfs/flightgear
*** openrti/openrti
*** simgear/simgear
** othersrc
*** OpenSceneGraph
**** OpenSceneGraph-3.0.1
**** OpenSceneGraph-3.1.0
**** OpenSceneGraph-3.2.0
*** plib-1.8.5
**** plib
** download_and_compile.sh
** stable
*** build
*** install
** 2.12.0
*** build
*** install
** 3.0.0
*** build
*** install
** master
*** build
*** install
** next
*** build
*** install
If you setup the following directories and file on first use, the script will use them when you  run ../download_and_compile.sh from stable, next, master, 2.12.0, 3.0,0 or stable.
* <your working directory for building flightgear>
** download_and_compile.sh
** stable
** next
** master
** 2.12.0
** 3.0.0
Example:
cd  <your working directory for building flightgear>
wget download_and_compile.sh wget https://www.gitorious.org/fg/fgmeta/raw/master:download_and_compile.sh
mkdir -p stable
mkdir -p next
cd stable
../download_and_compile.sh -sxv
cd ../next
../download_and_compile.sh -p n -xv
== Future Script Versions ==
=== Alternative Coding approaches  to implement the features -- Ways to structure the script so it can be easily understood and maintained===
'''Use bash associative arrays to hold information relevant to each component.'''
https://www.gitorious.org/fg/fg-download-and-compile-fgmeta  select branch: feature/bash-associative-arrays
The information needed for the work to be done on each component is similar.  We could create a bash associative array for each component with items for each piece of information required:
* process this component or not
* git, svn, zip or other file url
* branch and revision information
* where to put sources, build and install
* which version of fgdata to use
* additional configuration options
* get original sources Y/N
* update sources Y/N
* configure or reconfigure Y/N
* build  Y/N
* install  Y/N
===Preventing multiple downloads of fgdata, or other sources such as OSG===
This  allows a number of complex build scenarios to be tried without placing an unnecessary burden on gitorious and other source archives, not to mention your own network bandwidth.
Unless the user chooses to use the new method, everything just stays where it is in each build tree.
The method relies on the existance of named directories to trigger the new behavior. Currently no parameters control this, and there is no means of converting between an existing structure and the new one.
Previously, you could just download the script and in the same directory, just run it. To use the new scheme, you download the script as usual, and create several sub-directories
to hold sources for OSG and plib, one or more version of fgdata, and builds for stable, next and specific versions
FG-git
  + download_and_compile.sh
  + othersrc
  + fgdata-3.0.0
  + fgdata-2.12.1
  + next
  + 2.12.1 or stable
  + 3.0.0
You can populate the directories with previously downloaded svn and .git repositories, or rely on the script to populate them for you.
othersrc is for packages that are required for flightgear but are not part of flightgear per se.  This includes OSG and plib, but not openrti, which is handled as if it were part of flightgear.
FG-git
  + othersrc
    +OpenSceneGraph-3.0.1
    +OpenSceneGraph-3.2.0
    +plib-1.8.5
  +fgdata-3.0.0
    +fgdata
  +fgdata-2.12.1
    +fgdata
  +fgdata-3.1.0
    +fgdata
===Special Handling for fgdata===
Because of its size, fgdata is handled differently with regards to symlinks.  The script will analyze the options, existing hyperlinks and available previously downloaded fgdata versions and use a selected strategy based on the option value and on what sources and data are already there.
{| class=wikitable
! -L option
! Interpretation
|-
| none || Don't use symlinks, unless directories to support them are present
|-
| -L 0 || Don't use symlinks, unless used previously by the current build
|-
| -L 1 || Use symlinks to fgdata-git and checkout the required version
|-
| -L 2 || Use symlinks to fgdata-version. If fgdata-version does not exist, copy locally if available, otherwise download
|}
{| class=wikitable
! Scenario
! -L option
! install/fgdata contains
! Analysis Result
! Strategy
|-
| 1 || -L 2 ||anything or nothing||forces the use of a symlink to fgdata-version
No analysis needed
|Apply Rule 1
|-
| 2 || -L 1 ||anything or nothing ||forces the use of a symlink to fgdata-git
No analysis needed
|Apply Rule 2
|-
| 3 || -L 0 ||non-symlinked fgdata or install/fgdata does not exist || The option forces the original script behavior || No action required. No symlink used
|-
| 4 || -L 0 ||symlinked to fgdata-git || We're all set || No action required. Use the existing symlink
|-
| 5 || -L 0 ||symlinked to fgdata-version and the version is the one required || We're all set ||No action required. Use the existing symlink
|-
| 6 || -L 0 ||symlinked to fgdata-version and the version is <u>not</u> the one required ||Use fgdata version copies|| Change the symlink. Apply rule 1.
|-
| 7 || none ||non-symlinked fgdata or nothing || an fgdata-git or fgdata-version does exist || Symlink fgdata-git or required fgdata-version based on rule 0
|-
| 8 || none ||non-symlinked fgdata or nothing || no fgdata-git or fgdata-version exists    || No action required. No symlink used
|}
{| class=wikitable
! Rule #
! Rule
|-
| 0 || ''Decide which of rules 1 or 2 to use when -L is its default value -1''
* if any fgdata-version exists, use rule 1
* otherwise use rule 2
|-
| 1 || ''Use separate directories for each fgdata version''
* create fgdata-version directory if it does not exist
* if fgdata-version contains no fgdata, and <u>any</u> can be found, copy it into fgdata-version and leave it. Later we will set it to the correct version
* delete the current install/fgdata, directory or symlink
* create the symlink to fgdata-version as install/fgdata
|-
| 2 || ''Use a single directory and one copy of fgdata for each fgdata version as needed''
* create fgdata-git directory if it does not exist
* if fgdata-git contains no fgdata, and <u>any</u> can be found, copy or move it into fgdata-git and leave it. Later we will set it to the correct version
* delete the current install/fgdata, directory or symlink
* create the symlink to fgdata-git as install/fgdata
|}
If implemented properly, this should support the approach for using an fgmeta cmake described below.  The script workflow would be:
* interpret options needed to setup logging
* setup logging by recursively calling the script
* interpret all options
* do development environment tasks such as apt-get update and apt-get installs for development packages
* analyze situation with regard to usage of symlinks
* setup commands related to symlinks as an array of commands
* execute the symlink commands
* setup commands needed to establish the values of variables to drive cmake and make
* setup the cmake commands
* execute cmake and make commands
Our current usage of cmake creates a makefile that is then run using make.  I believe there is an alternative approach that does the build directly in the cmake step.  If that's true, we could make the build <u>very</u> simple


[[User:Callahanp|Callahanp]] ([[User talk:Callahanp|talk]]) 17:37, 17 November 2013 (UTC)
[[User:Callahanp|Callahanp]] ([[User talk:Callahanp|talk]]) 17:37, 17 November 2013 (UTC)
=== makeffile and cmake idea (needs discussion) ===
This is currently under development as download_and_compile.sh version 2.  This version will have the same parameter and option handling, but the sections of the script that download and build the sources will be much simpler.  The script may just output a specialized build script with no logic in it and execute it.  The output scripts would be usable from the download_and_compile script or could be used independently to do a specific build. [[User:Callahanp|Callahanp]] ([[User talk:Callahanp|talk]]) 17:37, 17 November 2013 (UTC)


{{cquote|<nowiki>I also did some work (and pushed it) to support CMake superbuilds using fgmeta. It works on Mac and Windows, and probably on Linux with some small tweaks, so this might be a way to simplify future versions of the download_and_compile script, since most of the steps (download & build OSG, download and build PLIB) are handled by the super-build.</nowiki><ref>{{cite web |url=http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg40142.html|title=<nowiki>Re: [Flightgear-devel] new team clone on gitorious.org specifically for download_and_compile.</nowiki>|author=<nowiki>James Turner</nowiki>|date=<nowiki>Sat, 01 Jun 2013 02:16:43 -0700</nowiki>}}</ref>|<nowiki>James Turner</nowiki>}}
{{cquote|<nowiki>I also did some work (and pushed it) to support CMake superbuilds using fgmeta. It works on Mac and Windows, and probably on Linux with some small tweaks, so this might be a way to simplify future versions of the download_and_compile script, since most of the steps (download & build OSG, download and build PLIB) are handled by the super-build.</nowiki><ref>{{cite web |url=http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg40142.html|title=<nowiki>Re: [Flightgear-devel] new team clone on gitorious.org specifically for download_and_compile.</nowiki>|author=<nowiki>James Turner</nowiki>|date=<nowiki>Sat, 01 Jun 2013 02:16:43 -0700</nowiki>}}</ref>|<nowiki>James Turner</nowiki>}}
Line 358: Line 115:


When running the download_and_compile script, it would be nice if the version of FlightGear being installed is displayed early on (e.g. so that it appears at the top of the compilation_log.txt file). Is it easy to add this to the script? Or is there another obvious way to find the version? In the download_and_compile script the last stable revision is stated. But is this line always correct?
When running the download_and_compile script, it would be nice if the version of FlightGear being installed is displayed early on (e.g. so that it appears at the top of the compilation_log.txt file). Is it easy to add this to the script? Or is there another obvious way to find the version? In the download_and_compile script the last stable revision is stated. But is this line always correct?
This is improved in the developemnt version.  Let us know if this is enough.


== when connection is down ==
== when connection is down ==
Line 387: Line 142:


<references/>
<references/>
==Options and Parameters for a new Version download_and_compile==
Revamp options and parameters.
Here's the kinds of things we have in options and parameters at this point
{| class=wikitable
! value
! meaning
|-
| colspan="2"  |'''Development Environment'''
|-
| -a: [y or n]  || apt-get update
|-
| -p: [y or n] || download install required development packages
|-
| colspan="2" |'''Obtain Sources'''
|-
| -d: [y or n] || download sources
*Update the local repository from upstream
*Update sources from local repository
|-
| -m || Create separate local repositories for each source version '''(new)'''
|-
| colspan="2" |'''Configuration'''
|-
| -r: [y or n] || configure or reconfigure
|-
| -w || Verbose makefile option
|-
|colspan="2"|'''Compile and Build'''
|-
| -c: [y or n] || do the compilation and building
|-
| -j: n|| Threads for the make command. Defaults to Number of Processors + 1 '''(new)'''
|-
| -O: n|| Optimization level for g++
|-
| -g|| Compile with debug| append a d or -d to any component name listed in parameters and that one will be built with debug
|-
|colspan="2"|'''Script Behavior'''
|-
| -h|| display help
|-
| -x ||Sets -x in the script for script debugging
|-
| -v ||Sets -v in the script for script debugging
|-
|colspan="2"|'''Deprecated'''
|-e ||Build with new event model||Is this still needed?
|-
|-
| -u || Update Mode - supress reconfigure.
*This feature is deprecated because -r n will do the same thing.
|-
| -B: ||Branch
*This feature is deprecated because branch is not specified as a parameter
|-
| -R: ||Revision
*This feature is not needed as a separate item Revision can be specified in the parameters
|-
|-G: || version no for osg
*This feature is not needed - version of osg can be specified in the parameters
|-
| -z || Stop after one module was added to allow easier debugging by preventing the script from writing too much -x and -v output.  The extra output made errors harder to identify. 
*More robust error handling and reporting in the script would make this unnecessary.
|}
*Parameters tell what things will be included and which version to build
**all
**plib
**osg
**osg-version
**events
**openrti
**simgear
**flightgear or fgfs
**stable
**2.12.1 etc
**master
**next
[[User:Callahanp|Callahanp]] ([[User talk:Callahanp|talk]]) 04:46, 17 November 2013 (UTC)
Callahanp
982

edits

Retrieved from "https://wiki.flightgear.org/Special:MobileDiff/75817"

Navigation menu