Talk:Scripted Compilation on Linux Debian/Ubuntu: Difference between revisions

Jump to navigation Jump to search

Talk:Scripted Compilation on Linux Debian/Ubuntu (view source)

Revision as of 17:37, 17 November 2013

2,894 bytes added ,  17 November 2013
no edit summary
No edit summary
Line 2: Line 2:
== Current Status of development on the team clone: fg-download-and-compile-fgmeta.git ==
== Current Status of development on the team clone: fg-download-and-compile-fgmeta.git ==


* Team clone master branch: 1.9.12 - needs additional testing by more users
* FGMeta master and next currently contain script version 1.9.12
* Team clone next does not contain anything that will be released
* Team clone branches
* Team clone 1.9.13 branch soon to be deleted and replaced with a "feature branch": feature_logging
** master is a copy of what is on fgmeta
* Team clone 1.9.14 branch soon to be deleted and replaced with a "feature branch": feature_directories
** feature_logging - changes related to logging are extensive
*** removal of | tee -a $LOGFILE, 2>&1 | tee -a $LOGFILE and  >> $LOGFILE
*** addition of a section to recursively call the script with original options and parameters plus 2>&1 | tee -a $LOGFILE
*** renaming the log file: download_and_compile.log
*** addition of a separate log file download_and_compile_summary.log containing only time stamped results of build steps
** feature_symlinks
*** A robust implementation of directory management allowing a single downloaded set of sources and fgdata to support multiple builds
** feature_fgmeta_cmake
*** an implementation that makes use of the fgmeta cmake super build


fg-download-and-compile-fgmeta.git is currently a one man show.  YOU could make it a real team.  Additional testers and coders welcome.


=== Script Overview  -- Discussion of why the script was originally developed and where it might go from here ===
=== Script Overview  -- Discussion of why the script was originally developed and where it might go from here ===


The download_and_compile.sh script was originally developed as an easy way to get the latest flightgear features in the hands of non-developers and to entice new developers to the project with an easy way to get a workable build environment in place.
The download_and_compile.sh script was originally developed as an easy way to get the latest flightgear features in the hands of non-developers and to entice new developers to the project with an easy way to get a workable build environment in place. Suggestions have been made to move the script in the direction of a more general purpose tool that might also support work on build servers.
 
=== Script Requirements -- in general terms, what the script should produce ===
=== Script Requirements -- in general terms, what the script should produce ===


Line 18: Line 28:
# download or update sources as needed
# download or update sources as needed
# build any recent version of flightgear and associated launchers and utilities
# build any recent version of flightgear and associated launchers and utilities
# build different variations of flightgear such as with a different event handler, with openrti or compilation with the -g (debug) option.
# download fgdata  
# download fgdata  
# set of scripts to start flightgear, launchers and utilities
# set of scripts to start flightgear, launchers and utilities
Line 24: Line 35:
# minimal load on network resources
# minimal load on network resources
# support for building with different versions of underlying packages such as boost, OSG, etc
# support for building with different versions of underlying packages such as boost, OSG, etc
# logging of build steps


=== Current Script Features ===
=== Current Script Features ===
Line 55: Line 67:


The script will build everything in the directory it is run from.
The script will build everything in the directory it is run from.
The script does not need to be in the directory you are building in.  If you build separate versions, the current version of the script will download everything for each version.  to avoid this, you can set up symbolic links to sources and to the correct version of fgdata.
The script does not need to be in the directory you are building in.  If you build separate versions, the current version of the script will download everything for each version.  to avoid this, you can set up symbolic links to sources and to the correct version of fgdata before running the script.


== Future Script Versions ==
== Future Script Versions ==
Line 87: Line 99:
Another would be to have an additional log files with timestamped log entries indicating what is to be done, what has completed successfully and what exactly failed.
Another would be to have an additional log files with timestamped log entries indicating what is to be done, what has completed successfully and what exactly failed.


'''Preventing multiple downloads of fgdata, or other sources such as OSG'''
===Preventing multiple downloads of fgdata, or other sources such as OSG===


https://www.gitorious.org/fg/fg-download-and-compile-fgmeta  select branch: feature/directories
https://www.gitorious.org/fg/fg-download-and-compile-fgmeta  select branch: feature/directories
Line 102: Line 114:


Alternatives:
Alternatives:
* Use an option to drive the creation and re-creation of hyperlinks
* Use an option to drive the creation and re-creation of symlinks
** No option, build as usual, no symbolic links are created, <u>but will be used if already in place for the build in progress</u>
** No option, build as usual, no symbolic links are created, <u>but will be used if already in place for the build in progress</u>
** -L 0 same as No Option
** -L 0 same as No Option
** -L 1 Use hyperlinks for a single copy of sources.  Checkouts will be used to set the sources to the correct version.
** -L 1 Use hyperlinks for a single copy of sources.  Checkouts will be used to set the sources to the correct version.
** -L 2 Use hyperlinks for multiple copies of sources, one per version.
** -L 2 Use hyperlinks for multiple copies of sources, one per version.
===Special Handling for fgdata===


==Special Handling for fgdata==
Because of its size, fgdata is handled differently with regards to symlinks.  The script will analyze the options, existing hyperlinks and available previously downloaded fgdata versions and use a selected strategy
 
Because of its size, fgdata is handled differently with regards to hyperlinks.  The script will analyze the options, existing hyperlinks and available previously downloaded fgdata versions and use a selected strategy
{| class=wikitable
{| class=wikitable
! -L option
! -L option
Line 129: Line 140:
! Analysis Result
! Analysis Result
! Strategy
! Strategy
|-
| 1 || -L 2 ||forces the use of a symlink to fgdata-version||No analysis needed||Apply Rule 1
|-
| 2 || -L 1 ||forces the use of a symlink to fgdata-git||No analysis needed||Apply Rule 2
|-
| 3 || -L 0 ||non-symlinked fgdata or install/fgdata does not exist || The option forces the original script behavior with respect to both sources and fgdata || No action required. No symlink used
|-
| 4 || -L 0 ||symlinked to fgdata-git || We're all set || No action required. Use the existing symlink
{-
{-
| 1 || none ||non-symlinked fgdata || no fgdata-git or fgdata-version exists|| No action required. No symlink used
| 5 || -L 0 ||symlinked to fgdata-version and the version is the one required || We're all set ||No action required. Use the existingsymlink
|-
|-
| 2 || none ||non-symlinked fgdata || an fgdata-git or fgdata-version does exist|| Symlink fgdata-git or required fgdata-version based on rule 1
| 6 || -L 0 ||symlinked to fgdata-version and the version is <u>not</u>the one requiried || Change the symlink. Apply rule 1.
|-
|-
| 3 || -L 0 ||non-symlinked fgdata or does not exist || No action required. No symlink used
| 7 || none ||non-symlinked fgdata || an fgdata-git or fgdata-version does exist || Symlink fgdata-git or required fgdata-version based on rule 0
|-
|-
| 4 || -L 0 ||symlinked to fgdata-git || No action required. Use the symlink
| 8 || none ||non-symlinked fgdata || no fgdata-git or fgdata-version exists    || No action required. No symlink used
{-
|}
| 5 || -L 0 ||symlinked to fgdata-version and the version is the one requiried || No action required. Use the symlink
{| class=wikitable
! Rule #
! Rule
|-
| 0 || ''Decide which of rules 1 or 2 to use when -L is its default value -1''
* if any fgdata-version exists, use rule 1
* otherwise use rule 2
|-
|-
| 5 || -L 0 ||symlinked to fgdata-version and the version is <u>not</u>the one requiried || Change the symlink. Apply rule 2.
| 1 || ''Use separate directories for each fgdata version''
* create fgdata-version directory if it does not exist
* if fgdata-version contains no fgdata, and <u>any</u> can be found, copy it into fgdata-version and leave it. Later we will set it to the correct version
* delete the current install/fgdata, directory or symlink
* create the symlink to fgdata-version as install/fgdata
|-
|-
| 3 || -L 1 ||The current build contains
| 2 || ''Use a single directory and one copy of fgdata for each fgdata version as needed''
* If an fgdata specific to the required version is present, it will be used.
* create fgdata-git directory if it does not exist
* If an fgdata specific to the required version is not present and fgdata-git is present, fgdata-git will be used with checkout to the required version.
* if fgdata-git contains no fgdata, and <u>any</u> can be found, copy or move it into fgdata-git and leave it. Later we will set it to the correct version
* If an fgdata specific to another version is present, and an fgdata-git version is not present an fgdata specific to the required version will be created using a copy of the existing version and a checkout to the required version on the copy
* delete the current install/fgdata, directory or symlink
* If no fgdata is present, and -L0 or L1 is specified, fgdata-git will be created and used
* create the symlink to fgdata-git as install/fgdata
* If no fgdata is present, and -L0 or L1 is specified, fgdata-git will be created and used
|}
|}


If implemented properly, this should support the approach for using an fgmeta cmake described below.  The script workflow would be:
* interpret options needed to setup logging
* setup logging by recursively calling the script
* interpret all options
* do development environment tasks such as apt-get update and apt-get installs for development packages
* analyze situation with regard to usage of symlinks
* setup commands related to symlinks as an array of commands
* execute the symlink commands
* setup commands needed to establish the values of variables to drive cmake and make
* setup the cmake commands
* execute cmake and make commands
Our current usage of cmake creates a makefile that is then run using make.  I believe there is an alternative approach that does the build directly in the cmake step.  If that's true, we could make the build <u>very</u> simple
[[User:Callahanp|Callahanp]] ([[User talk:Callahanp|talk]]) 17:37, 17 November 2013 (UTC)
=== makeffile and cmake idea (needs discussion) ===
This is currently under development as download_and_compile.sh version 2.  This version will have the same parameter and option handling, but the sections of the script that download and build the sources will be much simpler.  The script may just output a specialized build script with no logic in it and execute it.  The output scripts would be usable from the download_and_compile script or could be used independently to do a specific build. [[User:Callahanp|Callahanp]] ([[User talk:Callahanp|talk]]) 17:37, 17 November 2013 (UTC)
{{cquote|<nowiki>I also did some work (and pushed it) to support CMake superbuilds using fgmeta. It works on Mac and Windows, and probably on Linux with some small tweaks, so this might be a way to simplify future versions of the download_and_compile script, since most of the steps (download & build OSG, download and build PLIB) are handled by the super-build.</nowiki><ref>{{cite web |url=http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg40142.html|title=<nowiki>Re: [Flightgear-devel] new team clone on gitorious.org specifically for download_and_compile.</nowiki>|author=<nowiki>James Turner</nowiki>|date=<nowiki>Sat, 01 Jun 2013 02:16:43 -0700</nowiki>}}</ref>|<nowiki>James Turner</nowiki>}}
# use of makefile or cmake as a basis for or as part of the script.
Advantages:
# Cross Platform.
# fits with general direction things seem to be moving in.
Disadvantages:
# Cmake and makefiles can be difficult for the un-initiated.
How about a walkthrough using flightgear builds as an example?


'''Let CMake drive the process'''
Opinion:


Not sure how to do this yet.  Thoughts? [[User:Callahanp|Callahanp]] ([[User talk:Callahanp|talk]]) 23:02, 30 September 2013 (UTC)
# This is a great idea, Should it be a separate alternative script? [[User:Callahanp|Callahanp]] 11:34, 31 May 2013 (UTC)


: Agreed, the cmake-based approach sounds really sensible, and would also help unify the build infrastructure,i.e. regarding the [[FlightGear Build Server]], to make the setup better reproducable, but also allow CI to be set up locally, i.e. as part of NetBeans or Eclipse. The question is which parts of the script could be reasonably moved into some form of CMake/superbuild script - certain things are highly distro/OS/platform specific. And I'm not sure just how flexible cmake is here ?
=== Proposed features -- New features of the script that have yet to be implemented ===
=== Proposed features -- New features of the script that have yet to be implemented ===


Line 202: Line 263:


== More Details ==  
== More Details ==  
=== makeffile and cmake idea (needs discussion) ===
{{cquote|<nowiki>I also did some work (and pushed it) to support CMake superbuilds using fgmeta. It works on Mac and Windows, and probably on Linux with some small tweaks, so this might be a way to simplify future versions of the download_and_compile script, since most of the steps (download & build OSG, download and build PLIB) are handled by the super-build.</nowiki><ref>{{cite web |url=http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg40142.html|title=<nowiki>Re: [Flightgear-devel] new team clone on gitorious.org specifically for download_and_compile.</nowiki>|author=<nowiki>James Turner</nowiki>|date=<nowiki>Sat, 01 Jun 2013 02:16:43 -0700</nowiki>}}</ref>|<nowiki>James Turner</nowiki>}}
# use of makefile or cmake as a basis for or as part of the script.
Advantages:
# Cross Platform.
# fits with general direction things seem to be moving in.
Disadvantages:
# Cmake and makefiles can be difficult for the un-initiated.
How about a walkthrough using flightgear builds as an example?
Opinion:
# This is a great idea, Should it be a separate alternative script? [[User:Callahanp|Callahanp]] 11:34, 31 May 2013 (UTC)
: Agreed, the cmake-based approach sounds really sensible, and would also help unify the build infrastructure,i.e. regarding the [[FlightGear Build Server]], to make the setup better reproducable, but also allow CI to be set up locally, i.e. as part of NetBeans or Eclipse. The question is which parts of the script could be reasonably moved into some form of CMake/superbuild script - certain things are highly distro/OS/platform specific. And I'm not sure just how flexible cmake is here ?


=== Clear description of what is being built in the log file ===
=== Clear description of what is being built in the log file ===
Callahanp
982

edits

Retrieved from "https://wiki.flightgear.org/Special:MobileDiff/64643"

Navigation menu