Avoiding multiple downloads of FGData on Linux: Difference between revisions
(cat: Core developer documentation; Spelling; Headings; Copy editing) |
m (Johan G moved page Avoiding Multiple Downloads of fgdata to Avoiding multiple downloads of FGData on Linux: Non-camel case title; Clarifying that this is on linux) |
||
(2 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
When building different versions of FlightGear, its important to note that each version has its own version of FGData, and FlightGear will check the version to ensure it is compatible. | |||
Since [[$FG_ROOT#$FG_DATA|FGData]] is so large (At least 1.2 GB as of 07/06/15), it makes sense to download the whole FGData only once. You can keep versions of it separately from the install tree that download_and_compile.sh creates and create symbolic links to them before using the script. That way when you decide to rebuild from scratch, at least the FGData will remain. | |||
The <code>download_and_compile.sh</code> script can build either the next/master branch, or the stable version of [[FlightGear]]. | |||
The instructions below tell how to do this for existing or new uses of download_and_compile.sh. The technical details are not dependent on where you put your builds. You can use the directory scheme I use or your own existing scheme. | The instructions below tell how to do this for existing or new uses of <code>download_and_compile.sh</code>. The technical details are not dependent on where you put your builds. You can use the directory scheme I use or your own existing scheme. | ||
The concept as a whole works great. I've been using it for months. The instructions are a work in progress and have yet to be fully tested. | The concept as a whole works great. I've been using it for months. The instructions are a work in progress and have yet to be fully tested. | ||
== The main ideas == | == The main ideas == | ||
* You may only need one copy of each version of | * You may only need one copy of each version of FGData you are working with | ||
* | * FGData normally lives as a sub-directory in <code>install/fgfs/</code> | ||
* Once it exists, install/fgdata can moved and replaced by a symbolic link to | * Once it exists, <code>install/fgdata</code> can moved and replaced by a symbolic link to FGData's real directory location | ||
== Technical | == Technical points and command examples == | ||
* <code>mkdir -p install/fgfs</code> will create the install directory and subdirectory if either does not yet exist | * <code>mkdir -p install/fgfs</code> will create the install directory and subdirectory if either does not yet exist | ||
* <code>mv</code> can move a directory | * <code>mv</code> can move a directory | ||
Line 20: | Line 20: | ||
* <code>ln -s targetname linkname</code> creates linkname as a symbolic link to the target. The target may or may not yet exist and that's normally ok. Not in this case, because of the way we create the directory with git. | * <code>ln -s targetname linkname</code> creates linkname as a symbolic link to the target. The target may or may not yet exist and that's normally ok. Not in this case, because of the way we create the directory with git. | ||
* The symbolic link must be created <b>after</b> downloading | * The symbolic link must be created <b>after</b> downloading | ||
* Important: You can move the FlightGear source directories and download_and_compile.sh | * Important: You can move the FlightGear source directories and <code>download_and_compile.sh</code> to different folders at will. Git will not care. All the Git specific files move right along with the sources. Its the same for SVN. However, the same does not hold true for the files created during a build by cmake. The build and install folders cannot be moved because the files in them have hard coded paths in some cases. They'll need to be deleted and rebuilt. | ||
== Background info == | == Background info == | ||
I do my builds of different versions and options in separate sub-directories under ~/work/fg. Sometimes I'll create an extra tree for debugging purposes. I don't want to have to copy all of | I do my builds of different versions and options in separate sub-directories under <code>~/work/fg</code>. Sometimes I'll create an extra tree for debugging purposes. I don't want to have to copy all of FGData just to do the work on a debug version. I also copy the sources for the various components to avoid re downloading those as well. In download_and_compile.sh, Git takes care of any changes needed to switch to between master and stable as needed. | ||
== Here's how I handle | == Here's how I handle FGData... == | ||
Normally fgdata lands in install/fgfs. | Normally fgdata lands in <code>install/fgfs</code>. | ||
I've moved my 2.10.0 and 2.11.0 | I've moved my 2.10.0 and 2.11.0 FGData versions and replaced them with symbolic links to the new location as needed. | ||
Its as simple as that. | Its as simple as that. | ||
== Detailed directions == | == Detailed directions == | ||
In short the steps are: | In short the steps are: | ||
# Create a directory for each version of | # Create a directory for each version of FGData you are going to use | ||
# Create a directory for each version or variation of FlightGear you want to build | # Create a directory for each version or variation of FlightGear you want to build | ||
# Do you first build of FlightGear | # Do you first build of FlightGear | ||
# Retrofit previous use of download_and_compile.sh | # Retrofit previous use of <code>download_and_compile.sh</code> | ||
# Prepare to build another version | # Prepare to build another version | ||
# Building with options | # Building with options | ||
=== Create a directory for each version of | === Create a directory for each version of FGData you are going to use === | ||
mkdir -p ~/work/fg/fgdata_2.10.0 | mkdir -p ~/work/fg/fgdata_2.10.0 | ||
mkdir -p ~/work/fg/fgdata_2.11.0 | mkdir -p ~/work/fg/fgdata_2.11.0 | ||
Eventually a particular version of | Eventually a particular version of FGData will be in each | ||
=== Create a directory for each version or variation of FlightGear you want to build === | === Create a directory for each version or variation of FlightGear you want to build === | ||
download_and_compile.sh will be run from these directories | <code>download_and_compile.sh</code> will be run from these directories | ||
Mine are sub-directories of <code>~/work/fg</code> for example: | |||
mkdir -p ~/work/fg/master | mkdir -p ~/work/fg/master | ||
mkdir -p ~/work/fg/stable | mkdir -p ~/work/fg/stable | ||
mkdir -p ~/work/fg/master_debug | mkdir -p ~/work/fg/master_debug | ||
mkdir -p ~/work/fg/master_debug_rti | mkdir -p ~/work/fg/master_debug_rti | ||
Note that I never build at the ~/work/fg level. download_and_compile.sh is always run from a sub-directory | Note that I never build at the <code>~/work/fg level</code>. <code>download_and_compile.sh</code> is always run from a sub-directory | ||
=== Do you first build of | === Do you first build of FlightGear === | ||
You can skip this step if you've already done one. | You can skip this step if you've already done one. | ||
If you've never used download_and_compile.sh before you're ready to go. | If you've never used <code>download_and_compile.sh</code> before you're ready to go. | ||
cp download_and_compile.sh ~/work/fg/master | cp download_and_compile.sh ~/work/fg/master | ||
cd ~/work/fg/master | cd ~/work/fg/master | ||
./download_and_compile.sh ALL | ./download_and_compile.sh ALL | ||
That will build just about everything and will download | That will build just about everything and will download FGData into <code>~/work/fg/master/install/fgfs/fgdata</code> | ||
~/work/fg/master/install/fgfs/fgdata | |||
=== Retrofit previous use of download_and_compile.sh === | === Retrofit previous use of download_and_compile.sh === | ||
Download_and_compile.sh placed the | Download_and_compile.sh placed the FGData directory in the <code>install/fgfs</code> directory. | ||
We're going to move it. N | We're going to move it. N | ||
From the directory where you | From the directory where you run <code>download_and_compile.sh</code>: | ||
<code> | |||
</code> | |||
If you want to adopt my tree naming scheme, then move the source directories for the various components to the appropriate build directory such as ~/work/fg/master. Note that you cannot move the build and install directories. They'll be rebuilt in the new location. The rebuild is needed because we use Cmake. | mv install/fgfs/fgdata ~/work/fg/fgdata_2.11.0 | ||
ln -s /home/userid/work/fg/fgdata_2.11.0/fgdata install/fgfs/fgdata | |||
If you want to adopt my tree naming scheme, then move the source directories for the various components to the appropriate build directory such as <code>~/work/fg/master</code>. Note that you cannot move the build and install directories. They'll be rebuilt in the new location. The rebuild is needed because we use Cmake. | |||
Test to see that everything is hooked up. | Test to see that everything is hooked up. | ||
./download_and_compile.sh DATA | ./download_and_compile.sh DATA | ||
=== Prepare to build another version === | === Prepare to build another version === | ||
In step 1, we created a spot for two FGData versions. So far only one is actually there, <code>fgdata_2.11.0/fgdata</code>. Lets fix that so we don't have to download another copy for 2.10.0 | |||
cd ~/work/fg/fgdata_2.8.0 | cd ~/work/fg/fgdata_2.8.0 | ||
cp -r ~/work/fg/fgdata_2.11.0 fgdata | cp -r ~/work/fg/fgdata_2.11.0 fgdata | ||
That's right, we just made a copy of 2.11.0's fgdata and are calling it 2.8.0. | That's right, we just made a copy of 2.11.0's fgdata and are calling it 2.8.0. | ||
While we're at it, lets copy the 2.11.0 source code | While we're at it, lets copy the 2.11.0 source code | ||
cd ~/work/fg/stable | cd ~/work/fg/stable | ||
cp -R ~/work/fg/master/download_and_compile.sh ./ | cp -R ~/work/fg/master/download_and_compile.sh ./ | ||
Line 108: | Line 104: | ||
cp -R ~/work/fg/master/fgcom ./ | cp -R ~/work/fg/master/fgcom ./ | ||
cp -R ~/work/fg/master/fgcomgui ./ | cp -R ~/work/fg/master/fgcomgui ./ | ||
And replace the link to the right version | And replace the link to the right version | ||
mkdir -p /installfgfs | mkdir -p /installfgfs | ||
ln -s ~/work/fg/fgdata/2.8.0 install/fgfs | ln -s ~/work/fg/fgdata/2.8.0 install/fgfs | ||
ls -lah install/fgfs/fgdata # show the symlink to fgdata | ls -lah install/fgfs/fgdata # show the symlink to fgdata | ||
ls install/fgfs/fgdata # make sure there's something there. | ls install/fgfs/fgdata # make sure there's something there. | ||
Now you can build stable | |||
./download_and_compile.sh -s ALL | ./download_and_compile.sh -s ALL | ||
=== Building with options === | === Building with options === | ||
You can use download_and_compile to build master or stable with various options such as -i for | You can use <code>download_and_compile.sh</code> to build master or stable with various options such as <code>-i</code> for OpenRTI or <code>-g</code> to build a debug version. | ||
You can do that in your master or stable that you just built without these options by deleting the install directories and using the options you want to rebuild with. ( | You can do that in your master or stable that you just built without these options by deleting the install directories and using the options you want to rebuild with. (Remember to redo the link as in step 3 if you delete <code>install/fgfs</code>. The link will go, but <code>~/work/fgdata_*/fgdata</code> will stay.) | ||
An alternative is to use a separate directory. All that's needed is that link to fgdata before you start. | An alternative is to use a separate directory. All that's needed is that link to fgdata before you start. | ||
Three of the <code>download_and_compile.sh</code> options actually build differently. | |||
* <code>-i</code> builds with OpenRTI | |||
* <code>-e</code> builds with –with-eventinput | |||
* <code>-g</code> builds for debugging | |||
If you create a separate directory with the correct install/fgfs/fgdata symbolic link, you can build either master or stable with any of these options, independently of your main builds in master or stable. | If you create a separate directory with the correct <code>install/fgfs/fgdata</code> symbolic link, you can build either master or stable with any of these options, independently of your main builds in master or stable. | ||
You have choices. You can use the options in ~/work/fg/stable and ~/work/fg/master, or you can do things in separate directories. Whatever suits your purpose. | You have choices. You can use the options in <code>~/work/fg/stable</code> and <code>~/work/fg/master</code>, or you can do things in separate directories. Whatever suits your purpose. | ||
[[Category:Core developer documentation]] | [[Category:Core developer documentation]] |
Revision as of 13:11, 14 July 2015
When building different versions of FlightGear, its important to note that each version has its own version of FGData, and FlightGear will check the version to ensure it is compatible.
Since FGData is so large (At least 1.2 GB as of 07/06/15), it makes sense to download the whole FGData only once. You can keep versions of it separately from the install tree that download_and_compile.sh creates and create symbolic links to them before using the script. That way when you decide to rebuild from scratch, at least the FGData will remain.
The download_and_compile.sh
script can build either the next/master branch, or the stable version of FlightGear.
The instructions below tell how to do this for existing or new uses of download_and_compile.sh
. The technical details are not dependent on where you put your builds. You can use the directory scheme I use or your own existing scheme.
The concept as a whole works great. I've been using it for months. The instructions are a work in progress and have yet to be fully tested.
The main ideas
- You may only need one copy of each version of FGData you are working with
- FGData normally lives as a sub-directory in
install/fgfs/
- Once it exists,
install/fgdata
can moved and replaced by a symbolic link to FGData's real directory location
Technical points and command examples
mkdir -p install/fgfs
will create the install directory and subdirectory if either does not yet existmv
can move a directorycp -R
will copy a directory recursively.ln -s targetname linkname
creates linkname as a symbolic link to the target. The target may or may not yet exist and that's normally ok. Not in this case, because of the way we create the directory with git.- The symbolic link must be created after downloading
- Important: You can move the FlightGear source directories and
download_and_compile.sh
to different folders at will. Git will not care. All the Git specific files move right along with the sources. Its the same for SVN. However, the same does not hold true for the files created during a build by cmake. The build and install folders cannot be moved because the files in them have hard coded paths in some cases. They'll need to be deleted and rebuilt.
Background info
I do my builds of different versions and options in separate sub-directories under ~/work/fg
. Sometimes I'll create an extra tree for debugging purposes. I don't want to have to copy all of FGData just to do the work on a debug version. I also copy the sources for the various components to avoid re downloading those as well. In download_and_compile.sh, Git takes care of any changes needed to switch to between master and stable as needed.
Here's how I handle FGData...
Normally fgdata lands in install/fgfs
.
I've moved my 2.10.0 and 2.11.0 FGData versions and replaced them with symbolic links to the new location as needed.
Its as simple as that.
Detailed directions
In short the steps are:
- Create a directory for each version of FGData you are going to use
- Create a directory for each version or variation of FlightGear you want to build
- Do you first build of FlightGear
- Retrofit previous use of
download_and_compile.sh
- Prepare to build another version
- Building with options
Create a directory for each version of FGData you are going to use
mkdir -p ~/work/fg/fgdata_2.10.0 mkdir -p ~/work/fg/fgdata_2.11.0
Eventually a particular version of FGData will be in each
Create a directory for each version or variation of FlightGear you want to build
download_and_compile.sh
will be run from these directories
Mine are sub-directories of ~/work/fg
for example:
mkdir -p ~/work/fg/master mkdir -p ~/work/fg/stable mkdir -p ~/work/fg/master_debug mkdir -p ~/work/fg/master_debug_rti
Note that I never build at the ~/work/fg level
. download_and_compile.sh
is always run from a sub-directory
Do you first build of FlightGear
You can skip this step if you've already done one.
If you've never used download_and_compile.sh
before you're ready to go.
cp download_and_compile.sh ~/work/fg/master cd ~/work/fg/master ./download_and_compile.sh ALL
That will build just about everything and will download FGData into ~/work/fg/master/install/fgfs/fgdata
Retrofit previous use of download_and_compile.sh
Download_and_compile.sh placed the FGData directory in the install/fgfs
directory.
We're going to move it. N
From the directory where you run download_and_compile.sh
:
mv install/fgfs/fgdata ~/work/fg/fgdata_2.11.0 ln -s /home/userid/work/fg/fgdata_2.11.0/fgdata install/fgfs/fgdata
If you want to adopt my tree naming scheme, then move the source directories for the various components to the appropriate build directory such as ~/work/fg/master
. Note that you cannot move the build and install directories. They'll be rebuilt in the new location. The rebuild is needed because we use Cmake.
Test to see that everything is hooked up.
./download_and_compile.sh DATA
Prepare to build another version
In step 1, we created a spot for two FGData versions. So far only one is actually there, fgdata_2.11.0/fgdata
. Lets fix that so we don't have to download another copy for 2.10.0
cd ~/work/fg/fgdata_2.8.0 cp -r ~/work/fg/fgdata_2.11.0 fgdata
That's right, we just made a copy of 2.11.0's fgdata and are calling it 2.8.0.
While we're at it, lets copy the 2.11.0 source code
cd ~/work/fg/stable cp -R ~/work/fg/master/download_and_compile.sh ./ cp -R ~/work/fg/master/fgfs ./ cp -R ~/work/fg/master/plib ./ cp -R ~/work/fg/master/simgear ./ cp -R ~/work/fg/master/OpenSceneGraph ./ cp -R ~/work/fg/master/fgrun ./ cp -R ~/work/fg/master/fgcom ./ cp -R ~/work/fg/master/fgcomgui ./
And replace the link to the right version
mkdir -p /installfgfs ln -s ~/work/fg/fgdata/2.8.0 install/fgfs ls -lah install/fgfs/fgdata # show the symlink to fgdata ls install/fgfs/fgdata # make sure there's something there.
Now you can build stable
./download_and_compile.sh -s ALL
Building with options
You can use download_and_compile.sh
to build master or stable with various options such as -i
for OpenRTI or -g
to build a debug version.
You can do that in your master or stable that you just built without these options by deleting the install directories and using the options you want to rebuild with. (Remember to redo the link as in step 3 if you delete install/fgfs
. The link will go, but ~/work/fgdata_*/fgdata
will stay.)
An alternative is to use a separate directory. All that's needed is that link to fgdata before you start.
Three of the download_and_compile.sh
options actually build differently.
-i
builds with OpenRTI-e
builds with –with-eventinput-g
builds for debugging
If you create a separate directory with the correct install/fgfs/fgdata
symbolic link, you can build either master or stable with any of these options, independently of your main builds in master or stable.
You have choices. You can use the options in ~/work/fg/stable
and ~/work/fg/master
, or you can do things in separate directories. Whatever suits your purpose.