FlightGear Git

From FlightGear wiki
Revision as of 21:03, 21 February 2013 by Quix0r (talk | contribs) (Removed dead link as it is now only available through BitTorrent (eaten up a lot bandwitdth))
Jump to navigation Jump to search

Git is a version control system used by the FlightGear project to store all of the files required to build the FlightGear, including all the programs, the data (e.g. aircraft), supporting tools, etc. Git tracks updates to every file, as developers from around the world work together concurrently to create new versions.

While new FlightGear features and additions are in development, they are available from Git before they are available in the standard release version. Using Git allows users to build the newest possible version of FlightGear from the latest source files, to experiment with new aircraft or other features. However, it's not a beginner's tool. Using Git can expose the user to unstable features that show ugly error messages, or crash the computer.

In May 2010, after a hardware failure on the CVS servers, the FlightGear project changed its version control system from CVS to Git. The Git repositories are located at Gitorious and a mirror is hosted at the Mapserver.

Due to the "recent" switch, we are currently doing our best on providing manuals for obtaining FlightGear through Git. The following articles belong to that "work in progress":

Motivation

Much has been written on the advantages of Git over CVS. For the FlightGear project, some advantages are:

  • Much better support for branches and merging branches. This is especially important for creating bug-fix releases for major releases while still allowing work on the next major release to continue. It is also very nice for a developer's personal work-flow.
  • Easier path for contributors to submit changes and developers to integrate them.
  • Much better support for everyday tasks like searching the project history for changes, viewing changes, bisecting the project history to find the original source of a bug.

Also, as noted before, the CVS servers had a hardware failure on May 2010, speeding up FlightGear's change to Git.

Manipulating the GIT

Work-flow Overview

GIT-Workflow.png

For a detailed description see the description of the four repositories (A,B,C,D) in the following chapters.

A) Repositories and branches

For historical reasons there continue to be separate repositories for SimGear and FlightGear. For most users and developers the most interesting ones for those 2 are:

  • next: The current tip of new development. This branch should always compile and run, but various things could be broken.
  • release/*: The branch subdirectory containing former and (if there is already defined one) upcoming releases.

For all available branches, see SimGear branches and/or FlightGear branches

FlightGear data' branches of interest are:

  • master: The one that is used with the next branch of the code.
  • release/*: The one that is used with the release branch of the code.

For all available branches see https://gitorious.org/fg/fgdata/trees

In addition, there's a 'fgmeta project, intended as a super-project to track compatible simgear & flightgear commits, particular in the context of release and stable branches. "fgmeta" contains the main repositories as git submodules:

  • simgear
  • flightgear
  • fgrun

To work with the fgmeta project, you'll need to

For other available projects have a look at our gitorious.org. Use the "Source tree" button to see all available branches unique to those projects.

Warning:

  • The "fgdata" project will be used in all following examples - thus the "Prime" local repository will always be named "master"!
  • If you test changes, make sure all branches have the same status - otherwise your setup may not be functional!

B) Create your own workspace on Gitorious

Prior to be able to manipulate GIT-data you must be a registered user and be logged in as such. See

https://gitorious.org/users/new
https://gitorious.org/login

Then check on http://gitorious.org/fg for the available projects. See e.g.:

  • flightgear
  • simgear
  • fgdata
  • getstart
  • etc.

Create your own "GIT-Clone" (as a space to work in), by activating the "Clone repository" button of the wanted project. By default you will be cloning the next or master branch of that project --> that is the actual repository including the newest, not yet formally distributed, changes. That is probably what you want if you want to

  • assist with reporting bugs before they are released as a proper release version
  • or participate in the design
  • or similar

The created GIT-clone will be on you private page https://gitorious.org/~YourUID ("YourUID" being your ID you logged in with, and do not forget the "~" in front of it!))

C) Create a "master" on your local computer

All the following is shown as performed in an UBUNTU "Terminal" environment! Please adapt especially the directory trees according to your Operating-System!

  1. Create a unique folder on your computer, e.g. "..\GIT"
  2. Change into that folder and use the following "git clone" commands to clone the wanted repository
    • if you need/want more informations about unique GIT-commands, input "git help [command]" (e.g. "git help clone")
  3. Use one of the following commands (or similar) from within that "..\GIT"-folder:
  • for cloning SimGear use:
git clone git://gitorious.org/fg/simgear.git [target]
To successfully compile fgfs Flight- and Sim-Gear's working tree they both must have the same state.
  • for cloning FlightGear use:
git clone git://gitorious.org/fg/flightgear.git [target]
To successfully compile fgfs Flight- and Sim-Gear's working tree they both must have the same state.
  • for cloning fgdata use:
git clone git://gitorious.org/fg/fgdata.git [target]
By executing one of the above commands the folder target will be created and a clone will be stored inside it. If no target is given the new folder's name is the GIT-name (i.e.: "../GIT/simgear", "../GIT/flightgear", "../GIT/fgdata").

cd into the new folder [target]

You are now inside the working tree of this git repository. The repository itself is stored inside the subfolder ../GIT/[target]/.git, which we must not edit manually. Git will take care of that.
Notice the "." in front of the name: In Linux that indicates a hidden file, for which you might have to enable "show hidden files"!
git branch
will show the active branches. e.g.
..GIT/fgdata$ git branch
* master
  master.local  (<-- this branch will not be there initially!)
Right after download the active GIT-repository-branch will be "next", which in the example is called "master". The "*" in front means that this is now the active one! i.e. the content of the active working tree is the bleeding edge of the source code.
git branch -a
will list all branches of a git repository. The one marked with an asterisk (*) is the active one. e.g.:
..GIT/fgdata$ git branch -a
* master
  master.local  (<-- this will not be there initially!)
  remotes/origin/HEAD -> origin/master
  remotes/origin/PRE_OSG_PLIB_20061029
  remotes/origin/Short_Empire-public
  remotes/origin/Work
  remotes/origin/master
  remotes/origin/release/2.4.0
  remotes/origin/release/2.6.0
  remotes/origin/releases/2.2.0

You now have your local "GIT-master". But: To make individual changes (like editing a joystick file), add a plane to fgdata or patch the source code. It is always recommended to create a local branch inside that "master" repository. In the chapter "Edit" we will describe a way to achieve this. But have in mind, that this is very basic and only suits for users who have minor changes.

Special for fgdata:

At the time of writing the data repository is about 3.4 GB. Continuing an interrupted cloning of a repository is not supported within git. Therefore, if you have a slow or unstable connection to the internet, it is recommended to download the fgdata.bundle.

Also have in mind, that the repository plus the working tree will be more than twice the size of the download on your local filesystem.

There is neither a "next" branch nor any tags in that repository. If e.g. you want to build FlightGear 2.0.0 you must fetch the data (FlightGear-data-2.0.0.tar.bz2) at one of the mirrors.

fgdata.bundle

For the FlightGear-data there is a git-bundle (snapshot) available that can be retrieved with your favourite download manager. This way you can resume interrupted downloads. After unpacking the fgdata.bundle only a comparatively small amount of data has to be transferred from the git server to synchronize your repository.

Get the fgdata.bundle.torrent (tracker; md5|sha1|sha512). See also the Develop sections in http://wiki.flightgear.org/FlightGear_Git_on_Windows.

Do the following steps to extract the bundle and bring the repository up to date:

$ git clone fgdata.bundle fgdata
$ cd fgdata
$ git checkout -b master-tmp
Switched to a new branch 'master-tmp'
$ git remote rm origin
$ git remote add origin git://gitorious.org/fg/fgdata
$ git fetch origin
$ git branch --track master origin/master
$ git checkout master
$ git branch -D master-tmp

If you get an error at git fetch origin try:

$ git remote rm origin
$ git remote add origin git://mapserver.flightgear.org/fgdata/
$ git fetch origin

You should be suspicious if based on the printed progress you estimate the data download during the fetch will exceed 1GB (assuming the bundle is not terribly outdated).

D) Edit and commit data

Edit

Switch into the branch in which you want to edit your changes (without affecting your master yet!). We will use for that branch the name "master.local". Thus:

git checkout master
..GIT/fgdata$ git checkout master
Switched to branch 'master'
will change the active branch to master. Means, that the content of the working tree will be changed to that state.
In case you get an error like
GIT/fgdata$ git checkout master
Already on 'master'
Your branch is ahead of 'origin/master' by 2 commits.
you should re-sync your master with e.g.:
git fetch
git checkout -b master.local master
..GIT/fgdata$ git checkout -b master.local master
Switched to a new branch 'master.local'
Generates the new "master.local" and switches to it.
In case you get the response:
fatal: A branch named 'master.local' already exists.
  • you have to define another name for that branch
  • or delete it:
    Try first: "git branch master.local -d" ⇒ that will delete it if it is "clean"
    If needed do: "git branch -D master.local" ⇒ that forces a delete - even if there is uncommitted work in it !!


You may now

  • apply your planned changes into the affected "/GIT/subdir"
  • start FlightGear to verify they are proper/functional
  • then check what was done:

git status
the output may take some time to appear!
see e.g. a dummy-example:
..GIT/fgdata$ git status
# On branch master.local
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#	modified:   Aircraft/ATC-ML/Version
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#	Aircraft/ATC-ML/Version (copy)
#	Aircraft/DUMMY/
no changes added to commit (use "git add" and/or "git commit -a")
...... etc.
This gives you a list of files that have been altered or that are new. In the above (dummy example) you see:
  • the already existing file "Aircraft/ATC-ML/Version" was modified
  • a new file "Aircraft/ATC-ML/Version (copy)" was added
  • and a new directory "Aircraft/DUMMY/" was added, including all its files

If that is what you wanted/needed continue with the following (otherwise correct your work first)

So we want to "add" those changes:

git add "Aircraft/ATC-ML/Version (copy)"
git add Aircraft/ATC-ML/Version
git add Aircraft/DUMMY/

If you now perform the git status again you should see something similar to:

../GIT/fgdata$ git status
# On branch master.local
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#	modified:   Aircraft/ATC-ML/Version
#	new file:   Aircraft/ATC-ML/Version (copy)
#	new file:   Aircraft/DUMMY/ATC-ML (copy).xml
#	new file:   Aircraft/DUMMY/ATC-ML-set.xml
#	new file:   Aircraft/DUMMY/ATC-ML.ac
#	new file:   Aircraft/DUMMY/ATC-ML.nas
  ... etc.
#

If that is all you need - you may "commit" it:

"Commit"

git commit -m "your short, accompanying Announcement"
will commit the updates to the master.local branch. It shows a list compatible to the one created with the "git status" before, but this "git commit" created something new! (Yes: You could have omitted the "git status" and just did the "commit" - but we suggest to "double-check" it this way!)
git log
..GIT/fgdata$ git log
commit 7658c67c16a79900acf99dbfd85499b720dcc920
Author: YourName <Your eMail>
Date:   Mon Jun 4 19:14:29 2012 +0200
 
   your short, accompanying Announcement
This is the "commit" as it will be tracked within the GIT.
You definitely should now copy the ID (the first 8 digits may be enough) to use it later as a unique identifier!

Now return to your local "master" with:

git checkout master
git fetch git://gitorious.org/fg/fgdata.git
replace in that line the "fgdata.git" with the repository you are actually working on
this will update the master to the newest GIT-level (there may have been changes during all that time!)
git cherry-pick 7658c67c
replace the "7658c67c" with the first digits of your commit-ID (see above "git log")
will pull the commit into the "master"
git rebase origin/master
finally update the "master" with the "commit" on top of it

Push your changes to your clone on Gitorious

git push git@gitorious.org:~YourUID/fg/YourUIDs-fgdata.git

replace "YourID" with your login-name for Gitorious, do not forget the "~" in front of it!
replace in that line the "fgdata.git" with the repository you are actually working on

will finally push your updates to your space inside "gitorious.org"

Now you can announce to GIT that you have a change that may be merged into the original GIT:

  • Open your git-clone on Gitorious: https://gitorious.org/~YourUID/fg/YourUID-fgdata
  • and click the "Request merge" button on the right:
Creating a merge request at Gitorious.

Write a short summary, in which you explain what your commit(s) do(es). Then choose the following settings:

Target directory: fgdata, flightgear or simgear
Target branch: master, next or whatever
Source branch: master, next or whatever

Next select the commits you would like to include in your merge request and click "Create merge request".

Note that it usually doesn't suffice to just send a merge request via gitorious (unfortunately), but that you'll often also need to "ping" 1-3 core developers who have previously worked with the subsystems affected by the patch (and send a reminder once in a while).

It makes sense to watch the forums/mailing lists or just the git history to see which developers are currently active. To get a list of people who have previously worked with the affected subsystems/files, just take a look at the commit log of the corresponding files and get in touch with the developers via the devel list or forum.

Also, it may help to send a follow-up (merge request review) via the issue tracker here: https://code.google.com/p/flightgear-bugs/issues/entry?template=Review%20request

Currently, it isn't uncommon for merge requests to take 2-4 weeks, and sometimes even 1-2 months before being reviewed and committed. Your chances of getting your code committed upstream are much better directly after a release, than shortly before a release obviously (feature freeze).

Commit the changes into the GIT

This step is for developers with fgdata commit access only!

For a list of those authorized see: https://gitorious.org/+flightgear-developers/memberships and http://gitorious.org/+fgdata-developers/memberships (fgdata only)

To push a merge request to fgdata, the following work-flow can be followed:

git checkout -b integration						# the integration branch should not yet exist. You can use any name you'd like
 git pull git://gitorious.org/fg/fgdata.git refs/merge-requests/1	# make sure you change 1 to the id of the merge request
 git rebase master							# place the new commits on top of existing commits (without ugly "Merge branch into ...")
 git checkout master
 git cherry-pick b049e1d						# repeat this for each commit you'd like to push
 git push

Supporting information

Some more helpful commands

git help
git help [command]
git apply
apply a patch to files and/or to the index http://www.kernel.org/pub/software/scm/git/docs/git-apply.html
git checkout -f
may be used to throw away any local changes to the working tree. Use with care, as any option that name is -f or --force, and only after reading git checkout help!
git pull
may be used for future updates

Tracking a release branch

git branch -t -l release/2.6.0 origin/release/2.6.0
git checkout -b release/2.6.0

Git tutorials and resources

Related content

  • FlightGear Git: splitting fgdata, an initiative to split the aircraft out of the fgdata repository, in order to decrease its size and thus improve access to the average user/developer.