FlightGear and Git
|FlightGear Git on:|
Git is a version control system used by the FlightGear project to store all of the files required to build the FlightGear, including all the programs, the data (e.g. aircraft), supporting tools, etc. Git tracks updates to every file, as developers from around the world work together concurrently to create new versions.
While new FlightGear features and additions are in development, they are available from Git before they are available in the standard release version. Using Git allows users to build the newest possible version of FlightGear from the latest source files, to experiment with new aircraft or other features. However, it's not a beginner's tool. Using Git can expose the user to unstable features that show ugly error messages, or crash the computer.
In May 2010, after a hardware failure on the CVS servers, the FlightGear project changed its version control system from CVS to Git. The Git repositories are located at Gitorious and a mirror is hosted at the Mapserver.
Due to the "recent" switch, we are currently doing our best on providing manuals for obtaining FlightGear through Git. The following articles belong to that "work in progress":
- FlightGear Git on Mac OS X
- FlightGear Git on Windows
- Howto:Start using git provides a quick introduction to using a "source code management system"
Much has been written on the advantages of Git over CVS. For the FlightGear project, some advantages are:
- Much better support for branches and merging branches. This is especially important for creating bug-fix releases for major releases while still allowing work on the next major release to continue. It is also very nice for a developer's personal work-flow.
- Easier path for contributors to submit changes and developers to integrate them.
- Much better support for everyday tasks like searching the project history for changes, viewing changes, bisecting the project history to find the original source of a bug.
Also, as noted before, the CVS servers had a hardware failure on May 2010, speeding up FlightGear's change to Git.
Manipulating the GIT
For a detailed description see the description of the four repositories (A,B,C,D) in the following chapters.
A) Repositories and branches
- next: The current tip of new development. This branch should always compile and run, but various things could be broken.
- release/*: The branch subdirectory containing former and (if there is already defined one) upcoming releases.
FlightGear data' branches of interest are:
- master: The one that is used with the next branch of the code.
- release/*: The one that is used with the release branch of the code.
For all available branches see https://gitorious.org/fg/fgdata/trees
In addition, there's a 'fgmeta project, intended as a super-project to track compatible simgear & flightgear commits, particular in the context of release and stable branches. "fgmeta" contains the main repositories as git submodules:
To work with the fgmeta project, you'll need to
- clone the fgmeta project: git://gitorious.org/fg/fgmeta.git
- run "git submodule init"
- run "git submodule update"
For other available projects have a look at our gitorious.org. Use the "Source tree" button to see all available branches unique to those projects.
- The "fgdata" project will be used in all following examples - thus the "Prime" local repository will always be named "master"!
- If you test changes, make sure all branches have the same status - otherwise your setup may not be functional!
B) Create your own workspace on Gitorious
Prior to be able to manipulate GIT-data you must be a registered user and be logged in as such. See
Then check on http://gitorious.org/fg for the available projects. See e.g.:
Create your own "GIT-Clone" (as a space to work in), by activating the "Clone repository" button of the wanted project. By default you will be cloning the next or master branch of that project --> that is the actual repository including the newest, not yet formally distributed, changes. That is probably what you want if you want to
- assist with reporting bugs before they are released as a proper release version
- or participate in the design
- or similar
The created GIT-clone will be on you private page https://gitorious.org/~YourUID ("YourUID" being your ID you logged in with, and do not forget the "~" in front of it!))
C) Create a "master" on your local computer
All the following is shown as performed in an UBUNTU "Terminal" environment! Please adapt especially the directory trees according to your Operating-System!
- Create a unique folder on your computer, e.g. "..\GIT"
- Change into that folder and use the following "git clone" commands to clone the wanted repository
- if you need/want more informations about unique GIT-commands, input "git help [command]" (e.g. "git help clone")
- Use one of the following commands (or similar) from within that "..\GIT"-folder:
- for cloning SimGear use:
- git clone git://gitorious.org/fg/simgear.git [target]
- To successfully compile
fgfsFlight- and Sim-Gear's working tree they both must have the same state.
- for cloning FlightGear use:
- git clone git://gitorious.org/fg/flightgear.git [target]
- To successfully compile
fgfsFlight- and Sim-Gear's working tree they both must have the same state.
- for cloning fgdata use:
- git clone git://gitorious.org/fg/fgdata.git [target]
- By executing one of the above commands the folder target will be created and a clone will be stored inside it. If no target is given the new folder's name is the GIT-name (i.e.: "../GIT/simgear", "../GIT/flightgear", "../GIT/fgdata").
cd into the new folder [target]
- You are now inside the working tree of this git repository. The repository itself is stored inside the subfolder ../GIT/[target]/.git, which we must not edit manually. Git will take care of that.
- Notice the "." in front of the name: In Linux that indicates a hidden file, for which you might have to enable "show hidden files"!
- will show the active branches. e.g.
..GIT/fgdata$ git branch * master master.local (<-- this branch will not be there initially!)
- Right after download the active GIT-repository-branch will be "next", which in the example is called "master". The "*" in front means that this is now the active one! i.e. the content of the active working tree is the bleeding edge of the source code.
git branch -a
- will list all branches of a git repository. The one marked with an asterisk (*) is the active one. e.g.:
..GIT/fgdata$ git branch -a * master master.local (<-- this will not be there initially!) remotes/origin/HEAD -> origin/master remotes/origin/PRE_OSG_PLIB_20061029 remotes/origin/Short_Empire-public remotes/origin/Work remotes/origin/master remotes/origin/release/2.4.0 remotes/origin/release/2.6.0 remotes/origin/releases/2.2.0
You now have your local "GIT-master". But: To make individual changes (like editing a joystick file), add a plane to fgdata or patch the source code. It is always recommended to create a local branch inside that "master" repository. In the chapter "Edit" we will describe a way to achieve this. But have in mind, that this is very basic and only suits for users who have minor changes.
Special for fgdata:
At the time of writing the data repository is about 3.4 GB. Continuing an interrupted cloning of a repository is not supported within git. Therefore, if you have a slow or unstable connection to the internet, it is recommended to download the fgdata.bundle.
Also have in mind, that the repository plus the working tree will be more than twice the size of the download on your local filesystem.
There is neither a "next" branch nor any tags in that repository. If e.g. you want to build FlightGear 2.0.0 you must fetch the data (FlightGear-data-2.0.0.tar.bz2) at one of the mirrors.
For the FlightGear-data there is a git-bundle (snapshot) torrent (tracker; md5|sha1|sha512) available. This way you can resume interrupted downloads. After unpacking it only a comparatively small amount of data has to be transferred from the git server to synchronize your repository. See also the Develop sections in FlightGear Git on Windows.
Do the following steps to extract the bundle and bring the repository up to date:
$ git clone fgdata.bundle fgdata $ cd fgdata $ git checkout -b master-tmp Switched to a new branch 'master-tmp' $ git remote rm origin $ git remote add origin git://gitorious.org/fg/fgdata $ git fetch origin $ git branch --track master origin/master $ git checkout master $ git branch -D master-tmp
If you get an error at
git fetch origin try:
$ git remote rm origin $ git remote add origin git://mapserver.flightgear.org/fgdata/ $ git fetch origin
You should be suspicious if based on the printed progress you estimate the data download during the fetch will exceed 1GB (assuming the bundle is not terribly outdated).
D) Edit and commit data
Switch into the branch in which you want to edit your changes (without affecting your master yet!). We will use for that branch the name "master.local". Thus:
git checkout master
..GIT/fgdata$ git checkout master Switched to branch 'master'
- will change the active branch to master. Means, that the content of the working tree will be changed to that state.
- In case you get an error like
GIT/fgdata$ git checkout master Already on 'master' Your branch is ahead of 'origin/master' by 2 commits.
- you should re-sync your master with e.g.:
git checkout -b master.local master
..GIT/fgdata$ git checkout -b master.local master Switched to a new branch 'master.local'
- Generates the new "master.local" and switches to it.
- In case you get the response:
fatal: A branch named 'master.local' already exists.
- you have to define another name for that branch
- or delete it:
- Try first: "git branch master.local -d" ⇒ that will delete it if it is "clean"
- If needed do: "git branch -D master.local" ⇒ that forces a delete - even if there is uncommitted work in it !!
You may now
- apply your planned changes into the affected "/GIT/subdir"
- start FlightGear to verify they are proper/functional
- then check what was done:
- the output may take some time to appear!
- see e.g. a dummy-example:
..GIT/fgdata$ git status # On branch master.local # Changes not staged for commit: # (use "git add <file>..." to update what will be committed) # (use "git checkout -- <file>..." to discard changes in working directory) # # modified: Aircraft/ATC-ML/Version # # Untracked files: # (use "git add <file>..." to include in what will be committed) # # Aircraft/ATC-ML/Version (copy) # Aircraft/DUMMY/ no changes added to commit (use "git add" and/or "git commit -a") ...... etc.
- This gives you a list of files that have been altered or that are new. In the above (dummy example) you see:
- the already existing file "Aircraft/ATC-ML/Version" was modified
- a new file "Aircraft/ATC-ML/Version (copy)" was added
- and a new directory "Aircraft/DUMMY/" was added, including all its files
If that is what you wanted/needed continue with the following (otherwise correct your work first)
So we want to "add" those changes:
git add "Aircraft/ATC-ML/Version (copy)"
git add Aircraft/ATC-ML/Version
git add Aircraft/DUMMY/
If you now perform the git status again you should see something similar to:
../GIT/fgdata$ git status # On branch master.local # Changes to be committed: # (use "git reset HEAD <file>..." to unstage) # # modified: Aircraft/ATC-ML/Version # new file: Aircraft/ATC-ML/Version (copy) # new file: Aircraft/DUMMY/ATC-ML (copy).xml # new file: Aircraft/DUMMY/ATC-ML-set.xml # new file: Aircraft/DUMMY/ATC-ML.ac # new file: Aircraft/DUMMY/ATC-ML.nas ... etc. #
If that is all you need - you may "commit" it:
git commit -m "your short, accompanying Announcement"
- will commit the updates to the master.local branch. It shows a list compatible to the one created with the "git status" before, but this "git commit" created something new! (Yes: You could have omitted the "git status" and just did the "commit" - but we suggest to "double-check" it this way!)
..GIT/fgdata$ git log commit 7658c67c16a79900acf99dbfd85499b720dcc920 Author: YourName <Your eMail> Date: Mon Jun 4 19:14:29 2012 +0200 your short, accompanying Announcement
- This is the "commit" as it will be tracked within the GIT.
- You definitely should now copy the ID (the first 8 digits may be enough) to use it later as a unique identifier!
Now return to your local "master" with:
git checkout master
git fetch git://gitorious.org/fg/fgdata.git
- replace in that line the "fgdata.git" with the repository you are actually working on
- this will update the master to the newest GIT-level (there may have been changes during all that time!)
git cherry-pick 7658c67c
- replace the "7658c67c" with the first digits of your commit-ID (see above "git log")
- will pull the commit into the "master"
git rebase origin/master
- finally update the "master" with the "commit" on top of it
Push your changes to your clone on Gitorious
git push email@example.com:~YourUID/fg/YourUIDs-fgdata.git
- replace "YourID" with your login-name for Gitorious, do not forget the "~" in front of it!
- replace in that line the "fgdata.git" with the repository you are actually working on
- will finally push your updates to your space inside "gitorious.org"
Now you can announce to GIT that you have a change that may be merged into the original GIT:
- Open your git-clone on Gitorious: https://gitorious.org/~YourUID/fg/YourUID-fgdata
- and click the "Request merge" button on the right:
Write a short summary, in which you explain what your commit(s) do(es). Then choose the following settings:
- Target directory: fgdata, flightgear or simgear
- Target branch: master, next or whatever
- Source branch: master, next or whatever
Next select the commits you would like to include in your merge request and click "Create merge request".
Note that it usually doesn't suffice to just send a merge request via gitorious (unfortunately), but that you'll often also need to "ping" 1-3 core developers who have previously worked with the subsystems affected by the patch (and send a reminder once in a while).
It makes sense to watch the forums/mailing lists or just the git history to see which developers are currently active. To get a list of people who have previously worked with the affected subsystems/files, just take a look at the commit log of the corresponding files and get in touch with the developers via the devel list or forum.
Also, it may help to send a follow-up (merge request review) via the issue tracker here: https://code.google.com/p/flightgear-bugs/issues/entry?template=Review%20request
Currently, it isn't uncommon for merge requests to take 2-4 weeks, and sometimes even 1-2 months before being reviewed and committed. Your chances of getting your code committed upstream are much better directly after a release, than shortly before a release obviously (feature freeze).
Commit the changes into the GIT
This step is for developers with fgdata commit access only!
For a list of those authorized see: https://gitorious.org/+flightgear-developers/memberships and http://gitorious.org/+fgdata-developers/memberships (fgdata only)
To push a merge request to fgdata, the following work-flow can be followed:
git checkout -b integration # the integration branch should not yet exist. You can use any name you'd like git pull git://gitorious.org/fg/fgdata.git refs/merge-requests/1 # make sure you change 1 to the id of the merge request git rebase master # place the new commits on top of existing commits (without ugly "Merge branch into ...") git checkout master git cherry-pick b049e1d # repeat this for each commit you'd like to push git push
Some more helpful commands
git help [command]
- apply a patch to files and/or to the index http://www.kernel.org/pub/software/scm/git/docs/git-apply.html
git checkout -f
- may be used to throw away any local changes to the working tree. Use with care, as any option that name is -f or --force, and only after reading
git checkout help!
- may be used for future updates
git remote - adding remote repositories
Now, before anyone unnecessarily goes through the pain of cloning fgdata again, you just need to add the git URL to your new personal clone at gitorious as a remote in your local fgdata clone to be able to push to gitorious.
git remote add my-fgdata g...@gitorious.org:~andersg/fg/anders-fgdata.git
Stores the URL to my fgdata clone at gitorious under the name my-fgdata in my local git clone of fgdata. (You want the gitorious SSH URL for your repository)
git push my-fgdata my-branch:master
Pushes the local branch named my-branch to my-fgdata (i.e. my clone of fgdata at gitorious) where the branch will be named master.
Switching to a remote branch
You can create a local version of a remote branch with "git branch releases-2.2.0 origin/releases-2.2.0"or "git checkout -b releases-2.2.0 origin/releases-2.2.0". Git doesn't create a local branch automatically. It exists locally as a remote branch. Create a local branch based on it with git branch -t -l my-2.2.0 origin/releases/2.2.0
git pull will also work if you have committed local changes but will make your local history messy (and the official history too if your changes are ever merged back into the official history). If you don't care about the messyness of your local history read no further than point 1 below and use git pull without hesitation :)
If you do have changes you want to keep I'd recommend using git rebase to keep them "on top" of the official work:
1. First commit your changes to your local branch.
git status - show you what files you have modified. git add file1 file2 etc - adds the files you want to commit git commit - creates a commit with the changes you have added.
2. Fetch the latest stuff from the main repository.
3. Rebase your local branch on top of the latest official state. For the FlightGear and SimGear sources this would be
git rebase origin/next
For fgdata it is
git rebase origin/master
4. If you get conflicts you can drop your local conflicting commit by
git rebase --skip
or resolve the conflicts, git add the changed files and continue the rebase with
git rebase --continue
(Use of git status is needed here to see which files are in conflict).
As an additional safe-guard you may create a name for your previous work before you rebase so that you can easily recover it if the rebase goes bad. Assuming your branch is called my-branch the following command creates a back-up point:
git branch my-branch.20110205 my-branch
Merging Topic Branches
You want to merge your branch back into master. merge is the right word:
- git checkout master
- git merge newidea
done. If the merge creates conflicts, git will tell you so. To fix them, simply edit the files and add them to the index (git add fixed_file) and when you are done do a git commit. A merge usually creates a new commit anyway, since it's a new version of your source tree.
If your newidea branch is nice and tidy and a straight continuation of your current master branch (i.e. they have not diverged) you can just merge the newidea branch into master (if they have not diverged it will just be a fast-forward of the master branch).
You can create a new branch to keep track of your old master point first if you like:
git branch old-master master
git branch -h or --help will show many useful options to git branch e.g. for creating, renaming and deleting branches.
As long as you don't rewrite the history you can always create a new branch starting at any old commit so there is no particular need to create such branches before you need them (except maybe to help remembering where that point was).
If the branches have diverged I would consider cherry-pick over the commits from the newidea branch into master (if they are not very many) and perhaps also tidy them up with using interactive rebasing before publishing the new master state. This is particularily useful if master is a public branch that receives commits from other developers - it avoids the rather ugly multiple levels of merges we see in e.g. fgdata. (See also git rebase).
gitk --all is a very useful tool to see where your different branches are in the history and how they relate to each other.
While on the master branch:
git merge newidea
Basically, you have to resolve your conflicts at that point if you want to keep your commit. Even if you merged your changed branch (e.g. with git pull) rather than rebasing it you'd get the conflicts.
- git status to check which files are in conflict.
- git add <files> to register the state you want them to have.This may include cleaning out merge conflict from text files before adding them.use git checkout local-weather -- the/file to restore your version and git checkout master -- the/file to restore the upstream version.
- git rebase --continue to continue the rebase.
For your own local work I recommend committing it in small logical units - that makes it easier to use git rebase --skip to remove local edits when they become obsolete due to upstream updates.
Btw. if you don't have any particular need to checkout the master branch just
git fetch origin/master git rebase origin/master
on the local-weather branch will do.
But do remember to use origin/master rather than just master in git diff and git checkout -- some/file commands in that case, since your local master branch will not be updated by fetch and rebase.
git merge vs. rebase
Each rebase moves your local commits to be on-top of all upstream commits.
For example, my local SimGear branch currently looks like this: git log output:
commit 4087b34f7ebbdb54b62afb205dc2e1ca225dc68b Author: Anders Gidenstam <and...@gidenstam.org> Date: Tue Mar 29 22:44:53 2011 +0200 Experimental Nasal GC work: Added a GC thread. commit d94d1a907d6ec001ab9ba497bc03aaeff55f923c Author: Anders Gidenstam <and...@gidenstam.org> Date: Sun Oct 3 16:59:50 2010 +0200 Turn the creation of a variable without the var keyword an error in Nasal. commit c7c3fae5c2cd21cf81e7a94911568adba926f680 Author: ThorstenB <bre...@gmail.com> Date: Sat May 7 19:40:01 2011 +0200 Also remove visual_enviro.cxx from the VC90 build.
That is my two local commits appear as later than any upstream commit, although they are "older" (as seen by their dates).
This is the effect of the rebase. If I had merged my branch with the upstream branch instead of rebasing it on top of the upstream branch my commits would have been somwhere way down in the history.
However, note that each rebase creates new commits since the commit ID is a consequence of the history preceding the commit. The old "copies" of my commits from previous rebases are no longer part of my local branch.
Keeping topic branches in sync with upstream
Any time someone pushes a change to the gitorious repository here is the approximate procedure to update my local clones/branches (this is the git replacement for the old cvs update command):
cd "primary-fgdata" git pull <error> - oops I have a branch checked out currently git checkout master <error> - oops commit any changes in the current branch - git diff - git commit git checkout master (try again, it works) git pull (now it works) git checkout "primary-branch" git merge master (to sync the upstream changes with my own "wip" branch)
But this is just in the main fgdata clone, Now cd over to my --local branch clone.
cd "../fgdata-clone" git pull (merge upstream changes from my local master repository that have been merged into the master branch in the previous step.) git diff (see what I changed locally) git commit (commit my local changes) git push (push these changes back into the primary branch in my original clone of the gitoriious repository) <error> - oops I have the branch checked out in my primary local repository - cd "../fgdata-primary" - git checkout master - cd ../fgdata-clone" git push (now it works!)
Woohoo, everything should now be consistent and in sync and all the upstream changes should be fully merged.
Tracking a release branch
git branch -t -l release/2.6.0 origin/release/2.6.0 git checkout -b release/2.6.0
Messed up branches
It's worth experimenting with "git reflog" in situations like this. That tracks a list of HEAD references in strict chronological order (i.e. what has HEAD been in the past, not what commits were done).
In cases where you've completely mucked up the revision history, you can look at this to see what you were doing before, recover the commit ID, and do a reset --hard to that.
"Backing out" is done with git reset --hard last_good_commit. Often the name of the last good commit is HEAD^, the last commit. However, after a botched merge it is good to verify that with git log or graphically with gitk.
A merge commit has two parent commits (leaving aside octopus commits). If you are not happy with the results of the merge, usually you want to revert back to the parent that was on your branch. The reflog can be useful for checking this, but usually the parent of the botched merge on your branch is HEAD^.
If you've pushed a commit to a public repo and then it later turns out that the commit wasn't a good idea, then you want "git revert" which creates the reverse patch for a given commit. However, if you make a real hash out of a public repo you may still want git reset.
Backing it out might be a bit tricky, but you can rename your messed up branch out of the way easily with git branch -m oldname newname.
To cherry-pick commits from your other repository into a branch you first fetch the branch you want to pick, e.g.
git fetch theOtherRepro.git theotherbranch:suitableName/theotherbranch
Or just theotherbranch:suitableOthername
Then you can inspect the commits on it with git log -u theOtherRepro/theotherbranch
And finally cherry-pick the one you want with git cherry pick <commitID>
It's a good idea to always use a clean local copy (e.g. git branch -t mrClean origin/master) of origin/master to cherry-pick commits to before pushing to origin, and leave that branch around since the next time you just need to check it out, do git pull which will be a clean fast forward and cherry-pick and push again.
Resetting the repository
If you find yourself having constant trouble with GIT and being contstantly asked to "rebase" your commitments, please don't delete anything, instead follow AndersG's instructions below.
If you have a local clone of fgdata you have everything at hand - starting over is just creating a new local branch that tracks master, e.g.:
git branch my-new-master origin/master git checkout my-new-master
And update it to latest with (I recommend using --rebase in the future since that will keep your local commits after all up stream commits): git pull --rebase
to investigate what ever uncommitted changes you may have in your tree. If you want to throw all such changes away, use git reset --hard
Or you can use git stash to save them for later.
If you have an old branch with your work and want to reapply selected commits to the new one, git cherry-pick is a useful command.
Updating old FGData gitorious clones
If you haven't updated your gitorious clone in a while, so that it significantly diverges from the main fgdata repository, pushing all changes would eat up lots of resources (bandwith, CPU) - thus, it is generally better to back up all important data/branches, and then delete your gitorious clone, to start over with a fresh clone - that way, you'll save tons of bandwidth and time. Cloning fgdata only takes a fraction of the time that would be required to push hundreds of megabytes of data separately, this is because cloning fgdata is a "local" operation for gitorious - so that bandwith is not the bottleneck.
Git tutorials and resources
- Git documentation and tutorials
- Git Basics 
- The Git Cheat Sheet and the Git Cheat Sheet Extended Edition
- Egg, a cool Git emacs mode.
- A guide to using Git on Windows
- Git on Windows Go! (Setting up msysgit on Windows)
- qgit - interactive git repository viewer and frontend
- Additional Resources WRT running git on Win32
- FlightGear Git: splitting fgdata, an initiative to split the aircraft out of the fgdata repository, in order to decrease its size and thus improve access to the average user/developer.