FlightGear build server: Difference between revisions

From FlightGear wiki
Jump to navigation Jump to search
m (reodering and adding more details about the benefits of automated builds)
(added SF release candidate URL)
 
(47 intermediate revisions by 15 users not shown)
Line 1: Line 1:
= Intro =
{{Release}}
The '''FlightGear build server''', sometimes called the '''nightly build server''' or just '''Jenkins''', is a system based on [http://www.jenkins-ci.org Jenkins] which helps core developers to check that their source code compiles correctly for all OS. It is also responsible for building release packages for Linux, Windows and Mac.


A *prototype* of a [http://hudson-ci.org/ Hudson] based [http://en.wikipedia.org/wiki/Continuous_integration build server] for building FG (including OSG and SimGear) can be found at http://zakalawe.ath.cx:8080/
{{Note|
The output files on Jenkins download.flightgear.org and sf.net should always be the same. Jenkins pushes them to download.flightgear.org from where they get distributed to sf.net (with some renaming-magic involved). Direct dowloads from Jenkins should be avoided as much as possible. <ref>{{cite web
  |url    =  https://sourceforge.net/p/flightgear/mailman/message/35539513/  
  |title  =  <nowiki> Re: [Flightgear-devel] [from forum] Torrenting FlightGear </nowiki>
  |author =  <nowiki> Torsten Dreyer </nowiki>
  |date  =  Dec 8th, 2016
  |added  =  Dec 8th, 2016
  |script_version = 0.36
  }}</ref>


This is currently running on a core developer's home box:
}}
"I've found some spare hardware to run the build server (Hudson) on, and it seems bearable for me,
but I suspect less pleasant for people on the other end of my cable connection."[http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg28260.html]


So, the server will need a proper home if it moves beyond the prototype stage.
Some core developers are working with Linux and have no access to a Windows installation in order to test if what they have implemented for Linux will works for Windows or Mac. The build server will automatically start a new compilation after each commit is pushed to the source code repository. We have the ability to compile the source code for Linux, Windows and Mac.


For people who don't know, a build server talks to some slaves, and grabs/builds/tests/packages code. The current server is talking to one slave,
A "nightly build" is one that is automatically generated each night (hence the name) from the most recent code. Since it has not received as much testing as normal FlightGear versions, you might encounter some bugs; on the contrary, you get access to the latest-and-greatest features before they are officially released.
which is an Ubuntu VM (Virtual Machine) which is building  the 'next' branch on Gitorious.
Any slave could be a VM, of course - they use CPU resources while building, but unlike other projects, our commit rate isn't that high - the slaves will be idle most of the time (A Mac slave is also possible, but requires some more work).


'''Note:''' If anyone wishes to volunteer a proper server (with a reasonably symmetric connection) to run Hudson, please get in touch using the [http://www.flightgear.org/mail.html mailing list] or the [http://flightgear.org/forums/ FlightGear forums] - any Unix will do, for Ubuntu/Debian there's an easy apt.get source available. All the setup can be done remotely, given SSH access. The disk, memory, CPU and bandwidth requirements  are pretty moderate, due to our low commit rate.
<ref>{{cite web
  |url    = https://forum.flightgear.org/viewtopic.php?p=283958#p283958
  |title  = <nowiki>Re: Tutorial: How to get Aircraft Center to work</nowiki>
  |author = <nowiki>elgaton</nowiki>
  |date  = May 1st, 2016
  |added = May 7th, 2016
  |script_version = 0.34
  }}
</ref>


It does chew a bit of disk-space, since the master stores the artifacts for the last N builds, where N is configurable. The artifacts are a hundred megabytes or so, since it's all the header files, libs and binaries, though compressed of course.
== News (06/2015) ==
{{Main article|Building FlightGear - Cross Compiling}}
{{FGCquote
  |mingw has came a long way, and i think the MXE openscenegraph package (currently at 3.2.1 on master!!), is beautifully maintained, now it builds almost all core libraries dynamically with some argument passing, even as a static target(MXE_TARGETS{{=}}'i686-w64-mingw32.static'), but it's those plugins again, with their linking errors! i think these are because i'm using the i686-w64-mingw32.static-g++ compiler as opposed to the shared one...<br/>
<br/>
so as soon as i can get the shared build environment running and solve all of it's dependancies for OSG, i think we can have a cross compiller in our hands!
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=242914#p242914
    |title=<nowiki>Re: [SOLVED] Install osgEarth feature on Win7 64b with FG gi</nowiki>
    |author=<nowiki>hamzaalloush</nowiki>
    |date=<nowiki>Thu May 14</nowiki>
  }}
}}


= Goals =
== SourceForge release candidate ==


The objective of such systems is that there should be *zero* human steps to create a release - not just out of laziness, but for repeatability. I.e don't write a checklist or 'howto' of creating a release, write a shell script that does the steps. (Or several). And check those scripts into a source control system, too.
As of 2018.1 release candidates are available from sourceforce https://sourceforge.net/projects/flightgear/files/release-candidate/ <https://sourceforge.net/projects/flightgear/files/release-candidate/>;<br/>


In general, such systems are good for capturing how repeatable a build process is -  and the experiences on each of the Linux/mac/Windows slaves seem to confirm this.
== Where to find it ==
The FlightGear build server is available at: {{build link}}. There is no artifact available because we don't want copy +1GB to Jenkins master each days. This job is executed once per day.


Quoting [http://www.joelonsoftware.com/articles/fog0000000043.html The Joel Test: 12 Steps to Better Code]:


2. Can you make a build in one step?
You can get the artifact at {project infrastructure|buildserver}}
By this I mean: how many steps does it take to make a shipping build from the latest source snapshot? On good teams, there's a single script you can
run that does a full checkout from scratch, rebuilds every line of code, makes the EXEs, in all their various versions, languages, and #ifdef
combinations, creates the installation package, and creates the final media -- CDROM layout, download website, whatever.
If the process takes any more than one step, it is prone to errors. And when you get closer to shipping, you want to have a very fast cycle of fixing
the "last" bug, making the final EXEs, etc. If it takes 20 steps to compile the code, run the installation builder, etc., you're going to go crazy and
you're going to make silly mistakes.
For this very reason, the last company I worked at switched from WISE to InstallShield: we required that the installation process be able to run, from
a script, automatically, overnight, using the NT scheduler, and WISE couldn't run from the scheduler overnight, so we threw it out. (The kind folks at
WISE assure me that their latest version does support nightly builds.)
3. Do you make daily builds?
When you're using source control, sometimes one programmer accidentally checks in something that breaks the build. For example, they've added a new
source file, and everything compiles fine on their machine, but they forgot to add the source file to the code repository. So they lock their machine
and go home, oblivious and happy. But nobody else can work, so they have to go home too, unhappy.
Breaking the build is so bad (and so common) that it helps to make daily builds, to insure that no breakage goes unnoticed. On large teams, one good
way to insure that breakages are fixed right away is to do the daily build every afternoon at, say, lunchtime. Everyone does as many checkins as
possible before lunch. When they come back, the build is done. If it worked, great! Everybody checks out the latest version of the source and goes on 
working. If the build failed, you fix it, but everybody can keep on working with the pre-build, unbroken version of the source.
On the Excel team we had a rule that whoever broke the build, as their "punishment", was responsible for babysitting the builds until someone else
broke it. This was a good incentive not to break the build, and a good way to rotate everyone through the build process so that everyone learned how
it worked.


For additional information on the benefits of automated builds, please see: [http://www.joelonsoftware.com/articles/fog0000000023.html Daily Builds Are Your Friend (Joel on Software)].
{{FGCquote
  |We’ve improved the tool around daily build creation. Still some progress to be done but daily builds for Mac and Windows are uploaded to:<br/>
<br/>
https://sourceforge.net/projects/flightgear/files/unstable/
<br/>
(And hence, all the SF mirrors)<br/>
<br/>
And also directly available from:<br/>
<br/>
http://download.flightgear.org/builds/nightly/ <http://download.flightgear.org/builds/nightly/>;<br/>
<br/>
The ‘latest’ files in that directory are stable symlink, I am currently writing a descriptive web-page which will point to them since we expect the patch version number to evolve during 3.5 development (and I will be testing this soon, mostly to see what breaks!)<br/>
<br/>
All the usual caveats about un-tested code apply - although we will probably add some warnings over time to the builds themselves (splash screen etc) to remind people they’re not using a stable release.
  |{{cite web |url=http://sourceforge.net/p/flightgear/mailman/message/33628128/
    |title=<nowiki>[Flightgear-devel] Daily builds update</nowiki>
    |author=<nowiki>James Turner</nowiki>
    |date=<nowiki>2015-03-23</nowiki>
  }}
}}


In general, when people report that the 'current code doesn't compile', we can direct them to the Hudson page from now on.
== User info ==
[[File:About-dlg-2.10.png|180px|thumb|The about dialog]]


As a user you can help developers to track bugs from the latest source code, by tracking bugs you would report them on the [[mailing list]] or [http://sourceforge.net/p/flightgear/codetickets/ bug tracker] (not the forums).


= Status 07/2010 =
{{FGCquote
  |The [[Integrated Qt5 Launcher |GUI launcher]] included in these builds includes a new dialog to setup (and remember) the FGRoot using file picker. It does some version checking to hopefully make things as robust as possible for the user.
  |{{cite web |url=http://sourceforge.net/p/flightgear/mailman/message/33646906/
    |title=<nowiki>Re: [Flightgear-devel] Download description page</nowiki>
    |author=<nowiki>James Turner</nowiki>
    |date=<nowiki>2015-03-26</nowiki>
  }}
}}


The Mac build is pretty close to producing a nightly, though - we need to fix a genuine (and long-standing) configuration issue on Mac.
{{note|Take care to mention that you're using a nightly build, and to specify when it was built (you can find details in FlightGear's Help > About dialog)}}


The good news - thanks to some great documentation from Fred, the Windows build  is stable, using MSVC no less. You can grab a fgfs.exe (and .pdb) from any  successful build, and it should 'just work', if you drop it into a current 2.0 install. Future work on some improved packaging of these builds will follow.
If you are interested in doing this please follow these instructions:


The better news - all three platforms are now green (i.e, building correctly),  and should update reliably (which will be monitored). At present, the server admin gets emailed when a build breaks, and when it goes green again.
=== Linux ===
These build from Gitorious next, and there will probably be experiments to make it complain to IRC, or even to the mailing list, when the build breaks.
Well, unfortunately there is nothing interesting for you here :)


If desired, it is possible to add a mailing list or other individual addresses to the email  notifications.
Because Linux users have powerful tools to compile and use the latest source code, take a look at [[Scripted_Compilation_on_Linux_Debian/Ubuntu]]
Given our commit rate, it is not clear if it warrants a new mailing list or not - it depends how often we break the build :)


For any build, Hudson uses the Git changelogs to report what (and by whom!) is  new in the build.
=== Mac ===
Currently the master is being used to do the Linux builds, because it was easy - no particular reason it has to be done that way, though.
You need two things to run the latest source code: The compiled source code and the data.


* You can download the compiled source code at: [http://download.flightgear.org/builds/nightly/ FlightGear Mac nightly build]
* You can download the data with git: <code>{{fgdata clone}}</code>


= Hosting Options =
=== Windows ===
If you know of any others, please do feel free to add new hosting options here. Some of these are not necessarily useful for directly hosting Hudson, but instead for building FlightGear on different platforms using SSH. This applies in particular to the various build farms.
You need two things to run the latest source code: The compiled source code and the data.


* http://gcc.gnu.org/wiki/CompileFarm#How_to_Get_Involved.3F
* You can download the compiled source code at: [http://download.flightgear.org/builds/nightly/ FlightGear Win nightly build download]
* http://en.opensuse.org/Build_Service
* You can download the data with git: <code>{{fgdata clone}}</code>
* https://edge.launchpad.net/builders/
* http://hub.opensolaris.org/bin/view/Community+Group+testing/testfarm
* http://www.metamodul.com/10.html
* http://www.gnu.org/software/hurd/public_hurd_boxen.html
* http://www-03.ibm.com/systems/z/os/linux/support/lcds/


== Developer info ==
Here is a brief summary about the configuration of Jenkins.


= Benefits =
If need anything you would contact either James, Gene or Clément (the mailing list is a good place to contact them if you don't know where to look at them)
* lets developers know 'instantly' (within a few minutes) if their change broke 'some other platform', for example 64-bit or Mac (or Windows) (this is the big one, but only matters for developers)
* it can run tests automatically (although right now our test suite is pretty much zero)
* builds can be archived and uploaded somewhere. This doesn't help Linux much, but on Mac (and Windows, when it works), this means anyone can download a latest build and test it, with no need to install compilers, libraries or anything - just download a .zip and run bleeding-edge-FG.


The catch is, for this to be nice, requires some scripting. The current mac slave produces a zip, but you need to know some Terminal magic to actually run the code (set DYLD_LIBRARY_PATH, basically).
=== master ===
The ''master'' manages the distribution of builds among the slaves.


= Issues =
Basically, its job is to be connected to the slaves and ask them to execute the jobs which have been configured.
* The current mac slave produces a zip, but you need to know some terminal magic to actually run the code (set DYLD_LIBRARY_PATH, basically).


= Plans =
=== Linux ===
This is the Linux slave based on Fedora 21. In reality it is the same machine than ''master'' but with another login. That's why we want the SSH connection on ''localhost''.


The configuration is exportable as XML  files, the server is currently using the official Hudson apt-get package for Ubuntu, so it's a
This slave is configured to work in <code>/home/jenkins</code>, that's where we have plenty of free space disk on this machine.
fairly repeatable setup. Configuring the Windows slave VM with mingw is proving the biggest hassle - OSG is working.


'Soon' there will be a WinXP slave, with a MinGW build. Hopefully this will even extend to a NSIS installer script, if Fred has one lying around. At
=== Mac ===
which point we should have nightly installers available for Windows, and a happier Fred. (A VisualStudio build is also possible, but requires more
This is the Mac slave.
interaction with someone else, who has an externally-addressable/tunnel-able box with VS installed).


=== Windows ===
This is the Windows slave based on Windows Server 2008. It is located in the same LAN than master/Linux slave. There is MSVC2010 installed.


At which point, doing a release means clicking a button on a webpage (on Hudson), and letting the slaves grind away for an hour or so. Magic!
This slave is configured to work in <code>G:\jenkins</code>, that's where we have plenty of free space disk on this machine.


= Options =
=== Release process ===
The Linux slave has the responsibility to provide the base-package for the Linux-release and Windows-release job.


Because the Linux and Windows slave are on the same LAN it is not a problem to exchange 1 GB on the network.


* Another thing the server can do, is email/IRC people when the build breaks on Linux / FreeBSD / Mac / Win due to a commit - obviously very handy for the devs.  
<code>Windows-release</code>, <code>Mac-release</code> and <code>Linux-release</code> have to be triggered by-hand. The result is automatically uploaded to http://fgfs.goneabitbursar.com/releases/.


* So, another step is to email committers directly when they break the build. If you're a commiter to FG or SG, you may see some odd emails
[[Category:Core developer documentation]]
 
[[Category: Release]]
* set up a buildser...@flightgear.org address.
 
* Yet another thing it can do is run test suites - unfortunately we don't have many such tests at the moment.
 
* If anyone wants to get into providing nightly .debs or .rpms, that could also be done, but requires people who know those systems, and again can provide a suitable externally address slave to run the builds.
 
* If there's other configurations people wish to test (64-bit Linux, in particular), get in touch and they can be added.
 
* If it's just for running the monitor, then we probably should talk about putting it onto The MapServer as well
 
* Build jobs can run arbitrary shell scripts - they can tag things in CVS or Git, they can create tarballs, upload files to SFTP/FTP servers, the works. So, if Durk/Curt/Fred could codify, somewhere, the steps (in terms of 'things doable in a shell/.bat script') to create an FG pre-release and final-release, the process can be automated.
 
* Set up a cross compiler version of gcc at flightgear.org to automatically create binary packages (releases) of FlightGear for platforms such as Win32
 
= Related Discussions =
* http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg27592.html
* http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg27918.html
* http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg28260.html

Latest revision as of 10:28, 29 March 2018

The FlightGear build server, sometimes called the nightly build server or just Jenkins, is a system based on Jenkins which helps core developers to check that their source code compiles correctly for all OS. It is also responsible for building release packages for Linux, Windows and Mac.

Note

The output files on Jenkins download.flightgear.org and sf.net should always be the same. Jenkins pushes them to download.flightgear.org from where they get distributed to sf.net (with some renaming-magic involved). Direct dowloads from Jenkins should be avoided as much as possible. [1]


Some core developers are working with Linux and have no access to a Windows installation in order to test if what they have implemented for Linux will works for Windows or Mac. The build server will automatically start a new compilation after each commit is pushed to the source code repository. We have the ability to compile the source code for Linux, Windows and Mac.

A "nightly build" is one that is automatically generated each night (hence the name) from the most recent code. Since it has not received as much testing as normal FlightGear versions, you might encounter some bugs; on the contrary, you get access to the latest-and-greatest features before they are officially released.

[2]

News (06/2015)

1rightarrow.png See Building FlightGear - Cross Compiling for the main article about this subject.

Cquote1.png mingw has came a long way, and i think the MXE openscenegraph package (currently at 3.2.1 on master!!), is beautifully maintained, now it builds almost all core libraries dynamically with some argument passing, even as a static target(MXE_TARGETS='i686-w64-mingw32.static'), but it's those plugins again, with their linking errors! i think these are because i'm using the i686-w64-mingw32.static-g++ compiler as opposed to the shared one...


so as soon as i can get the shared build environment running and solve all of it's dependancies for OSG, i think we can have a cross compiller in our hands!


Cquote2.png

SourceForge release candidate

As of 2018.1 release candidates are available from sourceforce https://sourceforge.net/projects/flightgear/files/release-candidate/ <https://sourceforge.net/projects/flightgear/files/release-candidate/>;

Where to find it

The FlightGear build server is available at: http://build.flightgear.org:8080/. There is no artifact available because we don't want copy +1GB to Jenkins master each days. This job is executed once per day.


You can get the artifact at {project infrastructure|buildserver}}

Cquote1.png We’ve improved the tool around daily build creation. Still some progress to be done but daily builds for Mac and Windows are uploaded to:


https://sourceforge.net/projects/flightgear/files/unstable/
(And hence, all the SF mirrors)

And also directly available from:

http://download.flightgear.org/builds/nightly/ <http://download.flightgear.org/builds/nightly/>;

The ‘latest’ files in that directory are stable symlink, I am currently writing a descriptive web-page which will point to them since we expect the patch version number to evolve during 3.5 development (and I will be testing this soon, mostly to see what breaks!)

All the usual caveats about un-tested code apply - although we will probably add some warnings over time to the builds themselves (splash screen etc) to remind people they’re not using a stable release.


— James Turner (2015-03-23). [Flightgear-devel] Daily builds update.
(powered by Instant-Cquotes)
Cquote2.png

User info

The about dialog

As a user you can help developers to track bugs from the latest source code, by tracking bugs you would report them on the mailing list or bug tracker (not the forums).

Cquote1.png The GUI launcher included in these builds includes a new dialog to setup (and remember) the FGRoot using file picker. It does some version checking to hopefully make things as robust as possible for the user.
— James Turner (2015-03-26). Re: [Flightgear-devel] Download description page.
(powered by Instant-Cquotes)
Cquote2.png
Note  Take care to mention that you're using a nightly build, and to specify when it was built (you can find details in FlightGear's Help > About dialog)

If you are interested in doing this please follow these instructions:

Linux

Well, unfortunately there is nothing interesting for you here :)

Because Linux users have powerful tools to compile and use the latest source code, take a look at Scripted_Compilation_on_Linux_Debian/Ubuntu

Mac

You need two things to run the latest source code: The compiled source code and the data.

  • You can download the compiled source code at: FlightGear Mac nightly build
  • You can download the data with git: git clone git://git.code.sf.net/p/flightgear/fgdata/

Windows

You need two things to run the latest source code: The compiled source code and the data.

Developer info

Here is a brief summary about the configuration of Jenkins.

If need anything you would contact either James, Gene or Clément (the mailing list is a good place to contact them if you don't know where to look at them)

master

The master manages the distribution of builds among the slaves.

Basically, its job is to be connected to the slaves and ask them to execute the jobs which have been configured.

Linux

This is the Linux slave based on Fedora 21. In reality it is the same machine than master but with another login. That's why we want the SSH connection on localhost.

This slave is configured to work in /home/jenkins, that's where we have plenty of free space disk on this machine.

Mac

This is the Mac slave.

Windows

This is the Windows slave based on Windows Server 2008. It is located in the same LAN than master/Linux slave. There is MSVC2010 installed.

This slave is configured to work in G:\jenkins, that's where we have plenty of free space disk on this machine.

Release process

The Linux slave has the responsibility to provide the base-package for the Linux-release and Windows-release job.

Because the Linux and Windows slave are on the same LAN it is not a problem to exchange 1 GB on the network.

Windows-release, Mac-release and Linux-release have to be triggered by-hand. The result is automatically uploaded to http://fgfs.goneabitbursar.com/releases/.