From FlightGear wiki
Revision as of 12:43, 7 September 2016 by Wkitty42 (Talk | contribs) (areipped-up -> are zipped)

Jump to: navigation, search
This article is a stub. You can help the wiki by expanding it.

1rightarrow.png See Towards better troubleshooting for the main article about this subject.

the Windows builds (and the 3.0 release, assuming no issues are encountered) now includes a crash-reporting framework, to help getting higher quality crash reports [1]

If you experience a crash, please send a report using the dialog which appears - you can provide some additional info if you wish. Your log file, some system info (processor, time of day, Windows version, path to fgfs.exe) and a crash dump are zipped and sent to a server, and using some additional steps a real back-trace can be produced. (We need to write a privacy policy document for this, current the URL is a 404 - basically saying we promise not to do anything evil with the report info, although I’m not sure how much evil is possible since it’s the same info we ask for in a bug-report - I have switched off the options to include registry contents, and FGFS doesn’t touch the registry anyway) (For the curious, I’m not using Google Breakpad because it’s huge and seems somewhat inactive, instead I’m using CrashRpt which is very compact, completely open-source and new-BSD-licensed: - although we do need to include a copy of their license.txt file somewhere in the installer) [2]

CrashRpt was something James wanted to get on the release branch without being sure whether to disable it for non-release branches entirely, or support it in the Windows nightly builds, where it would potentially be useful. (In which case there are further change to be done, since the nightlies use a different installer script)[3]

Originally James only enabled CrashRpt for release builds, not nightlies, butit has been enabled since then. Also James explicity *disabled* the video capture support, because he felt the overhead of it would be too invasive when running FG, and the amount of video you’d need to record to make any difference to most crashes would be large.[4]

  1. zakalawe (Oct 19th, 2013). Re: Loading forever: "loading navigation data".
  2. James Turner (Jan 23rd, 2014). [Flightgear-devel] Release candidates.
  3. James Turner (Jan 26th, 2014). Re: [Flightgear-devel] Some feedback.
  4. James Turner (Jul 18th, 2014). Re: [Flightgear-devel] Win 64bit nightly is broken.

Help wanted

There is one other Windows area Jamees could really use some help with - he added CrashRpt integration several releases ago and has a web server full of crash data.

To make this useful someone with Windows skills needs to write some tooling to automaticly match a crash report to the archived PDB file and hence produce a usable back-trace.

We could then start analysing the most common sources of crashes. If that’s something you have any interest in, James can explain in more detail.[1]

someone who knows Windows (and has some time available) needs to actually glue everything together using those tools and scripts. Ideally in a way we can run on the Jenkins slave or similar as an hourly job. That’s the current piece of the flow I never found time to work on myself, yet. [2]

the build-ID need to be matched to the debug symbols (.PDB) which the Jenkisn Windows build archives for releases. What’s missing as I said in the emails a long time ago, is the scripting to turn mini-dump + PDB files into a a backtrace with function names, if not line numbers. That would be a huge benefit. I’m happy to give anyone read access to my server to try this, and adjust where / how Jenkins archives the PDBs - but not being on Windows, my energy and ability to move this forward myself is very low. [3]

Crash Report Analysis

there are the following solutions:

  1. get them from James' server and process them manually (e.g. via the script I mentioned in my previous e-mail, altered as necessary): pros: no changes in existing libraries/infrastructure, dumps remain private and accessible only to trusted developers; cons: Windows-only, some manual steps and script editing are needed;
  2. use an automated solution like Dr Dump [1], which can be configured to accept CrashRpt reports; pros: only a small code change is needed, the service automatically analyes crash reports and groups them according to their stack traces (it is only needed to upload .pdb symbols); cons: Windows-only, possible privacy concerns (dumps might contain portions of memory storing personal data, and some people might not be comfortable with storing them on a third-party service);
  3. change the crash reporter to Google Breakpad [2]/Crashpad [3] and use an already developed frontend, like Socorro [4] or the Mini Breakpad Server [5], or some custom scripting; pros: cross-platform bug reports; cons: integrating Breakpad is not trivial (as well as huge [6]); Socorro is heavyweight (at least in my opinion).

MD5 errors are normal - the address I gave you is the one where the error reports are uploaded by CrashRpt itself (I should have specified that), and the crashrpt.php script requires several parameters to be passed, including an MD5 hash of the minidump (I think), hence the message. There is currently no public interface to get crash reports.[5]

Right now, they are (unfortunately) archived there (see the end of [1]). You can help by writing a tool to match crash reports to PDB files; I've written a small script [2] which could help you get started.

[1] [2][6]

Cquote1.png Might sth like a bug-report (or maybe crash-report?) feature be of help? We got a lot of information in the property-tree, like GPU-drivers etc. For instance, do we have OS and stuff like that too? Might be fairly easy to put this important info into a tmp-file, and in the case we crash, add it to some kind of http-request to An experience-report might be of help, what frequently asked questions do you pose all the time? + the red box...
Cquote1.png people have been working on features to provide backtraces to developers without end-users having to be aware of the whole process, as in the whole crashrpt thing that Zakalawe implemented - but all this is still in its infancy unfortunately. Just keep in mind that FlightGear users are not dealing with a finished or "polished" product - things are very much in flux, and people can be part of the evolutionary process here.
— Hooray (Apr 22nd, 2014). Re: Forum communication.
(powered by Instant-Cquotes)


  1. James Turner (2015-03-23). Re: [Flightgear-devel] SimGear/FlightGear Portability Patches.
  2. James Turner (Mar 23rd, 2015). Re: [Flightgear-devel] SimGear/FlightGear Portability Patches.
  3. James Turner (May 4th, 2016). Re: [Flightgear-devel] Info on windows CrashRpts....
  4. Alessandro Menti (May 4th, 2016). Re: [Flightgear-devel] Info on windows CrashRpts....
  5. Alessandro Menti (May 4th, 2016). Re: [Flightgear-devel] Info on windows CrashRpts....
  6. Alessandro Menti (May 3rd, 2016). Re: [Flightgear-devel] Info on windows CrashRpts....