Triaging and improving bug reports

Testing and triaging tickets

SourceForge tracker is our bug tracker, it's not the smartest system but workable.

We encourage people to file tickets into the system, but some people report a ticket and then vanish or don't follow it up. Therefore we have a large collection of older bug reports to re-check and classify, and help with this is appreciated. If you're unsure about details, do ask on the developer list or similar. It's quite okay to ask 'what might be causing this?' and get more opinions. You need to ask for permissions to edit certain features of bugs, ask one of the admins for permissions.

We do also collect aircraft and scenery bug reports in the tracker, but these don't always make sense. For an aircraft which is unmaintained it's reasonable to track issues ourselves, but where development occurs in another place, or with an active development team, it's better to close the issue in our tracker, after pointing the aircraft developers at the issue. For scenery bug reports there is usually nothing we can do without a full scenery rebuild, unfortunately (trees on airport, taxiway underwater, missing new runways, etc).

Feature-request tickets also need handling: the tracker is not really ideal for these, better to propose them on the forum or the developer list and see if anyone is interested to work on them, or add them to a wiki page of requests. But they can be tagged with the 'feature' label and then marked as 'WontFix' : this gives us a way to find them with a query, but avoids them being counted towards the open bug totals for a milestone.

Add missing data

Especially, add labels for the area: replay, launcher, Canvas, crashes. Set the affected milestone: since SF only allows one milestone, not multiple, set the milestone the bug occurs in now: we don't care when the bug first appeared.. That likely means the current release. Consider renaming the ticket if the title is not accurate: new users often don't pick clear titles for bugs, since they focus on the wrong issue

For crashes, try to get a crash-trace with symbols, either from the reporter or if you can reproduce, your own build. For other bugs, ask to get the log file if it makes sense: especially FDM issues, or visual ones (missing texture or shaders that don't compile)

For visual things (broken GUI), screenshots are very helpful.

For aircraft, please ask for which version of the aircraft was tested: it needs to be a defined version in FGAddon or a git-repo, not just 'the latest F-14', since that changes continuously. Same for any add-on or custom scenery.

Check the bug still occurs

This is the most important: follow the steps to reproduce the bug. If you can't, ask the original reporter for more info, or find a friend to check (eg on a different platform or with a different video card). If multiple people can't reproduce the issue, consider closing the bug as 'not reproducible', but take care to allow for bugs which depend on the system configuration, eg language, OS, or other factors which are hard to identify.

If the steps to reproduce are unclear, set the state as 'Need Info' and ping the original reporter. If you find a bug which is in this state (waiting for more info from the original reporter) and they didn't update the ticket in say six months (or maybe a year), consider closing the ticket as 'WontFix', since there is no value in tracking such tickets. Often you need to check with the reporter how their system differs from a known working setup: are the running from a network share, have they changed their download directory, do they have a firewall from their ISP which blocks certain ports?

If you can reproduce the bug in the current version, update the milestone.

Simplification

Try to eliminate possible causes: if the bug was reported with a complex aircraft, does it also happen with the UFO? If it happens with a high shader setting, does it go away with a lower one? It does help to avoid strange customizations in your test setup: try to be testing something as close to stock as possible. This likely means keeping an unmodified version of the current and previous versions around, don't use a development build for triaging. Fortunately SSDs got cheap, you might need more disk space.

De-duplication

This requires some knowledge of other bugs, so consider asking on the developer list, but if the bug is about one aircraft, consider if the same bug might exist in other aircraft. For example, if sounds are broken in one aircraft, do they work in another one? Is something broken in all gliders, or all Yasim helicopters, or just this one particular aircraft? Since many aircraft copy code from each other, ask or check if the function is common and see what other aircraft might be affected. Search on similar labels or titles, unfortunately the SF tracker search is not very smart. (This is why good titles are important, to help finding duplicates)

If you do find a duplicate, add a comment on both tickets with a link to the other one. That way, when the original ticket is fixed, both bugs can be verified.

Expired / ancient bugs

If the bug is more than two years old, and is not moving forward (no way to reproduce, no further info from the original reporter), close it as 'expired' or 'invalid'. If it's a feature request which no one seems interested in after a year, close it as 'wont-fix'. Or if you feel someone might be be interested to work on it, raise on the developer list or forum.

Regressions

If possible, check the behaviour in older versions to see if it's a regression or always behaved the same way. This can take more time so be strategic about it : test 2018.3 first, and then decide if it's worth checking older or newer versions. But it's probably more useful to check twenty tickets in the current version, than spend four hours installing a historical one, to research a very old ticket.