Towards better troubleshooting: Difference between revisions
Line 6: | Line 6: | ||
== Background == | == Background == | ||
{{FGCquote | |||
|1= Whatever release plan we come up with, it will never be able to solve our issues with respect to bug reporting/fixing. We have all the required infrastructure to do better here: the bugtracker, a forum, a mailing list and we even send crash reports - at least on windows. What we do not have are the people dealing with that. We need: - people filing proper bug reports - people classifying/reviewing the bug reports - people analying the crash reports - people fixing the bugs | |||
|2= {{cite web | |||
| url = http://sourceforge.net/p/flightgear/mailman/message/34634520/ | |||
| title = <nowiki>Re: [Flightgear-devel] Some thoughts about the release process</nowiki> | |||
| author = <nowiki>Torsten Dreyer</nowiki> | |||
| date = Nov 20th, 2015 | |||
}} | |||
}} | |||
{{FGCquote | {{FGCquote | ||
|1= whether we do frequent automated releases or single hand-picked releases, the Achilles heel for the whole process is going to be getting enough test data and processing it properly. | |1= whether we do frequent automated releases or single hand-picked releases, the Achilles heel for the whole process is going to be getting enough test data and processing it properly. |
Revision as of 11:54, 20 November 2015
Troubleshooting |
---|
These are some ideas towards better troubleshooting that we usually come up with after each release to improve the quality of end-user bug reports. Most of these are fairly trivial to implement, it just seems that we don't typically have a need for them until someone shows up having a real problem who is not able to build from source... It mostly boils down to showing additional information in the about dialog, or including it in the startup log file created in $FG_HOME, or maybe even making it available in the new crash reporting tool eventually.
You may also want to check out this post on the forum: Re:Helping users in the long term
Background
Whatever release plan we come up with, it will never be able to solve our issues with respect to bug reporting/fixing. We have all the required infrastructure to do better here: the bugtracker, a forum, a mailing list and we even send crash reports - at least on windows. What we do not have are the people dealing with that. We need: - people filing proper bug reports - people classifying/reviewing the bug reports - people analying the crash reports - people fixing the bugs — Torsten Dreyer (Nov 20th, 2015). Re: [Flightgear-devel] Some thoughts about the release process.
(powered by Instant-Cquotes) |
whether we do frequent automated releases or single hand-picked releases, the Achilles heel for the whole process is going to be getting enough test data and processing it properly. — Thorsten Renk (Nov 19th, 2015). Re: [Flightgear-devel] Some thoughts about the release process.
(powered by Instant-Cquotes) |
Usually reports range from 'FG sucks' all the way to suggested C++ patches to fix an issue (with a strong bias towards the former). Ironically, we get at the same time too little and too much information - too little, because often people just write a frustrated post that FG version X crashes all the time and they reverted to the previous version - so we know there is an issue — Thorsten Renk (Nov 19th, 2015). Re: [Flightgear-devel] Some thoughts about the release process.
(powered by Instant-Cquotes) |
I feel that getting the information better organied and brought to the attention of the relevant maintainers would be the key to make better stable releases - but I am at a loss how to achieve that. — Thorsten Renk (Nov 19th, 2015). Re: [Flightgear-devel] Some thoughts about the release process.
(powered by Instant-Cquotes) |
however we do it in the future - this is what I see as the most pressing issue to solve - how to improve the flow of information from users to developers without making everyone unhappy in the process. — Thorsten Renk (Nov 19th, 2015). Re: [Flightgear-devel] Some thoughts about the release process.
(powered by Instant-Cquotes) |
Might sth like a bug-report (or maybe crash-report?) feature be of help? We got a lot of information in the property-tree, like GPU-drivers etc. For instance, do we have OS and stuff like that too? Might be fairly easy to put this important info into a tmp-file, and in the case we crash, add it to some kind of http-request to flightgear.org. An experience-report might be of help, what frequently asked questions do you pose all the time? + the red box... — chris (Nov 19th, 2015). Re: [Flightgear-devel] Some thoughts about the release process.
(powered by Instant-Cquotes) |
we should absolutely stop telling anyone to edit preferences.xml in FG_ROOT; any documentation or advice which says to should be changes ASAP.
— zakalawe (Sat Oct 26). Re: NavCache:init failed:Sqlite error:Sqlite API abuse.
(powered by Instant-Cquotes) |
Trivial
These are already available in the property tree, i.e. just requires editing about.xml:
- Expose threading mode Not done
- Expose renderer mode (classic vs. rembrandt / osgEarth) Not done
- Expose availability of Built-in Profiler Not done
- show location of $FG_HOME in help/about dialog as a button to open it via file explorer ?
- show checksum of preferences.xml using the md5() Nasal API in the About dialog to ensure that people didn't tamper with it
- consider adding the checksum of preferenfces.xml to the binary, so that startup code can identify if files were modified that should not be changed
- should show if fgfsrc is available/used
- use intel detection heuristics and adapt those to detect AMD/ATI cards and show a startup warning to inform the user that PUI/styling may interfere with effects/shaders
- we keep seeing people who install mismatching versions of aircraft/fgfs - introduce a version-check property so that -set.xml files can encode version requirements, which are checked by a script in $FG_ROOT/Nasal [1]
Core changes
These will typically require changes on the C++ side, but would still be good to have whenever someone makes a bug report:
- provide a startup option/property to disable aicraft-set.xml Nasal code for troubleshooting purposes
- log frequency of Nasal callback invocation to the console to detect fdm coupled code ?
- dump node-specific osg camera stats to the console/log file (or even splash screen) during startup for the scenery/aircraft ROOT nodes[2]
- separate OpenGL/GLSL logging/log file for troubleshooting graphics related startup errors rendering FG useless (e.g. faulty drivers) [3]
- case-sensitive path checking [4]
- write the current pid (process ID) to the property tree, so that this can be used for debugging purposes (or even just for the suffix of log files) Not done
- add startup options to override/ignore startup files and trigger a navdb cache rebuild [5] Not done
- expose number of active timers/listeners[6] Not done
- introduce a tag for FDM properties to optionally notice listeners being tied to FDM rate, as per [7]
- provide support for an optional TRACECB (callback) mode where multiple identical callbacks are reported ? Not done
- we could also provide a property and use it as a "soft limit" for the number of listeners per property before a warning is shown, e.g. ~20-50 ? Not done
I used two places to check how many Uniforms are created. EDXH (Isle of Helgoland in the North Sea) - barely any scenery; And KSFO with AI Traffic enabled, you know the scenery complexity.At EDXH, the Effect system attaches approx. 1500 Listeners to the property tree but only to 17 unique properties. At KSFO, the Effect system attaches some 10000 Listeners to the property tree - but also only to 17 unique properties. That's with shader quality set to zero (all off).
On my old Laptop, (1.6GHz dual core, GeForce Go 7400, 3Gb RAM) FlightGear has becom barely usable over the last few years. Framerates of max 20 at EDXH and single digits at KSFO. — Torsten Dreyer (2014-09-04). Re: [Flightgear-devel] crash in SGPropertyNode::fireValueChanged.
(powered by Instant-Cquotes) |
- Subsystem-level Memory Tracking for FlightGear (RAM/VRAM, CPU/GPU utilization) Not done
- For better diagnostics, and better end-user bug reports, we could consider exposing a cross-platform process and system utilities module via Nasal/CppBind, such as e.g. psutil (Windows, MacOS & BSD/Unix) Not done
- Nasal callback tracking (timers/listeners) [8]
- Nasal callback tracing, i.e. identical callbacks being called more than once per frame ?
- Expose scenery version info [9] [10] Not done
- Expose all startup arguments (would just be copied to a single property) and log them to the log file [11] Not done
- Expose complete content of fgfsrc (could be read into the property tree) Not done
- Expose cmake/compiler flags Not done
- Expose release/debug build info (osg,simgear & fg) Not done
- Expose architecture (32bit/64 bit) e.g. via __i386__ and __x86_64__ on Intel or via cmake [12] [13] Not done
- Expose amount of physical RAM available Not done
- Expose amount of used RAM Not done
- Expose degree of swapping Not done
- Provide some means to show GLSL errors in a dedicated place, i.e. the loglist ? Not done
- Maybe provide an option to attach the startup log file to a flight recorder "tape" so that people can more easily share reproducible flights ?
- Keep track of Nasal callbacks running via timers/listeners and maintain a list of origin (FILE/LINE, handle/identifier) so that we can inspect a list of "Nasal processes" [14]
- Investigate providing support for Nasal backtraces across timers/listeners as per Philosopher's posting [15]
- Extend Initializing Nasal early to provide a startup mode where the Nasal engine will be fully shut-down at run-time for better troubleshooting
Runtime stuff
- provide a switch to enable/disable updating of canvas/groups on/off (or placements?), to help with troubleshooting
- Investigate extending Canvas::Element to optionally keep track of memory required for each element (use/sub-class osg::ref_ptr), so that we can provide some runtime info Not done
- Provide a means to list all running Nasal callbacks invoked via timers/listeners[16] Not done
- Extend FGStatsHandler to expose Nasal GC stats via osg::StatsHandler Not done
The built-in osgviewer stats can be extended with custom stats, that works by subclassing osg::StatsHandler, this is already done in $FG_SRC/Viewer/FGEventHandler.?xx The class can be extended to add your own stats via osgViewer::StatsHandler::addUserStatsLine() |
- Canvas::Element/Canvas::Group should be extended to gather SGTimeStamp-based stats for each group/element, so that people can better tell what's going on behind the scenes [17] Not done
- Use a separate draw mask for cockpit panels (e.g. via a corresponding XML attribute that serves as a marker), so that the impact of heavy cockpits can be better evaluated Not done
- Whenever there's an exception while booting, we should try to re-init the sim and increase the log level to provide crashrpt with better info [18]