FlightGear benchmark: Difference between revisions

{{Stub}}

I think the devs should define a set of standard settings and perhaps a flight recording to benchmark flightgear on, to help determine a computer's suitability to run flightgear. Obviously user submitted benchmarkings are pretty different due to different software and settings, so a set of standards like the benchmarking in arma2/just cause2 etc. would be awesome.

eg. low settings benchmark would be a 10 min flight on a low-polygon airplane and simple terrain, and a high settings benchmark would be everything "maxed out" in high res, in heavily complex scenery, in like a thunderstorm with 100 AI aircraft perofrming CPU intensive maneuvers in close proximity etc. (I obviously don't know how fg works and which things are most CPU/GPU intensive)

Obviously a flightgear specific benchmark could be much more suitable for flightgear than a generic gaming benchmark and much more helpful for people figuring out what settings are best for their systems, so what do you guys think of the idea?

I am totally new to flightgear and I haven't seen any indication that fg has a benchmarking system, so I thought that would be nice. I could provide/specify a list of settings and record some flights (if said feature exists), although I doubt I have the fg experience and authority to do so, so I hope you guys can sort this out- besides, a benchmark system should be pretty easy to implement compared to like say adding more realism/ better graphics/ revamping the engine, so I think that including a set of specific benchmarking tools and settings is plausible

* maybe we really do need a benchmark .fgfsrc for proper comparison. [http://flightgear.org/forums/viewtopic.php?f=5&t=16204&p=156440&hilit=benchmark#p156437]

* Well, while ago, we talked about creating benchmarks in the form of custom aircraft-set.xml files, which would already contain all startup settings (resolution, bpp, shaders etc) [http://flightgear.org/forums/viewtopic.php?f=5&t=16204&p=156440&hilit=benchmark#p156440]

* the simplest option would seem to be using existing stuff. After all, this is just about recording and replaying properties. And that's exactly what the new flight recorder (replay tapes) system does. So we could simply abuse it a little to also provide a configuration to sample the various rendering properties (see rendering dialog) and implement a benchmark, which should give us a way to reproduce settings fairly well. [http://flightgear.org/forums/viewtopic.php?p=176025#p176025]

* We do have a so called "flight recorder/replay" system that can save flights. The whole system is property-driven, and it is possible to provide custom sets of properties that should be recorded. In other words, it would be possible to create a custom "flight recorder" configuration that doesn't just record aircraft settings, but also rendering related settings [http://flightgear.org/forums/viewtopic.php?f=6&t=19202&p=177733&hilit=benchmark#p177741]

* Still, the idea of creating a simple, easily recreatable benchmark flight, sounds good to me! What aircraft would be suitable? Is the UFO in the standard installation? I don't know, since I use the fgdata from git for years... Then I would make some flight over KSFO, which should be on every FG installation, and then? What would I do with the recorder tape? upload it somewhere? [http://flightgear.org/forums/viewtopic.php?f=18&t=19738&p=181625&hilit=benchmark#p181625]

* Even a very simple benchmark could be useful for feature-scaling and regression testing purposes, and if it's implemented in a non-interactive fashion, it could even help with regression testing. Ideally, a benchmark would start out with the bare minimum settings and then dynamically change settings on the fly to determine their effect on frame rate and frame spacing, to come up with a list configuration settings that work properly, while ensuring a satisfying simulator experience. We already have various building blocks in FG to do most of this, it's really just a matter of combining and integrating existing features to provide such a simple benchmark. From a troubleshooting perspective this could in fact also be useful, because we could ask users to open a certain dialog and run a certain benchmark and report the results here. [http://flightgear.org/forums/viewtopic.php?f=6&t=19202&p=177778&hilit=benchmark#p177778]

{{cquote|could some thought be given to producing a benchmark suite for Flightgear. It would need to take in all of the, by now well known, variables - making it by no means a simple beast to manage.

If this could be automated in some way it would be much easier to capture, and then submit, consistent data. <ref>{{cite web |url=http://www.mail-archive.com/flightgear-devel@lists.sourceforge.net/msg40294.html|title=<nowiki>[Flightgear-devel] Benchmark matrix</nowiki>|author=Alan Teeder|date=Thu, 20 Jun 2013 10:15:42 -0700}}</ref>|Alan Teeder}}