20,741
edits
Cppunit effort: Difference between revisions
m
→Background
(→Status) |
m (→Background) |
||
| Line 27: | Line 27: | ||
== Background == | == Background == | ||
Edward developed this as part of his [[FGPythonSys]] experiments to have full system test coverage. The test suite is based on the venerable, yet old CppUnit framework. Edward has spun this out into its own standalone branches rebased to 2016.4.0 and made independent of embedded Python. | |||
The test suite is an almost direct Python->C++ translation my own | |||
comprehensive test suite for one of my own software projects [4]. The | |||
benefits of such a framework, assuming a good test coverage, include: | |||
* Enabling quick and brutal refactorisation of the entire code base - ensuring that the end result works perfectly well. | |||
* Interest in independence and modularisation of all FlightGear components to simplify tests. | |||
* If all subsystems are tested for standalone creation, destruction, init(), reinit(), shutdown(), etc., then repetitive resetting of FlightGear should work flawlessly. | |||
* Allow valgrind and other heavy tools to be run on the absolute minimal code paths, by running individual tests. | |||
The functionality of the test suite can be very much expanded in the future. For example there are CppUnit classes for XML output, rather than text, so this test suite is ideal for automated testing and presenting the results through web pages (e.g. Jenkins). | |||
As a demonstration there are now 4 initial categories of tests: | |||
# System/functional tests (currently 0 tests). | |||
# Unit tests (currently 2 tests). | |||
# GUI tests (currently 0 tests). | |||
# Simgear unit tests (currently 2 test). | |||
There are an addition 242 unit tests in the FGPythonSys branches. One | |||
of the unit tests and one of the simgear unit tests are the old | |||
tests/test-mktime.cxx and tests/test-up.cxx programs respectively (the | |||
rest of the tests/ directory might be better suited for FGMeta). A | |||
list of the test suite features are: | |||
* Run 'make test_suite' to build the test suite as the $BUILD_DIR/test_suite/run_test_suite standalone binary (with full independence from fgfs) and to run the test suite. | |||
* The absence of CppUnit causes 'CppUnit not found' to be printed by CMake, then nothing happens. | |||
* No CMake installation target, so it remains in the build directory. | |||
* A setUp() function called before every test that can be customised for each grouping or suite of tests (for FGPythonSys, set up a new subsystem instance, and inject an empty property tree). | |||
* A tearDown() function pre suite unconditionally called after every test (for FGPythonSys, delete the subsystem). | |||
* Minimal output of '.', 'F', and 'E' per test for a pass, fail, or error state [6]. | |||
* Comprehensive simgear logstream and STDOUT+STDERR capture and reporting only for failed tests [7] (printf() function calls cannot be captured). | |||
A final synopsis of all tests. | |||
For full details, see the branches. The framework is powerful enough | |||
to bring in all types of tests under one roof, including all those | |||
with main() functions and currently scattered all over the place. | |||
<ref>{{cite web | |||
|url = https://sourceforge.net/p/flightgear/mailman/message/35661026/ | |||
|title = <nowiki> [Flightgear-devel] A FlightGear test suite using CppUnit. </nowiki> | |||
|author = <nowiki> Edward d'Auvergne </nowiki> | |||
|date = Feb 11th, 2017 | |||
|added = Feb 11th, 2017 | |||
|script_version = 0.36 | |||
}}</ref> | |||
== Challenges == | == Challenges == | ||