|
|
|
migrated from https://git.ligo.org/CW/software/lalsuite/-/wikis/manually-run-test-scripts
|
|
|
|
|
|
|
|
# general info
|
|
|
|
|
|
|
|
* LALPulsar maintainers: Karl Wette, David Keitel
|
|
|
|
* for developers: LALPulsar uses a special "test harness" which wraps convenience features such as automated setup of paths, extraction of reference files from tarballs, etc. This also brings with it a few peculariaties on how to run tests, see below.
|
|
|
|
|
|
|
|
# manually run test scripts
|
|
|
|
|
|
|
|
With the special LALPulsar "test harness", it's not possible to just do for example "./testComputeFstatistic_v2.sh" in the source directory. But don't worry, the special way isn't actually any more difficult, but more general and cleaner!
|
|
|
|
|
|
|
|
Here's a first example of how to run a single test:
|
|
|
|
|
|
|
|
$ cd lalpulsar/bin/Fstatistic/
|
|
|
|
$ make check TESTS=testComputeFstatistic_v2
|
|
|
|
...
|
|
|
|
make[3]: Entering directory '/home/karl/Software/lalsuite/lalapps/src/pulsar/Fstatistic'
|
|
|
|
PASS: testComputeFstatistic_v2.sh
|
|
|
|
============================================================================
|
|
|
|
Testsuite summary for LALApps 6.25.1.1
|
|
|
|
============================================================================
|
|
|
|
# TOTAL: 1
|
|
|
|
# PASS: 1
|
|
|
|
...
|
|
|
|
|
|
|
|
## run multiple tests
|
|
|
|
|
|
|
|
You can do the same thing to run multiple tests:
|
|
|
|
|
|
|
|
$ make check TESTS='testComputeFstatistic_v2 testComputeFstatistic_v2_resamp'
|
|
|
|
...
|
|
|
|
make[3]: Entering directory '/home/karl/Software/lalsuite/lalapps/src/pulsar/Fstatistic'
|
|
|
|
PASS: testComputeFstatistic_v2.sh
|
|
|
|
PASS: testComputeFstatistic_v2_resamp.sh
|
|
|
|
============================================================================
|
|
|
|
Testsuite summary for LALApps 6.25.1.1
|
|
|
|
============================================================================
|
|
|
|
# TOTAL: 2
|
|
|
|
# PASS: 2
|
|
|
|
...
|
|
|
|
|
|
|
|
## verbose output
|
|
|
|
|
|
|
|
Useful flags include:
|
|
|
|
|
|
|
|
* `DEBUG=1 make check ...` to print more messages (to log files by default)
|
|
|
|
* `NOCLEANUP=1 make check ...` to keep the temporary output directories even for successful tests
|
|
|
|
* `make check VERBOSE=1 ...` to print the output of any failed tests These can of course all be freely combined.
|
|
|
|
|
|
|
|
Unfortunately, there does not seem to be a good way any more to show the full output of all tests (including successful ones) on the command line. You can always look at the full `testXYZ.log` files instead; these are kept in the directory you ran make check from, not in the per-test output directories.
|
|
|
|
|
|
|
|
## rerun failed tests
|
|
|
|
|
|
|
|
Another useful feature, if you just want to rerun tests that have failed:
|
|
|
|
|
|
|
|
```
|
|
|
|
# testComputeFstatistic_v2_resamp is failing due to a bug
|
|
|
|
$ make check
|
|
|
|
...
|
|
|
|
make[3]: Entering directory '/home/karl/Software/lalsuite/lalapps/src/pulsar/Fstatistic'
|
|
|
|
PASS: testPredictFstat.sh
|
|
|
|
PASS: testComputeFstatistic_v2.sh
|
|
|
|
PASS: testComputeFstatistic_v2_singleF.sh
|
|
|
|
PASS: testComputeFstatistic_v2_binary.sh
|
|
|
|
PASS: testComputeFstatistic_v2_grids.sh
|
|
|
|
FAIL: testComputeFstatistic_v2_resamp.sh
|
|
|
|
PASS: testComputeFstatistic_v2_transient.sh
|
|
|
|
PASS: test_synthesizeLVStats.sh
|
|
|
|
============================================================================
|
|
|
|
Testsuite summary for LALApps 6.25.1.1
|
|
|
|
============================================================================
|
|
|
|
# TOTAL: 8
|
|
|
|
# PASS: 7
|
|
|
|
# SKIP: 0
|
|
|
|
# XFAIL: 0
|
|
|
|
# FAIL: 1
|
|
|
|
# XPASS: 0
|
|
|
|
# ERROR: 0
|
|
|
|
============================================================================
|
|
|
|
...
|
|
|
|
# Edit testComputeFstatistic_v2_resamp to fix bug, only rerun this test
|
|
|
|
$ make check RECHECK_LOGS=
|
|
|
|
...
|
|
|
|
make[3]: Entering directory '/home/karl/Software/lalsuite/lalapps/src/pulsar/Fstatistic'
|
|
|
|
PASS: testComputeFstatistic_v2_resamp.sh
|
|
|
|
============================================================================
|
|
|
|
Testsuite summary for LALApps 6.25.1.1
|
|
|
|
============================================================================
|
|
|
|
# TOTAL: 8
|
|
|
|
# PASS: 8
|
|
|
|
# SKIP: 0
|
|
|
|
# XFAIL: 0
|
|
|
|
# FAIL: 0
|
|
|
|
# XPASS: 0
|
|
|
|
# ERROR: 0
|
|
|
|
============================================================================
|
|
|
|
...
|
|
|
|
|
|
|
|
Unlike `make check TESTS=...`, this approach `make check RECHECK_LOGS=...` does not have to be run in the same directory of the test; it can be run from anywhere in the build tree. However, it will only trigger a rebuild if the test script itself has been modified; it won't work if e.g. the bug was fixed in a LALPulsar executable. |
|
|
|
\ No newline at end of file |
