Maintenance will be performed on git.ligo.org, chat.ligo.org, containers.ligo.org, and docs.ligo.org on Tuesday 7th July 2020 starting at approximately 10am PDT and lasting for around 15 minutes. There will be a short period of downtime towards the end of the maintenance window. Please direct any comments, questions, or concerns to uwm-help@cgca.uwm.edu.

Commit e5a39d59 authored by Jameson Graef Rollins's avatar Jameson Graef Rollins

tests compare against git hash, eliminate cached .h5

This updates the test infrastructure to store a git hash of the reference
code, instead of generated .h5 files.  The test infrastructure creates a
cache of reference .h5 files from the specified git hash if it doesn't
already exist, and then compares against that cache.

Comparisons against arbitrary git hashes can be specified manually.
The cache curves are generated from a shell script so that it uses the
checkout of the python code at the reference commit.  Multiple caches based
on git references are stored, and old caches are automatically pruned to
keep just the latest five.

This gets rid of the need for git-lfs and the cached .h5 files, requiring
instead the code be checked out from git to run the tests.
parent 0c79878c
Pipeline #120456 passed with stages
in 1 minute and 38 seconds
......@@ -2,25 +2,47 @@
The pygwinc project welcomes your contributions. Our policy is that
all contributions should be peer-reviewed. To facilitate the review,
please do not commit your work to this repository yourself. Instead,
fork the repository and send a merge request.
please do not commit your work to this repository directly. Instead,
please fork the repository and create a [merge
request](https://git.ligo.org/gwinc/pygwinc/-/merge_requests/new)
against the main pygwinc master branch.
`pygwinc` comes with a test suite that will calculate all canonical
IFO budgets with the current state of the code, and compare them
against cached hdf5 traces (stored in the repo with
[git-lfs](https://git-lfs.github.com/) ( [see
also](https://wiki.ligo.org/Computing/GitLFS)). Contributors should
run the tests before submitting any merge requests:
When submitting code for merge, please follow good coding practice.
Respect the existing coding style, which for `pygwinc` is standard
[PEP8](https://www.python.org/dev/peps/pep-0008/) (with some
exceptions). Make individual commits as logically distinct and atomic
as possible, and provide a complete, descriptive log of the changes
(including a top summary line). Review efficiency follows code
legibility.
`pygwinc` comes with a validation command that can compare budgets
between different versions of the code and produce a graphical report.
`pygwinc` stores a reference to the git commit that is considered the
latest good reference for the budget calculation functions, and the
validation is run against that reference by default. The command can
be run with:
```shell
$ python3 -m gwinc.test
```
The test suite will be run automatically upon push to git.ligo.org as
part of the continuous integration, and code that fails the CI will
not be merged, unless failures are well justified.
Use the '--plot' or '--report' options to produce visual comparisons
of the noise differences. The comparison can be done against an
arbitrary commit using the '--git-ref' option. Traces for referenced
commits are cached, which speeds up subsequent comparison runs
significantly.
Commits for code changes that modify budget calculations should be
followed up updates to test reference. This can be done with the
'--update-ref' option to the test command:
```shell
$ python3 -m gwinc.test --update-ref
```
If no specific reference is provided, a pointer to the last commit
will be made. *Note: reference updates need to be made in a separate
commit after the commits that change the noise calculations* (git
commits can't have foreknowledge of their own commit hash).
If your change affects the shape of a noise curve, your commit message
should make note of that, and provide a justification. It will also
be necessary to update the reference curves stored in cache, but that
should be done in a separate commit not a part of the original merge
request, so that all reviewers can see the changes introduced in the
CI test failure report.
Once you submit your merge request, an automated pipeline will run the
test command to validate your changes. If budgets differences are
detected the pipeline will fail, alerting reviewers that a more
detailed review is required. Test issues will need to be resolved
before code can be merged.
This diff is collapsed.
#!/bin/bash -e
if [ -z "$1" ] || [ -z "$2" ] ; then
echo "usage: $(basename $0) git_ref_hash cache_dir_path"
echo "generate a cache of IFO budget traces from a particular git commit"
exit 1
fi
ref_hash="$1"
cache_dir="$2"
mkdir -p $cache_dir
cache_dir=$(cd $cache_dir && pwd)
gwinc_dir=$cache_dir/gwinc
mkdir -p $gwinc_dir
git archive $ref_hash | tar -x -C $gwinc_dir
cd $gwinc_dir
export LOG_LEVEL=INFO
for ifo in $(python3 -c "import gwinc; print(' '.join(gwinc.IFOS))") ; do
python3 -m gwinc --save $cache_dir/${ifo}.h5 $ifo
done
echo $ref_hash > $cache_dir/ref_hash
9ff4ba5463895698f48c243e329c4138ab163c1f
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment