Convert stats removal scripts to module
I've tried to extract the code from the removal scripts to common src directories.
There's a few complex changes to be checked:
- Some repeated code between the scripts and sealgw.py were merged in this function:
load_coinc_from_gid
- The code first tries to load a coinc from a provided 'cache' directory, and if that fails, downloads from gracedb instead.
- I tried to maintain as much of the behaviour from the sealgw consumer as possible, such as timing the load/download.
-
load_coinc_xml
now accepts a list of which keys it would like to load, just to reduce compute when only one feature is needed.- I think a jupyter notebook was changed as a result, we should rerun to make sure it works.
- All the functions which
load_ligolw_xmldoc
performed to be able to read Py2 files is now a standalone function,convert_legacy_xmldoc
- I changed some of the command line args of the original script (e.g. removing
{lat_tag}
), I hope they're simplified. stats_removal.py now uses click and such. - Most of the original script is now moved to
bankstats.py
. There's few logic changes, but a lot of formatting changes, so it may be hard to verify. Main thing is to do test runs.