Use best far
We maintain backgrounds on 3 timescales, approximately 1 week, 1 day, and 2 hour. When deciding whether or not to upload an event, we select the timescale with the worst far, as it likely has the most reliable results. It's a very conservative approach, and when, for example, detectors increase in sensitivity overnight, it can cause us to miss events.
This MR selects the best far instead, but also adds nevent and livetime requirements for each timescale to be considered.
I'm aiming to merge this with feature flags to enable/disable the feature, and to customize the nevent & livetime thresholds used to customize each timescale.
Update
@manoj.kovalam has tested using a single nevent threshold only. The idea is that the number of data points required to model the noise should be the same regardless of the length of the timescale.
I've added a feature flag description now, and updated the readme's and such.
Tests
For code testing, I've done a few test runs with the feature enabled (with & without a threshold) and disabled.
I wrote a script in spiir-group (spiir-group/spiir!42) that determines which timescale was used for each zerolag. I haven't needed it here, but it could be useful if we want to test best far further. If I run it, I'll add it as a comment.
I ran the 8000s injection test:
- Disabled
/fred/oz016/tdavies/projects/testing/inj/8000_test/best_far/4_10_disabled
- Comparing with
/fred/oz016/tdavies/projects/testing/inj/8000_test/run-inj-sing-3/
I get the exact same zerolags.
- Comparing with
- Enabled with no specified threshold.
- The default threshold of 10m should be applied. This will cause only the 1wk timescale to be valid.
- Comparing with
/fred/oz016/tdavies/projects/testing/inj/8000_test/run-inj-sing-3/
I get a mix of similar and different FARs, as the 1 week timescale is always used.
- Enabled with a 1k threshold
- Both timescales should be enabled.
- Comparing with
/fred/oz016/tdavies/projects/testing/inj/8000_test/run-inj-sing-3/
, every zerolag has a different FAR, as of the multiple timescales, it always picks best instead of worst.
Manoj has finished some science testing, though it only really needs verifying in external review or when we remove the feature flag.