Minor fixes for file read offsets and far sanity checks
This branch includes a few fixes to v5, merged from the tag spiir-O4-review-v5.1
which is about to start testing on MDC.
The changes are:
- New default values for detrspmap interval and offset.
- 600s for offset, since the detrspmap gps_step is every 15 minutes, it's best to aim to update within the first interval.
If 600s isn't enough time to write the detrspmap, we'll need a different investigation/fix. - 43200s for detrspmap interval. The file contains 1 day of data, so we should update twice a day to make sure we have enough data for the next update.
e.g. If the pipeline was restarted 10 hours in, we'd load the detrspmap file with only 14 hours of data remaining. If we update every 24 hours, we'd be missing up-to-date detrspmap data for 10 hours each day.
(It may be worth verifying the pipeline's behaviour in this scenario...)
- 600s for offset, since the detrspmap gps_step is every 15 minutes, it's best to aim to update within the first interval.
- Fix 'offset' for detrspmap and marginalized stats.
-
!182 (merged) added an offset between writing and reading these files.
It was accidentally applied on every interval, meaning that in effect the interval wasinterval + offset
. - We now only apply it on the first interval (by updating the 'last refresh time' to be 'current time - offset')
-
!182 (merged) added an offset between writing and reading these files.
- Restored a sanity check on FAR in finalsink
- If a trigger's far is <= 0, we'll never upload it. This can happen if livetime is 0, but in
spiir_O4-review-v5
, we also check single_far on each detector, which would also be 0, so we shouldn't have had any bad uploads.
- If a trigger's far is <= 0, we'll never upload it. This can happen if livetime is 0, but in
- Filesink now correctly writes the new livetime & nevent columns.
- Filesink is currently unused and is a candidate for deletion, but I've tried modifying our scripts to use it and it Mostly works. It seems to start writing garbage some way in, I expect that it's unrelated but can double check if desired.