Draft: Dist Stats Subtraction
This feature adds capabilities to subtract dist stats. This was done to incorporate a chunk of data from ER15 along with O4a.
Changes made: -
- A new folder called
subtracted_dist_stats
is made which holds all the dist stats before the start time deleted from the specified chunk. The DAG depends on existence of this dir - Data discovery was made faster
- New executable called
gstlal_inspiral_diststat_subtract
was added which does the dist stats subtraction -
isub
ofHorizonHistory
was changed to remove all info before specified time and not follow a key system (Might result in problems if 1 key needs to be removed) - A new flag for
find_online()
was added which handles data discovery for subtraction -
delete_triggers
flag was deprecated - Logic of
gstlal_delete_triggers
was changed - The workflow can now handle multiple online chunks
Testing Done: -
- All of O4a Edward was reranked using the dist stats subtraction workflow. It ran without any errors, although the config looks ugly and has redundancy. This should be fixed in a later commit after consultation of motivation from previous commit. The results of the subtracted run are here. These look very similar to the non-subtracted run results here.
- The effect of subtraction was compared with some test runs using O4b data to show more pronounced effect. The non-subtracted plots for a single bin for the above test are shown here. While the subtracted plots for the same bin are shown here. The effect is more pronounced when using O4b data since a smaller time frame was chosen. Those plots can be seen here - non-subtracted plots and subtracted plots.
Edited by Shomik Adhicary