Stricter Gating for the TDCFs

This MR will apply stricter gating to the TDCFs. We encountered a problem using gstlal-calibration-1.5.7 where the TDCFs were not gated well enough during sweeps, or at any time when the lines are turned off. When the lines are off, the coherence calculation that we use to gate does not respond until the entire running median is filled with corrupted TDCF values. This propagated the the line subtraction TFs, which led to significant failure during and shortly after sweeps. See this plot (around 18:30 - 19:00 UTC) for an example. Also, see kappa_C at that time for an example of what caused it. This did not affect the production pipeline for a rather peculiar reason: the un-medianed TDCFs are computed as nan's instead of very large or very small values in gstlal-calibration-1.5.4. Although this behavior is not necessarily desired, it caused the running median to reject those values and prevented the line subtraction problem seen in the above linked plot.

Two new gating methods can be (optionally) applied to the TDCF calculation with this MR. First, we can gate with the line amplitude channels (with names like {IFO}:CAL-CS_LINE_[1-3]_OSC_CLKGAIN or CAL-PCAL[X/Y]_PCALOSC[1-9]_OSC_SINGAIN). Second, we can also gate with the excitation channels themselves. Here is the raitionale for adding both of these rather than just one:

• If we encounter a situation where the pipeline cannot access an excitation channel for some time (due to, e.g., a raw data dropout), only the excitation channel itself can tell us that.
• When another type of injection is occurring (e.g., a Pcal sweep), the injection channel will not carry zeros, so the corresponding amplitude channel is needed in that case. I have verified that both methods work. Below is a plot showing kappa_C and f_cc after the fix. Note that the excursion shown in the above plot of kappa_C is removed, as expected.

The next plot shows that I can reproduce the error offline using gstlal-1.5.7:

This last plot shows that the line subtraction works as expected after the fix. I used both gating methods to make this plot.

The sweep, as well as a broadband injection, can be seen in this plot. We make no effort to subtract these.

.gitlab-ci.yml

@@ -49,6 +49,11 @@ dist:
     - ./00init.sh
     - ./configure --enable-gtk-doc $EXTRA_CONFIG_FLAGS
     - make dist
+  rules:
+    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
+      when: always
+    - if: $CI_PIPELINE_SOURCE == "push"
+      when: always
   artifacts:
     paths:
       - gstlal-calibration-*.tar.*
@@ -64,6 +69,11 @@ srpm:
   variables:
     TARBALL: "gstlal-calibration-*.tar.*"
   image: igwn/builder:el8-testing
+  rules:
+    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
+      when: always
+    - if: $CI_PIPELINE_SOURCE == "push"
+      when: always
 
 # Build rpms
 build:
@@ -77,6 +87,11 @@ build:
     RPM_BUILD_CPUS: 4
   needs:
     - srpm
+  rules:
+    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
+      when: always
+    - if: $CI_PIPELINE_SOURCE == "push"
+      when: always
 
 # Lint rpms
 rpmlint:
@@ -94,6 +109,11 @@ rpmlint:
   script:
     # lint the built rpms _and_ the installed ones (for extra checks)
     - rpmlint *.rpm "*gstlal-calibration*"
+  rules:
+    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
+      when: always
+    - if: $CI_PIPELINE_SOURCE == "push"
+      when: always
   allow_failure: true
 
 # Tests
@@ -189,6 +209,11 @@ docker:rl8:
     - pushes
   except:
     - /gstlal-([a-z]+-|)[0-9]+\.[0-9]+\.[0-9]+-v[0-9]+/
+  rules:
+    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
+      when: always
+    - if: $CI_PIPELINE_SOURCE == "push"
+      when: always
 
 docker-release:rl8:
   interruptible: true
@@ -241,3 +266,8 @@ flake8:
     # these are presented in the code-quality box in the
     # merge_request UI
     FLAKE8_OPTIONS: "--exit-zero"
+  rules:
+    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
+      when: always
+    - if: $CI_PIPELINE_SOURCE == "push"
+      when: always