Several changes to fix line subtraction when lines turn on and off
@louis.dartez pointed out an issue on September 5 when the calibration lines were present in the NOLINES channel during a time when the calibration lines were turned off. This was caused by the fact that, due to the line subtraction gating, the subtraction signal was still being subtracted when the lines were turned off. Here is a spectrogram showing what happened:
In order to fix this, it was necessary to implement a method that distinguishes between zeros and missing data in line witness channels, so that the subtraction can continue to function during brief data dropouts, but actual zeros will cause the subtraction to stop (i.e., to subtract zeros). During these times, it is still important to gate the line subtraction TFs, so that they are not updated with incoherent data that corrupts the subtraction once the lines turn back on. Here is a spectrogram showing the improvement after the first fix:
Although this fixes the issue during the time when the lines are off, there is still a problem visible starting just after the 6-hour mark in the above plot. I discovered that this is because the line subtraction was not updating as intended due to secular changes in the TFs. (When significant secular change is detected in the line subtracion TFs, the algorithm is supposed to switch to a shorter running median so that the subtraction can adjust faster--see details in G2400875.) The reason, as I found, was that the GstController was not working in this case. The fix was to implement another way to control the timing of the line subtraction updates. The the resulting improvement in the spectrongram can be seen here:
Finally, I analyzed another stretch of data in response to this aLOG that revealed another minor issue. In this spectrogram, the problem can be seen from around 19 - 22 UTC. The problem was greatly reduced by fixing the fix to the line subtraction updating due to secular change in the TFs noted above, but there was one remaining issue: the gating was not triggered by the calibration measurements being done around that time because the lock state during calibration measurements (700) is greater than the lock state during low noise (600). So, I included an option in gstlal_compute_strain to require the lock state to be equal to a particular value rather than greater than that value. Here is a spectrogram of that time after the final fix:
