Skip to content

Merge O4 dev, minor cleanups, and associated gstlal fork changes

Timothy Davies requested to merge tdavies__merge_O4_dev into gstreamer_python_upgrade

I did another merge from O4 dev, but spent a while investigating & fixing some test issues which turned out to be due to the gstlal fork. See commits spiir-group/gstlal@e8a1fd96 and spiir-group/gstlal@45d010c6

The actual merge is large, but everything in share I've accepted from py2 (IIRC), and the rest is small.

I pushed them straight to that branch for convenience rather than making MRs, but the changes will likely need some explaining/reviewing before use. In general, it's making the elif gw_data_source_info.data_source == DataSource.Frames: branch on line 969 of gstlal/python/datasource.py follow the same process as the elif gw_data_source_info.data_source in (DataSource.FrameXMIT.value, DataSource.LVSHM.value, DataSource.DEVSHM.value): branch.

That's the same as what was done on the py2 version of the pipeline, but the gstlal code has since changed. I'll upload the resulting graphvis images. I didn't run into the issues on previous tests since other tests used segments to check gaps instead of state channels (which were broken).

On a finished 1w bg (unseeded) run:

  • segments are equivalent to py2 (format changes only) (i.e. gap detection is identical)
  • livetimes are identical (from the 2w marginalized stats files)
  • nevent is slightly different (542524 py3 foregrounds vs 542516 in py2)
  • coincs are slightly different (image below, py2 on right. Similar FARs)
    • Py3 found an extra trigger: 1245351229.118652,8.756968e-05,1. The other differences are a result of offline trigger control race conditions.
    • There's small differences in FAR.
    • Since the extra coinc is the first on a bank, I'd guess it's due to some difference in when we start uploads... I'd say let's wait to evaluate the injection run. FAR calculation may be vulnerable to small changes in snr but settle over time.

An injection test is now chugging along.

image

(from ligolw_print -t postcoh -c end_time -c far -c cohsnr -c cmbchisq -c bankid -c tmplt_idx 002/002_zerolag_1245319218_86400.xml.gz) image image

Merge request reports