Skip to content

lalapps_splitSFTs: fail if added record would break increasing timestamps

David Keitel requested to merge david-keitel/lalsuite:splitSFTs-check-increasing-timestamps into master

Description

lalapps_splitSFTs can automatically append new SFTs to existing merged files and updates their filename with the increased count and span. However, when looping over multiple inputs (or the program being looped externally), it assumes to be called in increasing timestamps order. If that's not the case (...e.g. if forgetting that python's os.listdir() and glob.glob() are both not sorted by default...) then it produces invalid unsorted merged SFTs. This is easily caught, as all the information is already available before updating the output file, as evidenced by the zero or negative span seen in the filename when this happens.

mkdir testSFTs
lalapps_Makefakedata_v4 --IFO=H1 --Tsft=1800 --startTime=700000000 --duration=3600 --fmin=50 --Band=1 --outSFTbname=./testSFTs/

lalapps_splitSFTs -n testSFTs -fs 50 -fb 0.1 -fe 50.1 -- testSFTs/H-1_H1_1800SFT_mfdv4-700001800-1800.sft

lalapps_splitSFTs -n testSFTs -fs 50 -fb 0.1 -fe 50.1 -- testSFTs/H-1_H1_1800SFT_mfdv4-700000000-1800.sft

ls testSFTs
H-1_H1_1800SFT_mfdv4-700000000-1800.sft  H-1_H1_1800SFT_mfdv4-700001800-1800.sft H-2_H1_1800SFT_NB_F0050Hz0_W0000Hz180-700001800-0.sft

lalapps_SFTvalidate testSFTs/*NB*
lalapps_SFTvalidate: 653212bc3338dd92e14b06037c8a3bc31a27f8da CLEAN: All modifications committed
testSFTs/H-2_H1_1800SFT_NB_F0050Hz0_W0000Hz180-700001800-0.sft is not a valid SFT. SFT GPS times not increasing between SFT blocks

After the patch:

lalapps_splitSFTs -n testSFTs -fs 50 -fb 0.1 -fe 50.1 -- testSFTs/H-1_H1_1800SFT_mfdv4-700001800-1800.sft

lalapps_splitSFTs -n testSFTs -fs 50 -fb 0.1 -fe 50.1 -- testSFTs/H-1_H1_1800SFT_mfdv4-700000000-1800.sft
XLAL Error - main (splitSFTs.c:673): New SFT timestamp 700001800 is before startTime+span=700001800+1800=700003600 from existing file testSFTs/H-1_H1_1800SFT_NB_F0050Hz0_W0000Hz180-700001800-1800.sft. Appending would yield an invalid merged SFT file.
XLAL Error - main (splitSFTs.c:673): Input domain error

API Changes and Justification

Backwards Compatible Changes

  • This change introduces no API changes
  • This change adds new API calls

Backwards Incompatible Changes

  • This change modifies an existing API
  • This change removes an existing API

Review Status

@karl-wette please take a look. Half the diff is just storing the span information in the SFT_RECORD struct so the filename doesn't need to be parsed again for the new check.

Merge request reports