SFT version 3 implementation
Description
This MR implements version 3 of the SFT specification. The 2 major changes are:
- Version 3 SFTs record the window function applied to the time series data before being Fourier transformed. A limited number of windows typically used for CW analyses are currently supported: rectangular, Hann, Tukey. Any other window is recorded as "unknown". Any window parameter (which must be in [0.0, 1.0]) is recorded as a fixed-point decimal with a resolution of 1/5000 (0.0002). Version 3 is fully backward-compatible with version 2 SFTs, which are treated as SFTs with an unknown window.
- A more prescriptive file & directory naming convention for public (i.e. published and distributed) SFTs, which records the observing run number, type of data (e.g. h(t), simulated), a production version number, channel name, and window function. SFTs with these filenames should be globally unique, facilitating replication between clusters. The narrow-band SFT naming scheme, which was informally implemented in
splitSFTs.c
, has also be incorporated into the specification.
Implementation details:
- New functions
XLALParseSFTFilenameIntoSpec()
andXLALBuildSFTFilenameFromSpec()
parse SFT filenames to/from a newSFTFilenameSpec
struct which breaks down all the possible information fields in an SFT filename. - A Python function
public_sft_directory()
is added to determine the specified directory name from a public SFT filename. This function has no LALSuite dependencies and therefore could easily be copied elsewhere to e.g. organise SFTs replicated with rucio. - The
SFTDescriptor
struct records the window type/parameter of input SFTs. - The
XLALWriteSFT2...()
functions now require window type/parameter information to write SFTs. -
lalpulsar_Makefakedata_v[45]
now only require the user to specify the window type/parameter with noise SFTs if the SFTs record the window type as "unknown". - I have cherry-picked Rodrigo's tests added to
testMakefakedata_v5.sh
to check the--SFTWindowBeta
parameter is used if/only if the window given in--SFTWindowType
requires it (cf. !2005 (closed)). -
lalpulsar_Makefakedata_v5
support writing SFTs with the public filename specification;SIM
is used in the filename to indicate simulated data. -
lalpulsar_splitSFTs
supports processing SFTs with the public filename specification. - New function
XLALRegisterSpecialCWDetector()
allows custom detectors to be registered for the[XYZ][0-9]
detector prefixes, which are implementation-defined according to the SFT specification. In particular LISA detectors are no longer hard-coded to theZ[1-9]
prefix.XLALGetSiteInfo()
now returns a const-pointer to aLALDetector
instead of allocating a new one, which therefore doesn't need to beXLALFree()
d.
Closes #336 (closed) #614 (closed)
API Changes and Justification
Backwards Compatible Changes
-
This change does not modify any class/function/struct/type definitions in a public C header file or any Python class/function definitions -
This change adds new classes/functions/structs/types to a public C header file or Python module
Backwards Incompatible Changes
-
This change modifies an existing class/function/struct/type definition in a public C header file or Python module -
This change removes an existing class/function/struct/type from a public C header file or Python module
The XLALWriteSFT2...()
functions now require window type/parameter information to write SFTs, and hence require additional arguments. I would actually prefer to break the API here, in order to force downstream packages to think about how the SFTs they are writing might have been windowed, and then supply the appropriate information. I think that's preferable to assuming a backward-compatible default (i.e. the window function is unknown) which would be uninformative at best, or incorrect at worst.
The function XLALGetSiteInfo()
now returns a const-pointer to a LALDetector
instead of allocating a new one, which save a memory allocation/free. Adapting downstream packages written in C should be straightforward; see the diff for examples. Python/Octave usage through SWIG should be unaffected.
The functions XLALOfficialSFTFilename()
, XLALGetOfficialName4SFT()
, and XLALGetOfficialName4MergedSFTs()
are removed, having been replaced by XLALParseSFTFilenameIntoSpec()
and XLALBuildSFTFilenameFromSpec()
. The arguments supplied to these functions are no longer sufficient to fully record all the possible information fields in an SFT filename, as now given by the SFTFilenameSpec
struct. As any modification to their arguments would be an API break anyway, it's simpler to just remove them.
XLALValidateSFTFile()
is renamed to XLALCheckSFTFileIsValid()
. The existing name shadows the SFTReferenceLibrary function ValidateSFTFile()
in the SWIG wrappers; both map to lalpulsar.ValidateSFTFile()
. I've found that it would be useful to access ValidateSFTFile()
directly, e.g. to recover the exact error code from the validation failure for further processing. This API change will need some care in downstream Python packages:
- Because of the name shadowing, call to
lalpulsar.ValidateSFTFile()
will still work, but will now callValidateSFTFile()
instead of the (renamed)XLALValidateSFTFile()
. - This will lead to a change in behaviour; while the (renamed)
XLALValidateSFTFile()
raised an exception of failure,ValidateSFTFile()
just returns an error code but does not raise an exception. So any Python code relying on catching an exception fromlalpulsar.ValidateSFTFile()
will need to be changed to uselalpulsar.CheckSFTFileIsValid()
-
lalpulsar.ValidateSFTFile()
will still print error messages on failure, however, so it should be clear that a SFT validation failure has occurred
Review Status
In addition to the CI builds I ran make && cd lalpulsar/ && make check
after every commit in this MR to check for any intermediate breakages.