Kipp Cannon authored 6 years ago and Kipp Cannon committed 6 years ago

- when collecting trigger histograms from scratch, initialize the numerator with the signal model so that the resulting ranking statistic file is closer to being in a usable state as-is, without having to mix in the output of gstlal_inspiral_create_prior_diststats
- this has been made possible by the recent change to the numerator's .__iadd__() method that causes overall histogram count to be preserved across marginalization operations
- teach gstlal_inspiral_pipe to disable numerator initialization in the create_prior_diststats jobs since it is not longer required

Kipp Cannon authored 6 years ago and Kipp Cannon committed 6 years ago

- the count is meaningless, it is only used to set the scale of the density estimation kernel, but for that reason it should not change as the result of addition operations
- in the current pipeline care is taken to never add more than a single copy of the numerator into any ranking statistic in order to control the total count
- this patch causes .__iadd__() to preserve the total count, allowing the analysis pipeline plumbing to be simplified since there is no reason to fear adding multiple copies of the numerator together

Koh Ueno authored 6 years ago and Kipp Cannon committed 6 years ago

- adjust gstreamer plugin .so filenames on gstreamer 1.14+ to work
around changes to name mangling used by the plugin scanner

updated offline feature extractor Makefile to have configurable channel list as well as updated job parameters for submitting Condor jobs efficiently

- allow parameter to be controlled from the command line.

- options that can be given multiple times and that have default values were appending the values from the command line to the defaults instead of overriding the defaults with the user-supplied values.

- enable 1 and 2 by default

- when initializing an SNR PDF cache with new entries, a math domain error
  can occur when an instrument combination is impossible.  this patch fixes
  it
- it's very odd that this code worked in the past.  Python must be changing
  its behaviour.  commit 29a7482f contains
  both the original version of this code and the data file it successfully
  generated at the time ... so ... it must have worked.  I don't know what
  else to say, but I'm not crazy.

multichannel_datasource.py: added option to force splitting of channels into subsets with an equal number of channels for serial processing

multichannel_datasource.py: added a 4s max buffer duration reblock element after demux when reading in frames to reduce memory footprint substantially

gstlal_feature_extractor: fixed issue where triggers were not being saved caused by a typo in save_format option, added message when triggers were persisted to disk on final write

gstlal_feature_extractor: removed check in DataSourceInfo causing the pipeline to not start up using live data, added an option for live data that allows features to be persisted to disk at a regular cadence, minor refactoring in MultiChannelHandler

gstlal_feature_extractor_pipe + offline Makefile: added request_disk feature to condor submissions to ensure there is enough local space to run jobs

multichannel_datasource.py: fixed bug introduced with change mkbasicmultisrc where conditional that checks if frame segments are set was flipped. This caused the pipeline to crash when not passing in segment lists

gstlal_feature_extractor_pipe: fixed bug where jobs were created for time stretches in which there was no analyzable time

gstlal_feature_extractor: changed method of local frame caching from copying over frames to generating new frames containing only channels needed for analysis, added auxcache.py to gstlal-ugly to facilitate this, and updated Makefile.am to reflect new file

lal_transferfunction:  fixed off-by-one bookkeeping bug in storing old data, and property-setting bug

gstlal_feature_extractor: changed save_hdf option to save_format for clarity, made hdf5 the default file format for saving features to disk

etg -> feature_extractor: the great migration. purged all mentions of ETG and replaced them with feature extractor or more aptly name depending on context

gstlal_etg_pipe + related Makefile: added concurrency option to limit the amount of concurrent I/O frame reads when submitting condor jobs

lal_transferfunction:  fixed Tukey window bug.  Also use comple conjugate when resampling transfer functions and going past the edges.