- sngl_inspiral's process column is "process:process_id"

- this patch adds a translator to convert gracedb event uploads from
  integer ID format to the legacy ilwd:char ID format.  it can be reverted
  if gracedb is updated to accept integer ID format files

- gstlal-calibration is not touched

chad.hanna authored 6 years ago and Kipp Cannon committed 6 years ago

inspiral.py: under max over coinc in gracedb upload since clustering has been added post coincidence

inspiral.py: choose only the best coinc for gracedb uploads, and fix performance issue of generating an xml doc when there are no uploads to do

- move Handler to its own module
- rename .gen_psd_xmldoc() to .__get_psd_xmldoc() following conventions used in inspiral.Data
- inspiral.py: excise gracedb code from Data object, encapsulate in GracedBWrapper class
- remove now unused write_psd_fileobj() from reference_psd.py
- move Data class to lloidhandler.py
- rename ._gatehandler() .__gatehandler() for consistency
- remove .flush(), unused locking wrapper of .__flush()
- remove .do_gracedb_alerts(), unused locking wrapper of .__do_gracedb_alerts()
- excise eye candy from Data class, move to stand-alone EyeCandy class
- merge what remains of Data and Handler into a single class
- excise segment tracking from result, move to stand-alone SegmentsTracker class
- use Handler's PSDs for GracedB uploads instead of getting new ones
  (addresses a FIXME)

- can rely on gracedb module import to work

- template bank files are .xml.gz files

infrastructure

- disable ln L assignment online until ranking statistic is sufficiently-well measured.  this is just a first hack to be better than nothing for the KAGRA testing being done.  it will surely be replaced by something else, later.

- for some reason inspiral.py has it's own, unmaintained, versions of command line parsing stuff that's in datasource.py.  I don't know what's going on, but the way it is doesn't work with KAGRA and I think this patch fixes it

- initial attempt to automatically disable FAP/FAR assignment until enough statistics have been collected to make it possible

- fixes "... has no attribute .coincs_document" crashes
- addresses some lingering FIXMEs and removes some cruft

- don't need to convert URLs to local filenames to use, that's what that function is for

- dump is a URL is set, not necessarily just because the data is there to dump

- .T050017_filename() moved homes and I forgot to update where the instruments come from

- fixes a bug in the filename construction code in the snapshot path
- uses CacheEntry to parse T050017 filenames instead of doing it by hand
- fixes a typo in a comment

- --likelihood-file is renamed to --ranking-stat-output to make its function more clear
- --likelihood-file-cache is renamed to --ranking-stat-output-cache to retain the symmetry
- --reference-likelihood-url is renamed to --ranking-stat-input to make its function more clear

- to --ranking-stat-pdf to more correctly describe what this command line option does.
- also switch to using the load_url() function, instead of load_filename(), to anticipate getting this data from a non-disk location in O3.

Kipp Cannon authored 6 years ago and Kipp Cannon committed 6 years ago

- correct documentation and add missing documentation for command line options
- --help message is reorganized into conceptual groups
- rework input sanity/safety checking and associated error messages
- make --likelihood-url optional, no file will be written if not supplied, and trigger histogram collection will be skipped for performance improvements
- teak gstlal_inspiral_pipe to not set --likelihood-url options on injection jobs

- closes out two FIXMEs

Kipp Cannon authored 6 years ago and Kipp Cannon committed 6 years ago

- this is a temporary hack to stop the collection of dist stats files for injection jobs.  this should put an end to the endless troubles injection jobs give us in this regard.

Kipp Cannon authored 6 years ago and Kipp Cannon committed 6 years ago

- when determining which instruments are on for histograming noise triggers, trigger times are LIGOTimeGPS but segment boundaries are floats and don't provide enough precision to tell if triggers near the segment boundaries are in or out of the segment
- "fix" by protracting the segment boundaries 1 ms.

Kipp Cannon authored 6 years ago and Kipp Cannon committed 6 years ago

- don't put triggers into SNR, \chi^2 histograms if --likelihood-url is not set:  because there's no point, the data is doing to /dev/null.

- the fix for a quadratic scaling problem included an off-by-one bug which prevented the denominator histograms from being correctly populated, artificially boosting the statistical significance of zero-lag events.  this fixes