- accidentally modified an in-use variable

Kipp Cannon authored 6 years ago and Kipp Cannon committed 6 years ago

- when analyzing times when not all of the detectors in a network are operating, the new coincidence code jams up.  this patch adds a work-around.  this mostly affects single-detector offline analyses (it does not affect online analyses at all, because all detectors are always processed continually).

- rewrite streamthinca as a direct interface to the new streaming
coincidence engine in snglcoinc.py instead of being a wrapper around the
offline coincidence engine in thinca.py

- "if coinc_event_index" is inside "if last_coincs", which already is testing that there are coincs in the index.

- this is to simplify changes that will be required as part of the coincidence code rewrite.  want to reduce references to last_coincs

- add hacks (some of which might prove to be permanent) to filter out
  sub-threshold triggers, preventing them from contaminating the ranking
  statistic PDFs

- the point here, folks, is that we're trying to *untangle* the handler
  code.  a lot of work went into separating the eye candy related stuff
  into its own box.  the process of surgically separating the two is not
  complete, and the kafka patches are re-introducing entanglement.

- also report end-of-stream book keeping activity

- the code that needed to be detangled appears to have detangled itself.

Chad Hanna authored 6 years ago and Kipp Cannon committed 6 years ago

python/lloidhandler.py: shorten the amount of data stored in deques to make downloads and aggregation faster

- left-over bug from rewrite

- move Handler to its own module
- rename .gen_psd_xmldoc() to .__get_psd_xmldoc() following conventions used in inspiral.Data
- inspiral.py: excise gracedb code from Data object, encapsulate in GracedBWrapper class
- remove now unused write_psd_fileobj() from reference_psd.py
- move Data class to lloidhandler.py
- rename ._gatehandler() .__gatehandler() for consistency
- remove .flush(), unused locking wrapper of .__flush()
- remove .do_gracedb_alerts(), unused locking wrapper of .__do_gracedb_alerts()
- excise eye candy from Data class, move to stand-alone EyeCandy class
- merge what remains of Data and Handler into a single class
- excise segment tracking from result, move to stand-alone SegmentsTracker class
- use Handler's PSDs for GracedB uploads instead of getting new ones
  (addresses a FIXME)

- can rely on gracedb module import to work

- template bank files are .xml.gz files

infrastructure

- disable ln L assignment online until ranking statistic is sufficiently-well measured.  this is just a first hack to be better than nothing for the KAGRA testing being done.  it will surely be replaced by something else, later.

- for some reason inspiral.py has it's own, unmaintained, versions of command line parsing stuff that's in datasource.py.  I don't know what's going on, but the way it is doesn't work with KAGRA and I think this patch fixes it

- initial attempt to automatically disable FAP/FAR assignment until enough statistics have been collected to make it possible

- fixes "... has no attribute .coincs_document" crashes
- addresses some lingering FIXMEs and removes some cruft

- don't need to convert URLs to local filenames to use, that's what that function is for

- dump is a URL is set, not necessarily just because the data is there to dump

- .T050017_filename() moved homes and I forgot to update where the instruments come from