TODO

General
=======

* Verify that all elements support discontinuities.
* Verify that all elements support gaps.


Plugins/elements to hand over to GStreamer
==========================================

* ``lal_gate`` element
* ``lal_reblock`` element
* ``lal_adder`` element: make the stock ``adder`` synchronous, or figure out how to use it
* ``lal_firbank`` element
* ``lal_matrixmixer`` element
* ``lal_nxydump`` element


Streaming with gstreamer
============================

* Figure out how to do live streaming through web (pcdev1 or ATLAS) of a OmegaGram  movie

Immediate development tasks in no particular order
==================================================

* revert reverts 2878f98fea97ad1148f1f847b7a2d3c827227dfb 10e1a0c7881eba52387706474ca659c46ee2efc1 8b2089258581050bf9e7c5b0b5b456b2da345bde 026505ac1dfe0ce6bd67347da47e69efa1c010ce and test and make them work.
* streamthinca/thinca.  ligolw_thinca requires sngl_inspiral objects with a .get_effective_snr() method.  we are now the only users of ligolw_thinca, and we don't use effective snr, so strip all that stuff out of ligolw_thinca, then move the remaining basic access methods to the C code in pylal so that ligolw_thinca can work with the bare pylal type, *then* remove the sngl_inspiral row object duplication loop from streamthinca so that we only ever work with the original triggers from the pipeline and never (possibly incomplete) copies of them.  this will fix the "mchirp=0" bug in the gracedb uploads.
* Audit all import statements:  there's a lot of useless imports, especially in gstlal-inspiral/bin/
* Upload segment lists with events; this might mean keeping track of "old" segments in the online jobs (for the previous 4 hours to cover boundary effects)
* We have a request to provide bottle routes for the snr history of the individual detectors in addition to the coinc snr.
* Remove the registry files and get url cron jobs and provide direct access to the running jobs from the outside
 * Note this involves firewall changes at CIT
 * Note we will need to make sure and cache results for at least ~5 minutes to prevent DOS
* Incorporate bank veto
* add to the gstlal summary page
  * population statement plot
  * plots showing likelihood distributions, signal and background pdfs, etc.
  * histograms for injection accuracy
* inspiral DAGs:
  * try to get it to do something sane on 1 core?
  * calc_likelihood command lines are too long:  use caches of likelihood files
  * gstlal_inspiral --svd-bank option:
    * is there a reason the instrument needs to be identified?  is the instrument not recorded in the file, if not why not, and would it suffice if it was?
    * could the file list be provided in a cache file?
* finish turning the appropriate bits of the Data class into a pipeline handler, and get the input and output segments recorded correctly
* complete the removal of deprecated data input and conditioning code from lloidparts.py
* block diagonal matrix support to reduce number of threads
* fix known gstreamer bugs:
  * audiofirfilter has "timestamp" bugs (?),
  * funnel won't work in time-ordered mode,
* fix known gstlal bugs:
  * compute strain in ER3 revealed bugs in something, can't remember what,
    has fallen off radar.
* framecpp demuxer and muxer cannot decode/encode gaps using "DataValid" bit vector stream.  figure out how and fix.  (probably required to implement offline pipeline's data staging, maybe not)
* framecpp demuxer and muxer cannot decode/encode FrHistory objects.  figure out how and fix.  (will be required for calibration work, not required for inspiral search)
* fix service discovery to allow more than one job to be running on the same computer and remove need for URL dump files to be exchanged via filesystem.
* complete port to swig bindings (long term, divest ourselves of pylal).
* review readiness:  remove dead code, create tests for elements, create validation tests for applications.
* incorporate rate estimation code into pipeline (have student working on this)
* show Jolien how to add an element to the gstlaldebug plugin

Completed tasks

* move headers into a gstlal/ sub-directory in the source tree so the same #include's work in and out of the source tree (e.g., simplifies moving elements between packages)
* merge fake data generation programs into one
* sub-sample interpolation of triggers (done 2013-06-06)
* Putting the lvalert listening scripts into the dag
* have offline dag split output across multiple directories (done 2013-06-06)
* inspiral DAGs: (Done 2014-02-15)
  * give jobs human-readable names
  * add data pre-staging and serial processing of banks in offline DAG
  * gstlal_inspiral --svd-bank option:
    * why use commas instead of providing the option multiple times? : provided multiple times now
* switch online code to framexmitsrc (after framexmitsrc is made to not jam up when data stops flowing) (done 2014-01-15)
* separate likelihood code from burst pipelines (done 2013-08-29)
* fix the numerator in the likelihood ratio ranking statistic to account
  for inter-instrument correlations in signal parameters (my student's work
  on implementing the rate estimation code might have turned up some code
  that'll do a good brute-force job of this) (done 2013-08-29)
* create a gstlal debug plugin
* create matrix mixer unit test with non-identity mix matrix
* send a pipeline to reviewers demoing the use of the whitener with music or other streams
* explain to Florent how to implement a whitener test using a different spectrum
* gstreamer bugs:
  * small gaps crash resampler,