doc/source/gstlal-inspiral/overview.rst

-Overview
-========
-
-FIXME

doc/source/gstlal-ugly/code.rst

-GstLAL ugly code
-================
-
-.. toctree::
-   :maxdepth: 2
-
-   bin/bin
-   python-modules/modules

doc/source/gstlal-ugly/gstlal-ugly.rst

-GstLAL ugly project
-===================
-
-.. toctree::
-   :maxdepth: 2
-
-   overview
-   code

doc/source/gstlal-ugly/overview.rst

-Overview
-========
-
-FIXME

doc/source/gstlal/code.rst

-GstLAL Code
-===========
-
-.. toctree::
-   :maxdepth: 2
-
-   bin/bin
-   python-modules/modules

doc/source/gstlal/gstlal.rst

-GstLAL project
-==============
-
-.. toctree::
-   :maxdepth: 2
-
-   overview
-   tutorials/tutorials
-   code
-

doc/source/gstlal/overview.rst

-Overview
-========
-
-FIXME

doc/source/index.rst

-.. GstLAL documentation master file, created by
-   sphinx-quickstart on Wed Apr 25 21:10:13 2018.
-   You can adapt this file completely to your liking, but it should at least
-   contain the root `toctree` directive.
-
-The GstLAL project documentation
+GstLAL documentation
 ================================
 
 `GstLAL` provides a suite of GStreamer elements that expose gravitational-wave data analysis tools from the LALSuite library for use in GStreamer signal-processing pipelines.
 
 Examples include an element to add simulated gravitational waves to an h(t) stream, and a source element to provide the contents of .gwf frame files to a GStreamer pipeline.
 
-.. _welcome-contents:
-
-Contents
+Overview
 -------------------------
 
+The GstLAL software package is used for the following activities:
+
+- ``gstlal`` provides core Gstreamer plugins for signal processing workflows with LIGO data and core python bindings for constructing such workflows.
+
+- ``gstlal-inspiral`` provides additional signal processing plugins that are specific for LIGO / Virgo searches for compact binaries as well as a substantial amount of python code for post-processing raw signal processing results into gravitational wave candidate lists. Several publications about the methodology and workflow exist, see :ref:`publications`
+
+- ``gstlal-burst`` provides additional signal processing plugins for use in astrophysical and noise transient burst searches.
+
+- ``gstlal-ugly`` is an incubator project for gradual inclusion in the other packages.
+
+
+.. toctree::
+    :caption: Getting Started
+    :maxdepth: 2
+
+    installation
+    quickstart
+    tutorials/tutorials
+
 .. toctree::
-   :maxdepth: 2
+    :caption: User Guide
+    :maxdepth: 2
 
-   overview
-   getting-started
-   projects
-   publications
+    cbc_analysis
+    psd_estimation
+    simulated_data
+    workflow_config
+    publications
+
+.. toctree::
+    :caption: Developer Guide
+    :maxdepth: 2
+
+    local_environment
+    container_environment
+    contributing
+    contributing_docs
+
+.. toctree::
+    :caption: API Reference
+    :maxdepth: 2
+
+    executables
+    api
+
+Build/Test Results
+-------------------------
+
+Results pages for the `Offline Tutorial Test <https://git.ligo.org/lscsoft/gstlal/blob/master/gstlal-inspiral/tests/Makefile.offline_tutorial_test>`_ are generated automatically and are located here:
+
+* `gstlal_offline_tutorial test dag <gstlal_offline_tutorial/1000000000-1000002048-test_dag-run_1/>`_
+* `gstlal_offline_tutorial test dag lite <gstlal_offline_tutorial/1000000000-1000002048-test_dag-run_1_lite/>`_
 
 .. _welcome-indices:
 
@@ -31,3 +69,5 @@ Indices and tables
 * :ref:`genindex`
 * :ref:`modindex`
 * :ref:`search`
+
+

doc/source/installation.rst

+.. _installation:
+
+Installation
+=============
+
+There are various ways to get started with GstLAL:
+
+  * :ref:`Install the latest release <install-release>`. Pre-built packages are available through various mechanisms.
+  * :ref:`Use a version provided in an IGWN reference distribution <install-igwn>`. This option is available to members of the International Gravitational-Wave Obervatory Network (IGWN). 
+  * :ref:`Building the package from source <install-source>`. This is needed for users who wish to contribute to the project.
+
+.. _install-release:
+
+Installing the latest release
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Docker container (recommended)
+""""""""""""""""""""""""""""""
+
+The following should pull a GstLAL container and enter an environment with
+GstLAL and all its dependencies pre-installed:
+
+.. code:: bash
+
+    $ docker run -it --rm containers.ligo.org/lscsoft/gstlal:latest
+
+Note that you will need `Docker <https://docs.docker.com/get-docker/>`_
+installed. If that is not an option (Docker requires sudo privileges), you can
+instead use `Singularity
+<https://sylabs.io/guides/3.7/user-guide/quick_start.html>`_ in place of Docker,
+which is available on many shared computing resources such as XSEDE and the OSG:
+
+.. code:: bash
+
+    $ singularity run docker://containers.ligo.org/lscsoft/gstlal:latest
+
+Conda installation
+"""""""""""""""""""
+
+Install conda using the `miniconda <https://docs.conda.io/projects/conda/en/latest/user-guide/install/>`_ installer, then run:
+
+.. code:: bash
+
+    $ conda install -c conda-forge gstlal-inspiral
+
+In order to check your installation, you can use:
+
+.. code:: bash
+
+    $ conda list gstlal-inspiral  # to check which version is installed
+    $ gstlal_play --help
+
+.. warning::
+   These packages don't make use of any :ref:`math optimizations <install-math_optimizations>`
+   and is not currently recommended for production or larger-scale analyses.
+
+.. _install-igwn:
+
+IGWN distributions of GstLAL
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If you are an IGWN member and have access to shared computing resources,
+up-to-date GstLAL libraries are available in the set of reference software
+environments maintained by the IGWN Computing and Software Working Group.
+
+LIGO Data Grid (LDG)
+"""""""""""""""""""""
+
+GstLAL packages are installed and available by default on the LDG. You can start
+using the GstLAL library immediately:
+
+.. code:: bash
+
+    $ gstlal_play --help
+
+IGWN Conda Distribution
+""""""""""""""""""""""""
+
+GstLAL is also available on the IGWN Conda Distribution in a variety of
+pre-packaged environments. For more information, see
+`computing.docs.ligo.org/conda/ <https://computing.docs.ligo.org/conda/>`_.
+
+.. _install-source:
+
+Building from source
+^^^^^^^^^^^^^^^^^^^^^
+
+Building from source is required for development (bug fixes, new features,
+documentation improvements). You can check out the latest source of GstLAL from
+git:
+
+.. code:: bash
+
+    $ git clone https://git.ligo.org/lscsoft/gstlal.git
+    $ cd gstlal
+
+
+Building and installing from source follows the normal GNU build procedures involving:
+
+ 1. ./00init.sh 
+ 2. ./configure 
+ 3. make 
+ 4. make install
+
+Since GstLAL is a collection of packages, there is a required build order to
+install the packages:
+
+ 1. gstlal
+ 2. gstlal-ugly
+ 3. gstlal-burst / gstlal-inspiral  (any order)
+
+If you are building from source, you will also need to install all dependencies
+before building GstLAL, including:
+
+  * fftw
+  * gsl
+  * gstreamer
+  * lalsuite
+  * ldas-tools-framecpp
+  * numpy
+  * pygobject
+  * python-ligo-lw
+  * scipy
+
+These dependencies can be installed in various ways, including conda, your
+favorite package manager (apt/yum), or from source. We also provide containers
+that are suitable for development.
+
+Singularity container
+""""""""""""""""""""""
+
+A development container is provided with all necessary dependencies to install
+GstLAL from source. Singularity also has extra features that make it possible to
+create writable containers, making it easy to get started with development:
+
+.. code:: bash
+
+    $ singularity build --sandbox --fix-perms gstlal-dev docker://containers.ligo.org/lscsoft/gstlal:master
+
+This will pull a container from the container registry from the main branch and
+builds it in 'sandbox' mode into ``/gstlal-dev``, which allows one to invoke it
+in writable mode once it's built, and is needed to install software into the
+container. This may take a few minutes to set up compared to a normal pull.
+
+Once that's finished, you can enter the container in writable mode to install GstLAL from source:
+
+.. code:: bash
+
+    $ singularity run --writable gstlal-dev
+    $ git clone https://git.ligo.org/lscsoft/gstlal.git
+    $ cd gstlal
+
+.. note::
+
+   It's possible to run into issues when adding bind mounts to a writable
+   Singularity container depending on how Singularity is configured. This
+   may cause the following error to occur:
+
+   .. code:: bash
+
+       $ singularity run --writable gstlal-dev
+       WARNING: By using --writable, Singularity can't create /cvmfs destination automatically without overlay or underlay
+       FATAL:   container creation failed: mount /cvmfs->/cvmfs error: while mounting /cvmfs: destination /cvmfs doesn't exist in container
+
+   In this case, one way to resolve this error is to add directories in the
+   container for each of the bind mounts explicitly, e.g.,:
+
+   .. code:: bash
+
+       $ mkdir gstlal-dev/cvmfs
+       $ singularity run --writable -B /cvmfs gstlal-dev
+       Singularity>
+
+
+Now you can follow the normal GNU build procedures to build and install GstLAL.
+It is also recommended to install GstLAL into the container's ``/usr``
+directory, done at the configure step, e.g.
+
+.. code:: bash
+
+    $ ./configure --prefix /usr
+
+
+.. _install-math_optimizations:
+
+Math-optimized Libraries
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+GstLAL relies heavily on Linear Algebra and FFT routines to drive various
+signal-processing components within the library. Being able to accelerate these
+math-heavy routines which rely on BLAS with MKL considerably improves performance
+and introduces significant speedups. The containers
+that are available via Docker and Singularity are linked against MKL to take
+advantage of these optimizations. There is no additional work needed from the user.
+
+Work is ongoing to provide Conda packages to allow for similar optimizations.

doc/source/local_environment.md

+# Local Development Environment
+
+The local development workflow consists of a few key points:
+
+- Managed conda environments using [conda-flow](https://git.ligo.org/james.kennington/conda-flow)
+- Using integrated development environments (IDEs) like [PyCharm]()/[CLion]()
+- Running applications that can consume `gstlal`, like [Jupyter Notebook]()
+
+## Creating the environment
+
+Thanks to conda-flow, creating the environment is simple. Before we can use conda-flow, it must be installed. In
+whatever environment you prefer, install conda-flow from pip:
+
+```bash
+pip install conda-flow
+```
+
+Once conda-flow is installed, There are locked environment files contained within the gstlal repo under
+`gstlal/gstlal/share/conda/envs`. Using conda-flow, we can make sure the local development conda environment is built:
+
+```bash
+conda-flow activate -n gstlal-dev -c /path/to/gstlal/gstlal/share/conda/conda-flow.yml
+```
+
+## Activating the environment
+
+The `activate` command within conda-flow is done through subprocesses, and consequently will *not* affect the parent
+process, such as the shell from which conda-flow is called. This is done to prevent unintended side effects; however, it
+also means that unlike `conda activate`, `conda-flow activate` will not activate the environment inside the shell. If
+you wish to activate the environment inside the shell, run `conda activate gstlal-dev`.
+
+## Using Developer Tools
+
+To use an IDE to develop `gstlal`, you will first need to start your IDE from within the appropriate conda environment.
+For example, to launch the PyCharm IDE, run:
+
+```bash
+conda-flow activate -n gstlal-dev --run-after "open -na ""PyCharm.app""" -c /path/to/gstlal/gstlal/share/conda/conda-flow.yml
+```
+
+### Python Development
+
+Note that the python source modules are not in a typical pythonic package structure (due to use of GNU build for c code
+within gstlal). This can present problems with package indexing / imports in the IDE. The solution is to provide the IDE
+with a map of the proper import paths, which can be done via two ways:
+
+1. Build `gstlal` (or at least the python components) and add the build directory as a source directory in your IDE
+1. Create a new source directory full of symlinks to the source files with a structure that mimicks the import paths.
+   There is a utility for constructing such symlinks at `gstlal/gstlal/share/conda/dev_links.py`
+
+## Launching Applications
+
+Conda-flow is capable of running arbitrary commands after activating the environment in the subprocess which is useful
+for launching applications in a controlled way. For example, to run a jupyter notebook:
+
+```bash
+conda-flow activate -n gstlal-dev --run-after "jupyter notebook" -c /path/to/gstlal/gstlal/share/conda/conda-flow.yml
+```
+

doc/source/overview.rst

-Overview
-========
-
-FIXME
-

doc/source/projects.rst

-Projects
-========
-
-.. toctree::
-   :maxdepth: 1
-
-   gstlal/gstlal
-   gstlal-inspiral/gstlal-inspiral
-   gstlal-calibration/gstlal-calibration
-   gstlal-burst/gstlal-burst
-   gstlal-ugly/gstlal-ugly
-
-
-

doc/source/psd_estimation.rst

+.. _psd-generation:
+
+PSD Generation
+================
+
+Using this workflow configuration (``config.yml``), we can generate a
+median-averaged PSD across 100000 seconds of Hanford and Livingston data in O2:
+
+.. code:: bash
+
+  # general options
+  start: 1187000000
+  stop: 1187100000
+  ifos: H1L1
+  
+  # data discovery options
+  source:
+    frame-type:
+      H1: H1_GWOSC_O2_16KHZ_R1
+      L1: L1_GWOSC_O2_16KHZ_R1
+    channel-name:
+      H1: GWOSC-16KHZ_R1_STRAIN
+      L1: GWOSC-16KHZ_R1_STRAIN
+    sample-rate: 4096
+    frame-segments-file: segments.xml.gz
+    frame-segments-name: datasegments
+  
+  # psd options
+  psd:
+    fft-length: 8
+    sample-rate: 4096
+  
+  # condor options
+  condor:
+    accounting-group: your.accounting.group
+    profile: ldas
+
+
+This sets general options such as the start and end times and the detectors to
+analyze (H1 and L1). Data discovery options specify how to retrieve the strain
+data as well as specifying the file used for science segments (``segments.xml.gz``),
+and the sampling rate. PSD-specific options such as the FFT stride used.
+
+Finally,
+workflow-specific options (via HTCondor) which specify options like the accounting group
+and the grid profile to use. If you have not done so yet, you can install grid profiles
+locally and check which grid profiles are currently available via:
+
+.. code:: bash
+
+  $ gstlal_grid_profile install
+  $ gstlal_grid_profile list  # ldas should show up as one of the possible options
+
+
+To launch the workflow via Condor with the configuration above, run:
+
+.. code:: bash
+
+  $ gstlal_query_gwosc_segments 1187000000 1187100000 H1L1
+  $ gstlal_psd_workflow -c config.yml
+  $ condor_submit_dag psd_dag.dag

doc/source/publications.rst

+.. _publications:
+
 Publications
 ============
 

doc/source/quickstart.rst

+.. _quickstart:
+
+Quickstart
+============
+
+WRITEME

doc/source/simulated_data.rst

+.. _simulated-data:
+
+Simulated Data
+===============
+
+See :ref:`simulated-data-tutorial`.