home.md

+# bilby_pipe documentation
+
+This is some rough documentation for `bilby_pipe`. The module is in early stages of development, so expect these instructions to become outdated. We will do our best to keep them in order.
+
+## Workflow for GraceDB events
+
+Given a gracedb event ID (for example `G184098`, the GW150914 event UID), we first write an ini file like this
+
+```file
+label = GW150914
+include-detectors = [H1, L1]
+coherence-test = True
+duration = 4
+outdir=bilby_output
+sampler = dynesty
+sampler-kwargs = {'nlive': 2000}
+channel_names = [H1:DCS-CALIB_STRAIN_C02, L1:DCS-CALIB_STRAIN_C02]
+prior_file = GW150914.prior
+
+executable = bbh_from_gracedb.py
+accounting = ligo.dev.o3.cbc.pe.lalinference
+```
+If that file was named `GW150914.ini`, then we can run ` bilby_pipe` using
+```bash
+$ bilby_pipe GW150914.ini --gracedb G184098
+```
+This will generate a directory like this
+```file
+GW150914.ini
+bilby_output/
+    GW150914_20181010_01.submit
+    GW150914_H1_20181010_01.submit
+    GW150914_H1L1_20181010_01.submit
+    GW150914_L1_20181010_01.submit
+    GW150914_combined_20181010_01.submit
+    summary.html
+    GW150915_logs/
+      GW150914_H1_20181010_01.error
+      GW150914_L1_20181010_01.error
+```
+(and so on). The job can then be submitted via
+```bash
+$ condor_submit_dag bilby_output/GW150914_20181010_01.submit
+```
+
+## Command line arguments
+
+`bilby_pipe` takes one position argument, the ini file. It also takes a number of keyword arguments (or flags). Here is the help for `bilby_pipe`:
+
+``` bash
+$ bilby_pipe --help
+09:58 bilby_pipe INFO    : Running bilby_pipe version: 0.0.1: (UNCLEAN) f7cd576 2018-10-10 09:15:34 +1100
+09:58 bilby INFO    : Running bilby version: 0.3.1: (CLEAN) 2fe1a0d 2018-10-10 16:57:48 +0100
+usage: Generate submission scripts for the job
+
+Args that start with '--' (eg. --submit) can also be set in a config file
+Config file syntax allows: key=value, flag=true, stuff=[a,b,c] (for details,
+see syntax at https://goo.gl/R74nmi). If an arg is specified in more than one
+place, then commandline values override config file values which override
+defaults.
+
+positional arguments:
+  ini                   The ini file
+
+optional arguments:
+  -h, --help            show this help message and exit
+  --submit              If given, build and submit
+  --exe-help            Print the help function for the executable
+  --include-detectors INCLUDE_DETECTORS [INCLUDE_DETECTORS ...]
+                        The names of detectors to include {H1, L1}
+  --coherence-test
+  --label LABEL         The output label
+  --outdir OUTDIR       The output directory
+  --accounting ACCOUNTING
+                        The accounting group to use
+  --executable EXECUTABLE
+                        Either a path to the executable or the name of the
+                        execuatable in the library
+  --executable_library EXECUTABLE_LIBRARY
+                        The executable library
+  --X509 X509           If given, the path to the users X509 certificate
+                        file.If not given, a copy of the file at the env.
+                        variable $X509_USER_PROXY will be made in outdir and
+                        linked in the condor jobs submission
+  -v, --verbose         verbose
+```
+
+As described in the help, the ini file is actually a list of command line arguments. That is, any of the above arguments can be given as command line arguments, e.g. `bilby_pipe INI_FILE --label GW150914` **or** given in the ini file itself. Note, command line arguments override the ini file.
+
+## The **ini/config** file
+
+One thing you may notice is that the example ini file above contains a number of inputs which are not inputs to `bilby_pipe`. Indeed, the `ini` file is actually used twice. Once by `bilby_pipe` to generate the submission scripts. Then by the **executable**. 
+
+## The **executable**
+
+The executable is a `bilby` script which defines what is to be run. This can be either one of the `bilby_pipe` library scripts (which can be found in `bilby_pipe/lib_scripts` or a path to any executable. The executable will be run with some fixed command line arguments like this:
+```bash
+$ ./executable.py "--ini INI_FILE --detectors DETECTORS --extra-arguments EXTRA-ARGUMENTS "
+```
+Here `INI_FILE` is the same file passed to `bilby_pipe`. The detectors is determined by `bilby_pipe` (for example if the coherence test is run for a 2 detector network, this will be `H1 L1` or `H1`, etc. Finally, the `EXTRA-ARGUMENTS` are any extra command line arguments passed to `bilby_pipe`; so, in the example above where we ran
+```
+$ bilby_pipe GW150914.ini --gracedb G184098
+```
+The `H1L1` submit script will effectively run
+```
+$ bbh_from_gracedb.py --ini GW150914.ini --gracedb G184098
+```
+
+The executable handles the extra arguments in the ini file and command line. Using the gracedb ID to find relevant frame files and generate data, setting up priors and a binary black hole likelihood.
+
+## Getting help for the executable
+Eventually, it is planned to have a comprehensive library of executable to cover standard workflows (and any additional behaviour should be covered by writing a new executable building of one of the base versions). However, you may like to know what inputs a given executable can take. To find this out, run
+
+```
+$ bilby_pipe --executable NAME --exe-help
+```
+
+*Note* this is currently broken. 
+
+## Summary pages and example output
+
+We have built a crude summary page (but see https://git.ligo.org/Monash/bilby_pipe/issues/7). You can see an example for the above GW150914 run [here](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_pipe_example/bilby_output/summary.html) and more generally the outputs of the run live [here](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_pipe_example/bilby_output/).