Changes

Page history
Update home authored by Gregory Ashton's avatar Gregory Ashton
Show whitespace changes
Inline Side-by-side
home.md
View page @ fc52f7be
# bilby_pipe documentation # bilby_pipe documentation
This is some rough documentation for `bilby_pipe`. The module is in early stages of development, so expect these instructions to become outdated. We will do our best to keep them in order. This documentation [has moved](https://lscsoft.docs.ligo.org/bilby_pipe/index.html)
\ No newline at end of file
## Workflow for GraceDB events
Given a gracedb event ID (for example `G184098`, the GW150914 event UID), we first write an ini file like this
```file
label = GW150914
include-detectors = [H1, L1]
coherence-test = True
duration = 4
outdir=bilby_output
sampler = dynesty
sampler-kwargs = {'nlive': 2000}
channel_names = [H1:DCS-CALIB_STRAIN_C02, L1:DCS-CALIB_STRAIN_C02]
prior_file = GW150914.prior
executable = bbh_from_gracedb.py
accounting = ligo.dev.o3.cbc.pe.lalinference
```
If that file was named `GW150914.ini`, then we can run ` bilby_pipe` using
```bash
$ bilby_pipe GW150914.ini --gracedb G184098
```
This will generate a directory like this
```file
GW150914.ini
bilby_output/
GW150914_20181010_01.submit
GW150914_H1_20181010_01.submit
GW150914_H1L1_20181010_01.submit
GW150914_L1_20181010_01.submit
GW150914_combined_20181010_01.submit
summary.html
GW150915_logs/
GW150914_H1_20181010_01.error
GW150914_L1_20181010_01.error
```
(and so on). The job can then be submitted via
```bash
$ condor_submit_dag bilby_output/GW150914_20181010_01.submit
```
## Command line arguments
`bilby_pipe` takes one position argument, the ini file. It also takes a number of keyword arguments (or flags). Here is the help for `bilby_pipe`:
``` bash
$ bilby_pipe --help
09:58 bilby_pipe INFO : Running bilby_pipe version: 0.0.1: (UNCLEAN) f7cd576 2018-10-10 09:15:34 +1100
09:58 bilby INFO : Running bilby version: 0.3.1: (CLEAN) 2fe1a0d 2018-10-10 16:57:48 +0100
usage: Generate submission scripts for the job
Args that start with '--' (eg. --submit) can also be set in a config file
Config file syntax allows: key=value, flag=true, stuff=[a,b,c] (for details,
see syntax at https://goo.gl/R74nmi). If an arg is specified in more than one
place, then commandline values override config file values which override
defaults.
positional arguments:
ini The ini file
optional arguments:
-h, --help show this help message and exit
--submit If given, build and submit
--exe-help Print the help function for the executable
--include-detectors INCLUDE_DETECTORS [INCLUDE_DETECTORS ...]
The names of detectors to include {H1, L1}
--coherence-test
--label LABEL The output label
--outdir OUTDIR The output directory
--accounting ACCOUNTING
The accounting group to use
--executable EXECUTABLE
Either a path to the executable or the name of the
execuatable in the library
--executable_library EXECUTABLE_LIBRARY
The executable library
--X509 X509 If given, the path to the users X509 certificate
file.If not given, a copy of the file at the env.
variable $X509_USER_PROXY will be made in outdir and
linked in the condor jobs submission
-v, --verbose verbose
```
As described in the help, the ini file is actually a list of command line arguments. That is, any of the above arguments can be given as command line arguments, e.g. `bilby_pipe INI_FILE --label GW150914` **or** given in the ini file itself. Note, command line arguments override the ini file.
## The **ini/config** file
One thing you may notice is that the example ini file above contains a number of inputs which are not inputs to `bilby_pipe`. Indeed, the `ini` file is actually used twice. Once by `bilby_pipe` to generate the submission scripts. Then by the **executable**.
## The **executable**
The executable is a `bilby` script which defines what is to be run. This can be either one of the `bilby_pipe` library scripts (which can be found in `bilby_pipe/lib_scripts` or a path to any executable. The executable will be run with some fixed command line arguments like this:
```bash
$ ./executable.py "--ini INI_FILE --detectors DETECTORS --extra-arguments EXTRA-ARGUMENTS "
```
Here `INI_FILE` is the same file passed to `bilby_pipe`. The detectors is determined by `bilby_pipe` (for example if the coherence test is run for a 2 detector network, this will be `H1 L1` or `H1`, etc. Finally, the `EXTRA-ARGUMENTS` are any extra command line arguments passed to `bilby_pipe`; so, in the example above where we ran
```
$ bilby_pipe GW150914.ini --gracedb G184098
```
The `H1L1` submit script will effectively run
```
$ bbh_from_gracedb.py --ini GW150914.ini --gracedb G184098
```
The executable handles the extra arguments in the ini file and command line. Using the gracedb ID to find relevant frame files and generate data, setting up priors and a binary black hole likelihood.
## Getting help for the executable
Eventually, it is planned to have a comprehensive library of executable to cover standard workflows (and any additional behaviour should be covered by writing a new executable building of one of the base versions). However, you may like to know what inputs a given executable can take. To find this out, run
```
$ bilby_pipe --executable NAME --exe-help
```
*Note* this is currently broken.
## Summary pages and example output
We have built a crude summary page (but see https://git.ligo.org/Monash/bilby_pipe/issues/7). You can see an example for the above GW150914 run [here](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_pipe_example/bilby_output/summary.html) and more generally the outputs of the run live [here](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_pipe_example/bilby_output/).