|
|
|
Here we record the most up-to-date review results for bilby, intended for O3. Briefly, we test the sampler on [fiducial BBH](#fiducial-bbh), [fiducial BNS](#fiducial-bns), [PP tests](#pp-tests), and [compare with LALInference on known events](#known-events-comparison). Reviews are set up around a family of [prior-files](#prior-files) which span the range of expected chirp masses; these prior files are intended to allow easy setup for the online-PE.
|
|
|
|
# O3 Bilby Review
|
|
|
|
|
|
|
|
### Versions
|
|
|
|
* `bilby`: Running bilby version: 0.5.1: (CLEAN) [5621baec](https://git.ligo.org/lscsoft/bilby/commit/5621baec1523b91516927f8a679efea0baa0f288) 2019-06-05 08:11:55 +1000
|
|
|
|
* `bilby-pipe`: Running bilby_pipe version: 0.2.0: (CLEAN) 0b5ca55 2019-06-04 18:42:23 -0700
|
|
|
|
* sampler: dynesty [currently using this branch](https://github.com/joshspeagle/dynesty/pull/129)
|
|
|
|
# Introduction
|
|
|
|
|
|
|
|
All results can be produced from the virtual environment:
|
|
|
|
```bash
|
|
|
|
$ source /home/gregory.ashton/virtualenvs/bilby-review/bin/activate
|
|
|
|
```
|
|
|
|
This is the home page for the review of the Bilby parameter estimation pipeline for O3. (Please keep clutter to a minimum here)
|
|
|
|
|
|
|
|
### Prior files
|
|
|
|
For testing bilby, we have selected a set of priors based on the [IMRPhenomPV2 ROQ basis](https://git.ligo.org/lscsoft/ROQ_data/):
|
|
|
|
# Developer Team
|
|
|
|
|
|
|
|
| Prior | Duration (s) | Notes |
|
|
|
|
| ------ | ------ | ------ |
|
|
|
|
| [high_mass ](https://git.ligo.org/lscsoft/bilby_pipe/blob/master/bilby_pipe/data_files/high_mass.prior) | 4 | |
|
|
|
|
| [4s](https://git.ligo.org/lscsoft/bilby_pipe/blob/master/bilby_pipe/data_files/4s.prior) | 4 | |
|
|
|
|
| [8s](https://git.ligo.org/lscsoft/bilby_pipe/blob/master/bilby_pipe/data_files/8s.prior) | 8 | |
|
|
|
|
| [16s](https://git.ligo.org/lscsoft/bilby_pipe/blob/master/bilby_pipe/data_files/16s.prior) | 16 | |
|
|
|
|
| [32s](https://git.ligo.org/lscsoft/bilby_pipe/blob/master/bilby_pipe/data_files/32s.prior) | 32 | |
|
|
|
|
| [64s](https://git.ligo.org/lscsoft/bilby_pipe/blob/master/bilby_pipe/data_files/64s.prior) | 65 | |
|
|
|
|
| [128s](https://git.ligo.org/lscsoft/bilby_pipe/blob/master/bilby_pipe/data_files/128s.prior) | 128 | lowSpin |
|
|
|
|
|
|
|
|
### Review script
|
|
|
|
# Review Team
|
|
|
|
|
|
|
|
To run all tests in the review, we have a [review script](https://git.ligo.org/lscsoft/bilby_pipe/blob/master/bilby_pipe/review.py). This is accessible via the command line:
|
|
|
|
```console
|
|
|
|
(bilby) 16:29 ~: $ bilby_pipe_review --help
|
|
|
|
16:29 bilby_pipe INFO : Running bilby_pipe version: 0.1.0: (UNCLEAN) 60dd4a6 2019-05-27 06:20:46 -0500
|
|
|
|
16:29 bilby INFO : Running bilby version: 0.5.0: (CLEAN) b3077544 2019-05-26 20:52:37 -0700
|
|
|
|
usage:
|
|
|
|
|
|
|
|
optional arguments:
|
|
|
|
-h, --help show this help message and exit
|
|
|
|
--submit Submit the job
|
|
|
|
--bbh Fiducial BBH test
|
|
|
|
--bns Fiducial BNS test
|
|
|
|
--roq Use ROQ likelihood
|
|
|
|
--pp-test PP test test
|
|
|
|
--prior {128s,16s,32s,4s,64s,8s,high_mass}
|
|
|
|
The default prior to use
|
|
|
|
--duration {4,8,16,32,64,128}
|
|
|
|
Signal duration
|
|
|
|
```
|
|
|
|
|
|
|
|
### Fiducial BBH
|
|
|
|
# Review
|
|
|
|
|
|
|
|
The injection values of fiducial BBH events can be found [here](https://git.ligo.org/lscsoft/bilby_pipe/blob/master/bilby_pipe/review.py#L17)
|
|
|
|
|
|
|
|
* [`bilby_pipe_review --bbh --prior high_mass --submit`](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_review/outdir_fiducial_bbh_high_mass/result/)
|
|
|
|
* [`bilby_pipe_review --bbh --prior 4s --roq --submit`](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_review/outdir_fiducial_bbh_4s_ROQ/)
|
|
|
|
* [`bilby_pipe_review --bbh --prior 128s --roq --submit`](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_review/outdir_fiducial_bbh_128s_ROQ/result/)
|
|
|
|
|
|
|
|
# Telecon Minutes
|
|
|
|
|
|
|
|
### Fiducial BNS
|
|
|
|
|
|
|
|
**TODO** add a fiducial BNS to the review script, should it just be IMRPhenomPV2 with tides?
|
|
|
|
# Final Review Statement
|
|
|
|
|
|
|
|
### PP tests
|
|
|
|
|
|
|
\ No newline at end of file |
|
|
|
We run PP tests using 100 segments drawn from the choice of priors, with/without the ROQ. Each bullet takes you to the PP test itself, followed by the print out of the individual p-values for each parameter.
|
|
|
|
|
|
|
|
| prior | command | pp-plot | SNR distribution|
|
|
|
|
| ------ | ------ | ------- | --- |
|
|
|
|
| high_mass | `bilby_pipe_review --pp-test --prior high_mass --submit` | [link](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_review/outdir_pp_test_high_mass/dynesty_pp.png) | [link](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_review/outdir_pp_test_high_mass/dynesty_optimal_SNR.png) |
|
|
|
|
| 4s | `bilby_pipe_review --pp-test --prior 4s --submit` | [link](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_review/outdir_pp_test_4s/dynesty_pp.png) | [link](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_review/outdir_pp_test_4s/dynesty_optimal_SNR.png) |
|
|
|
|
| 128s | `bilby_pipe_review --pp-test --prior 128s --submit` | [link](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_review/outdir_pp_test_128s_ROQ/dynesty_pp.png) | [link](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_review/outdir_pp_test_128s_ROQ/dynesty_optimal_SNR.png) |
|
|
|
|
|
|
|
|
|
|
|
|
### Known events comparison
|
|
|
|
|
|
|
|
All event comparisons for known events are stored in [bilby_pe_event_samples repo](https://git.ligo.org/gregory.ashton/bilby_pe_event_samples) |
