... | ... | @@ -2,125 +2,7 @@ This wiki page is intended to host the review materials and various misc. studie |
|
|
|
|
|
* [O3 review](O3-review)
|
|
|
|
|
|
## Review for O3
|
|
|
|
|
|
Here we record the most up-to-date review results for bilby, intended for O3. Briefly, we test the sampler on [fiducial BBH](#fiducial-bbh), [fiducial BNS](#fiducial-bns), [PP tests](#pp-tests), and [compare with LALInference on known events](#known-events-comparison). We define a family of [prior-files](#prior-files) which span the range of expected chirp masses; these prior files form the basis of different test cases and allow easy setup for the online-PE.
|
|
|
|
|
|
### Versions
|
|
|
* bilby: **TODO**
|
|
|
* bilby-pipe: **TODO**
|
|
|
* sampler: dynesty [currently using this branch](https://github.com/joshspeagle/dynesty/pull/129)
|
|
|
|
|
|
### Prior files
|
|
|
For testing bilby, we have selected a set of priors based on the [IMRPhenomPV2 ROQ basis](https://git.ligo.org/lscsoft/ROQ_data/):
|
|
|
|
|
|
* [high_mass ](https://git.ligo.org/lscsoft/bilby_pipe/blob/master/bilby_pipe/data_files/high_mass.prior) defaults to 4s duration
|
|
|
* [4s](https://git.ligo.org/lscsoft/bilby_pipe/blob/master/bilby_pipe/data_files/4s.prior)
|
|
|
* [8s](https://git.ligo.org/lscsoft/bilby_pipe/blob/master/bilby_pipe/data_files/8s.prior)
|
|
|
* [16s](https://git.ligo.org/lscsoft/bilby_pipe/blob/master/bilby_pipe/data_files/16s.prior)
|
|
|
* [32s](https://git.ligo.org/lscsoft/bilby_pipe/blob/master/bilby_pipe/data_files/32s.prior)
|
|
|
* [64s](https://git.ligo.org/lscsoft/bilby_pipe/blob/master/bilby_pipe/data_files/64s.prior)
|
|
|
* [128s](https://git.ligo.org/lscsoft/bilby_pipe/blob/master/bilby_pipe/data_files/128s.prior)
|
|
|
|
|
|
Drawing 200 samples from each prior, injecting into aLIGO sensitivity data and calculating the distribution of single-detector optimal SNRs:
|
|
|
|
|
|
![SNR_distributions](uploads/f267fade76c36d45537e5bae4731cbd0/SNR_distributions.png)
|
|
|
|
|
|
### Review script
|
|
|
|
|
|
To run all tests in the review, we have a [review script](https://git.ligo.org/lscsoft/bilby_pipe/blob/master/bilby_pipe/review.py). This is accessible via the command line:
|
|
|
```console
|
|
|
(bilby) 16:29 ~: $ bilby_pipe_review --help
|
|
|
16:29 bilby_pipe INFO : Running bilby_pipe version: 0.1.0: (UNCLEAN) 60dd4a6 2019-05-27 06:20:46 -0500
|
|
|
16:29 bilby INFO : Running bilby version: 0.5.0: (CLEAN) b3077544 2019-05-26 20:52:37 -0700
|
|
|
usage:
|
|
|
|
|
|
optional arguments:
|
|
|
-h, --help show this help message and exit
|
|
|
--submit Submit the job
|
|
|
--bbh Fiducial BBH test
|
|
|
--bns Fiducial BNS test
|
|
|
--roq Use ROQ likelihood
|
|
|
--pp-test PP test test
|
|
|
--prior {128s,16s,32s,4s,64s,8s,high_mass}
|
|
|
The default prior to use
|
|
|
--duration {4,8,16,32,64,128}
|
|
|
Signal duration
|
|
|
```
|
|
|
|
|
|
### Fiducial BBH
|
|
|
|
|
|
The injection values of fiducial BBH events can be found [here](https://git.ligo.org/lscsoft/bilby_pipe/blob/master/bilby_pipe/review.py#L17)
|
|
|
|
|
|
* [`bilby_pipe_review --bbh --prior high_mass`](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_tests/review/outdir_fiducial_bbh_high_mass/result/)
|
|
|
* [`bilby_pipe_review --bbh --prior 4s --roq`](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_tests/review/outdir_fiducial_bbh_4s_ROQ/)
|
|
|
* [`bilby_pipe_review --bbh --prior 128s --roq`](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_tests/review/outdir_fiducial_bbh_128s_ROQ/result/)
|
|
|
|
|
|
**TODO** decide how many of these we want
|
|
|
|
|
|
### Fiducial BNS
|
|
|
|
|
|
**TODO** add a fiducial BNS to the review script, should it just be IMRPhenomPV2 with tides?
|
|
|
|
|
|
### PP tests
|
|
|
|
|
|
We run PP tests using 100 segments drawn from the choice of priors, with/without the ROQ. Each bullet takes you to the PP test itself, followed by the print out of the individual p-values for each parameter.
|
|
|
|
|
|
* [`bilby_pipe_review --pp-test --prior 4s`](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_tests/review/outdir_pp_test_4s/dynesty_pp.png)
|
|
|
```
|
|
|
09:47 bilby INFO : a_1: 0.13422174803889164
|
|
|
09:47 bilby INFO : a_2: 0.3062470088898569
|
|
|
09:47 bilby INFO : chirp_mass: 0.407757456907951
|
|
|
09:47 bilby INFO : dec: 0.6532747415398629
|
|
|
09:47 bilby INFO : geocent_time: 0.042109294211107644
|
|
|
09:47 bilby INFO : luminosity_distance: 0.20298287569687393
|
|
|
09:47 bilby INFO : mass_2: 0.7116684031792058
|
|
|
09:47 bilby INFO : mass_ratio: 0.8560404839845693
|
|
|
09:47 bilby INFO : phase: 0.9983558213220685
|
|
|
09:47 bilby INFO : phi_12: 0.2250501576349725
|
|
|
09:47 bilby INFO : phi_jl: 0.7947066443973454
|
|
|
09:47 bilby INFO : psi: 0.04984965827183365
|
|
|
09:47 bilby INFO : ra: 0.4969983487941709
|
|
|
09:47 bilby INFO : theta_jn: 0.9225929393866728
|
|
|
09:47 bilby INFO : tilt_1: 0.8843490360361878
|
|
|
09:47 bilby INFO : tilt_2: 0.2590152944527398
|
|
|
09:47 bilby INFO : Combined p-value: 0.39468640327813737
|
|
|
```
|
|
|
|
|
|
* [`bilby_pipe_review --pp-test --prior high_mass`](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_tests/review/outdir_pp_test_high_mass/dynesty_pp.png)
|
|
|
```
|
|
|
|
|
|
```
|
|
|
|
|
|
* [`bilby_pipe_review --pp-test --prior 128s --roq`](https://ldas-jobs.ligo.caltech.edu/~gregory.ashton/bilby_tests/review/outdir_pp_test_128s_ROQ/dynesty_pp.png)
|
|
|
```
|
|
|
13:35 bilby INFO : a_1: 0.17293086257323464
|
|
|
13:35 bilby INFO : a_2: 0.5507756304564956
|
|
|
13:35 bilby INFO : chirp_mass: 0.07973494810998656
|
|
|
13:35 bilby INFO : dec: 0.4606878987905608
|
|
|
13:35 bilby INFO : geocent_time: 0.0636859419231064
|
|
|
13:35 bilby INFO : luminosity_distance: 0.3321788957823833
|
|
|
13:35 bilby INFO : mass_2: 0.09175320064298155
|
|
|
13:35 bilby INFO : mass_ratio: 0.017537311727554152
|
|
|
13:35 bilby INFO : phase: 0.2849057859042811
|
|
|
13:35 bilby INFO : phi_12: 0.15331929136204447
|
|
|
13:35 bilby INFO : phi_jl: 0.4050101300094443
|
|
|
13:35 bilby INFO : psi: 0.06708242507266793
|
|
|
13:35 bilby INFO : ra: 0.005805606437181634
|
|
|
13:35 bilby INFO : theta_jn: 0.6654418790695829
|
|
|
13:35 bilby INFO : tilt_1: 0.2000164343894656
|
|
|
13:35 bilby INFO : tilt_2: 0.1712076785534081
|
|
|
13:35 bilby INFO : Combined p-value: 0.0008182781963171137
|
|
|
```
|
|
|
|
|
|
**TODO** decide how many PP-tests we want and do them
|
|
|
**TODO** Like the 128s is failing as the setup isn't good enough. Could be remedied by either more live points/walks or running a few jobs in parallel.
|
|
|
|
|
|
### Known events comparison
|
|
|
|
|
|
All event comparisons for known events are stored in [bilby_pe_event_samples repo](https://git.ligo.org/gregory.ashton/bilby_pe_event_samples)
|
|
|
|
|
|
## Archived work
|
|
|
## Misc.
|
|
|
* [Sampler studies](sampler-studies)
|
|
|
* [Known event studies](https://git.ligo.org/lscsoft/bilby_pipe/wikis/archive/Event-studies)
|
|
|
* [Event PSDs](https://git.ligo.org/lscsoft/bilby_pipe/wikis/archive/Event-PSDs) |