Changes

Page history
Update O4 Review authored by Simon Stevenson's avatar Simon Stevenson
Show whitespace changes
Inline Side-by-side
O4-Review.md
View page @ 5638a813
...@@ -12,7 +12,7 @@ ...@@ -12,7 +12,7 @@
### Review statement ### Review statement
Parallel Bilby (pBilby) provides the infrastructure to use Bilby in a parallel computing setting. The review here follows the [Sampler Review Specification](https://docs.google.com/document/d/1kJAV_Q7gK03wNnEb3E5Juz3-Uiuh0R6yScgmzMGH3C0/edit#heading=h.3tjq4cvchjje). All results pass the review. The PP tests are unbiased, and the runs on real events are statistically identical to those produced previously for GWTC-3. For GW191109 (a prior dominated event), the differences between the recent rerun and the catalog are larger, [consistent with the main Bilby review](https://git.ligo.org/pe/O4/bilby_o4_review/-/issues/31). In addition, scaling tests and energy usage statistics are presented. Parallel Bilby (pBilby) provides the infrastructure to use Bilby in a parallel computing setting. The review here follows the [Sampler Review Specification](https://docs.google.com/document/d/1kJAV_Q7gK03wNnEb3E5Juz3-Uiuh0R6yScgmzMGH3C0/edit#heading=h.3tjq4cvchjje). All results pass the review. The PP tests are unbiased, and the runs on real events are statistically identical to those produced previously for GWTC-3. For GW191109 (a prior dominated event), the differences between the recent rerun and the catalog are larger, [consistent with the main Bilby review](https://git.ligo.org/pe/O4/bilby_o4_review/-/issues/31 "dynesty: GW191109_010717"). In addition, scaling tests and energy usage statistics are presented.
\-- Simon Stevenson 8th May 2023 \-- Simon Stevenson 8th May 2023
...@@ -24,8 +24,6 @@ Parallel Bilby (pBilby) provides the infrastructure to use Bilby in a parallel c ...@@ -24,8 +24,6 @@ Parallel Bilby (pBilby) provides the infrastructure to use Bilby in a parallel c
- The results are [on zenodo](https://zenodo.org/record/7884973) - The results are [on zenodo](https://zenodo.org/record/7884973)
- The comparison LVK results are the XPHM-no-cosmo posteriors. - The comparison LVK results are the XPHM-no-cosmo posteriors.
<details> <details>
<summary> <summary>
...@@ -42,13 +40,13 @@ Parallel Bilby (pBilby) provides the infrastructure to use Bilby in a parallel c ...@@ -42,13 +40,13 @@ Parallel Bilby (pBilby) provides the infrastructure to use Bilby in a parallel c
</details> </details>
#### pbilby 2.0.1 --> 2.0.2 #### pbilby 2.0.1 --> 2.0.2
During the review we updated pbilby to 2.0.2, in which the main changes were: During the review we updated pbilby to 2.0.2, in which the main changes were:
- documentation fixes, - documentation fixes,
- store the total-number of LnL calls in the result metadata - store the total-number of LnL calls in the result metadata
- store the result in both the `hdf5` and the `json` formats after sampling - store the result in both the `hdf5` and the `json` formats after sampling
## Review Tasks ## Review Tasks
| Task | Assigned | Analysed? | Reviewed? | | Task | Assigned | Analysed? | Reviewed? |
...@@ -59,7 +57,7 @@ During the review we updated pbilby to 2.0.2, in which the main changes were: ...@@ -59,7 +57,7 @@ During the review we updated pbilby to 2.0.2, in which the main changes were:
| [GW200208](#gw200208) | Rory | ✓ | :white_check_mark: [@simon-stevenson](#) | | [GW200208](#gw200208) | Rory | ✓ | :white_check_mark: [@simon-stevenson](#) |
| [Fiducial BBH Inj](#fiducial-bbh-inj) | Avi | ✓ | :white_check_mark: [@simon-stevenson](#) | | [Fiducial BBH Inj](#fiducial-bbh-inj) | Avi | ✓ | :white_check_mark: [@simon-stevenson](#) |
| [BBH Scaling test](#bbh-scaling-test) | Avi | ✓ | :white_check_mark: [@simon-stevenson](#) | | [BBH Scaling test](#bbh-scaling-test) | Avi | ✓ | :white_check_mark: [@simon-stevenson](#) |
| [BNS Scaling test](#bns-scaling-test) | Avi | ✓ | | | [BNS Scaling test](#bns-scaling-test) | Avi | ✓ | :white_check_mark: [@simon-stevenson](#) |
| [4s BBH PP-test](#4s-bbh-pp-test) | Avi | ✓ | :white_check_mark: [@simon-stevenson](#) | | [4s BBH PP-test](#4s-bbh-pp-test) | Avi | ✓ | :white_check_mark: [@simon-stevenson](#) |
| [16s BBH PP-test](#16s-bbh-pp-test) | Avi | ✓ | :white_check_mark: [@simon-stevenson](#) | | [16s BBH PP-test](#16s-bbh-pp-test) | Avi | ✓ | :white_check_mark: [@simon-stevenson](#) |
| [64s NSBH PP-test](#64s-bbh-pp-test) | Avi | ✓ | :white_check_mark: [@simon-stevenson](#) | | [64s NSBH PP-test](#64s-bbh-pp-test) | Avi | ✓ | :white_check_mark: [@simon-stevenson](#) |
...@@ -584,7 +582,6 @@ Ran the above [Fiducial 4s BBH Inj](#fiducial-bbh-inj) with nCPU 32, 64, 128. ...@@ -584,7 +582,6 @@ Ran the above [Fiducial 4s BBH Inj](#fiducial-bbh-inj) with nCPU 32, 64, 128.
</details> </details>
## BNS Scaling test ## BNS Scaling test
Analysed a 128s BNS injection with nCPU 8, 16, 32. Analysed a 128s BNS injection with nCPU 8, 16, 32.
...@@ -654,12 +651,11 @@ geocent_time = Uniform(minimum=-0.1, maximum=0.1, name=None, latex_label=None, u ...@@ -654,12 +651,11 @@ geocent_time = Uniform(minimum=-0.1, maximum=0.1, name=None, latex_label=None, u
</details> </details>
<details> <details>
<summary>Results</summary> <summary>Results</summary>
| ncpu | time | nsamples | log_evidence | log_evidence_err | | ncpu | time | nsamples | log_evidence | log_evidence_err |
|-------:|--------:|-----------:|---------------:|-------------------:| |------|------|----------|--------------|------------------|
| 8 | 82551.1 | 5764 | -519220 | 0.175312 | | 8 | 82551.1 | 5764 | -519220 | 0.175312 |
| 16 | 53819.2 | 5597 | -519220 | 0.174741 | | 16 | 53819.2 | 5597 | -519220 | 0.174741 |
| 32 | 43447.5 | 5693 | -519220 | 0.176116 | | 32 | 43447.5 | 5693 | -519220 | 0.176116 |
...@@ -670,8 +666,6 @@ geocent_time = Uniform(minimum=-0.1, maximum=0.1, name=None, latex_label=None, u ...@@ -670,8 +666,6 @@ geocent_time = Uniform(minimum=-0.1, maximum=0.1, name=None, latex_label=None, u
| ![](uploads/1fbb4291c8d0449acbd9a77210d87698/scaling_metadata.png){height="500"} | | ![](uploads/1fbb4291c8d0449acbd9a77210d87698/scaling_metadata.png){height="500"} |
| ![](uploads/b35c13ece365fc4bdba904007113ad67/scaling_1d_histograms.png){height="1000"} | | ![](uploads/b35c13ece365fc4bdba904007113ad67/scaling_1d_histograms.png){height="1000"} |
</details> </details>
## 4s BBH PP Test ## 4s BBH PP Test
...@@ -863,7 +857,6 @@ geocent_time = Uniform(minimum=-0.1, maximum=0.1, name=None, latex_label=None, u ...@@ -863,7 +857,6 @@ geocent_time = Uniform(minimum=-0.1, maximum=0.1, name=None, latex_label=None, u
## Convergence test: 128s BNS (ROQ) PP-test ## Convergence test: 128s BNS (ROQ) PP-test
## Energy/carbon footprint of review ## Energy/carbon footprint of review
| Job | Walltime \[hr\] | CPU Runtime \[hr\] | kg CO2^ | | Job | Walltime \[hr\] | CPU Runtime \[hr\] | kg CO2^ |
... ...
......