Skip to content

Changes

Page history
Update O4 Review authored by Avi Vajpeyi's avatar Avi Vajpeyi
Show whitespace changes
Inline Side-by-side
O4-Review.md
View page @ 881cf398
......@@ -164,7 +164,14 @@ We analysed this event twice with different `nlive` settings (2500, 3000).
| mass_2 | 0.00042 +/- 0.00009 |
![GW190425_effective_spin](uploads/5a974ba13f2263f51ba56985f150d663/GW190425_effective_spin.png)
| |
| ------ |
| |
| |
[GW190425_effective_spin]:uploads/5a974ba13f2263f51ba56985f150d663/GW190425_effective_spin.png
![GW190425_mass](uploads/d4eeb75db9d0db0ee5d64aabb24d534c/GW190425_mass.png)
......@@ -212,6 +219,288 @@ We analysed this event twice with different `nlive` settings (2500, 3000).
</details>
## Fiducial BBH Inj
<details><summary>Configs Contents</summary>
[Link to config](https://git.ligo.org/avi.vajpeyi/pbilby_o4_review/-/blob/main/injection_test/scaling_test/128_cpus_v1.ini)
```ini
nlive=2000
nact=50
sampling-seed=0
gaussian-noise=True
detectors=['H1', 'L1']
duration=4.0
generation-seed=1010
injection=True
injection-dict={'chirp_mass': 17.051544979894693, 'mass_ratio': 0.3183945489993522, 'a_1': 0.29526500202350264, 'a_2': 0.23262056301313416, 'tilt_1': 1.0264673717225983, 'tilt_2': 2.1701305583885513, 'phi_12': 5.0962562029664955, 'phi_jl': 2.518241237045709, 'luminosity_distance': 497.2983560174788, 'dec': 0.2205292600865073, 'ra': 3.952677097361719, 'theta_jn': 1.8795187965094322, 'psi': 2.6973435044499543, 'phase': 3.686990398567503, 'geocent_time':1238175800}
distance-marginalization=True
phase-marginalization=False
time-marginalization=True
jitter-time=True
trigger-time = 0
reference-frame=H1L1
time-reference=geocent
default-prior=BBHPriorDict
prior-dict={
chirp-mass: bilby.gw.prior.UniformInComponentsChirpMass(name='chirp_mass', minimum=14, maximum=20),
mass_ratio=bilby.gw.prior.UniformInComponentsMassRatio(name='mass_ratio', minimum=0.125, maximum=1),
mass_1=Constraint(name='mass_1', minimum=1.001398, maximum=1000),
mass_2=Constraint(name='mass_2', minimum=1.001398, maximum=1000),
a_1 = Uniform(name='a_1', minimum=0, maximum=0.99),
a_2 = Uniform(name='a_2', minimum=0, maximum=0.99),
tilt_1 = Sine(name='tilt_1'),
tilt_2 = Sine(name='tilt_2'),
phi_12 = Uniform(name='phi_12', minimum=0, maximum=2 * np.pi, boundary='periodic'),
phi_jl = Uniform(name='phi_jl', minimum=0, maximum=2 * np.pi, boundary='periodic'),
luminosity_distance = bilby.gw.prior.UniformSourceFrame(name='luminosity_distance', minimum=1e2, maximum=5e3, unit='Mpc'),
dec: Cosine(name='dec'), ra: Uniform(name='ra', minimum=0, maximum=2 * np.pi, boundary='periodic'),
cos-theta-jn: Uniform(minimum=-1, maximum=1, name='cos_theta_jn'),
psi: Uniform(name='psi', minimum=0, maximum=np.pi, boundary='periodic'),
phase: Uniform(name='phase', minimum=0, maximum=2 * np.pi, boundary='periodic'),
geocent_time = Uniform(minimum=-0.1, maximum=0.1, name=None, latex_label=None, unit=None, boundary=None)
}
waveform-approximant = IMRPhenomXPHM
```
</details>
<details><summary>Results</summary>
| Plots |
| ------ |
| <img src="uploads/a40d073630b115acbcd88d100c326b00/inj_masses.png" width="400"> |
| <img src="uploads/c227b1c2ee25ce25222d5b23c58329bb/inj_sky.png" width="400"> |
| <img src="uploads/89d74184c10d984b11d300bdce5b0c55/inj_spins.png" width="400"> |
| <img src="uploads/c9b5a555b919b82d552- The review is being conducted with pbilby 2.0.1, bilby 2.0.2.
- The review repo can be found [here](https://git.ligo.org/avi.vajpeyi/pbilby_o4_review)
- The configs/datafiles were copied from the [bilby O4 review](https://git.ligo.org/pe/O4/bilby_o4_review)
- The results ~~are~~ will be stored [on zenodo](https://sandbox.zenodo.org/record/1091264)
- The comparison LVK results are the XPHM-no-cosmo posteriors.
<details><summary>Summary of pBilby changes since O3</summary>
- Team of software engineers from ADACS refactored pbilby (to make it modular rather than a script) and added unit-tests
- Updated pbilby to use dynesty > 2.0 (has a bug-fix relating to passing random-seeds to MPI processes)
- Update pbilby to use bilby > 2
[Link to changelog](https://git.ligo.org/lscsoft/parallel_bilby/-/blob/master/CHANGELOG.md#v200-2023-03-17)
</details>
## Review Tasks
| Task | Assigned | Analysed? | Reviewed? |
|---------------------------------------|----------|-----------|-----------|
| [GW150914](#gw150914) | Avi | ✓ | |
| [GW190425](#gw190425) | Rory | ✓ | |
| [GW191109](#gw191109) | Rory | started | |
| [GW200208](#gw200208) | Rory | ✓ | |
| [Fiducial BBH Inj](#fiducial-bbh-inj) | Avi | ✓ | |
| [Scaling test](#scaling-test) | Avi | ✓ | |
| [4s BBH PP-test](#4s-bbh-pp-test) | Avi | ✓ | |
| 16s BBH PP-test | Avi | started | |
| 64s NSBH PP-test | Avi | started | |
| 128s BNS (ROQ) PP-test | Soichiro | contacted | |
[Energy usage statement](#energycarbon-footprint-of-review)
## GW150914
We analysed this event twice with different `nlive` settings (1500, 2500).
<details><summary>Configs Contents</summary>
[Link to config](https://git.ligo.org/avi.vajpeyi/pbilby_o4_review/-/blob/main/GW150914/config_pbilby.ini)
```ini
trigger-time=1126259462.391
n-simulation=0
channel-dict={ H1:DCS-CALIB_STRAIN_C02, L1:DCS-CALIB_STRAIN_C02, }
data_dict = {H1=../data/H1.gwf, L1=../data/L1.gwf}
detectors=['H1', 'L1']
duration=4.0
psd-dict={H1:psds/H1-psd.dat, L1:psds/L1-psd.dat}
post-trigger-duration=2.0
sampling-frequency=2048.0
psd-length=4
psd-maximum-duration=1024
maximum-frequency={ 'H1': 896, 'L1': 896, }
minimum-frequency={ 'H1': 20, 'L1': 20, }
distance-marginalization=True
phase-marginalization=False
time-marginalization=True
jitter-time=True
reference-frame=H1L1
time-reference=geocent
likelihood-type=GravitationalWaveTransient
default-prior=BBHPriorDict
deltaT=0.2
prior-dict={chirp-mass: bilby.gw.prior.UniformInComponentsChirpMass(minimum=21.418182160215295, maximum=41.97447913941358, name='chirp_mass', boundary=None), mass-ratio: bilby.gw.prior.UniformInComponentsMassRatio(minimum=0.05, maximum=1.0, name='mass_ratio', latex_label='$q$', unit=None, boundary=None), mass-1: Constraint(minimum=1, maximum=1000, name='mass_1', latex_label='$m_1$', unit=None), mass-2: Constraint(minimum=1, maximum=1000, name='mass_2', latex_label='$m_2$', unit=None), a-1: Uniform(minimum=0, maximum=0.99, name='a_1', latex_label='$a_1$', unit=None, boundary=None), a-2: Uniform(minimum=0, maximum=0.99, name='a_2', latex_label='$a_2$', unit=None, boundary=None), tilt-1: Sine(minimum=0, maximum=3.141592653589793, name='tilt_1'), tilt-2: Sine(minimum=0, maximum=3.141592653589793, name='tilt_2'), phi-12: Uniform(minimum=0, maximum=6.283185307179586, name='phi_12', boundary='periodic'), phi-jl: Uniform(minimum=0, maximum=6.283185307179586, name='phi_jl', boundary='periodic'), luminosity-distance: PowerLaw(alpha=2, minimum=10, maximum=10000, name='luminosity_distance', latex_label='$d_L$', unit='Mpc', boundary=None), theta-jn: Sine(minimum=0, maximum=3.141592653589793, name='theta_jn'), psi: Uniform(minimum=0, maximum=3.141592653589793, name='psi', boundary='periodic'), phase: Uniform(minimum=0, maximum=6.283185307179586, name='phase', boundary='periodic'), dec: Cosine(name='dec'), ra: Uniform(name='ra', minimum=0, maximum=2 * np.pi, boundary='periodic')}
enforce-signal-duration=True
calibration-model=CubicSpline
spline-calibration-nodes=10
calibration-prior-boundary=reflective
spline-calibration-envelope-dict={H1:calibration/H1.dat, L1:calibration/L1.dat}
sampler=dynesty
nlive=1500 # this was altered for the 2nd run
walks=100
nact=50
dynesty_sample='rwalk'
waveform-generator=bilby.gw.waveform_generator.WaveformGenerator
reference-frequency=20.0
waveform-approximant=IMRPhenomXPHM
catch-waveform-errors=True
frequency-domain-source-model=lal_binary_black_hole
n-parallel=4
nodes = 1
ntasks-per-node = 32
time = 48:00:00
```
</details>
<details><summary>Results</summary>
| nlive 1500 | nlive 2500 |
|------------|------------|
|![1509_a_es]|![1509_b_es]|
|![1509_a_m] |![1509_b_m] |
|![1509_a_p] |![1509_b_p] |
|![1509_a_s] |![1509_b_s] |
[1509_a_es]:uploads/98e6c0455f654e29b4e84d504b0e9c1c/1509_a_es.png
[1509_a_m]:uploads/69f2cf8ac26a0bab5701d8de5b6ab0e6/1509_a_m.png
[1509_a_p]:uploads/5836b7891fb3a2a3e712a7449896d348/1509_a_p.png
[1509_a_s]:uploads/7c303ccd3f34aa823c86d8482b8ceeb8/GW150914_nlive2500_spin.png
[1509_b_es]:uploads/d2a31f67177188a8a47be846efff5de4/GW150914_nlive2500_effective_spin.png
[1509_b_m]:uploads/bed041a3f5432e84ffb88c92bed9f35c/GW150914_nlive2500_mass.png
[1509_b_p]:uploads/aa3307c90b78cf9dff3cbd961af20f26/GW150914_nlive2500_skypos.png
[1509_b_s]:uploads/29fe5d5b5f9b783d0a6455746de1abb9/1509_a_s.png
**JS-Divergence table**
| | nlive 1500 | nlive 2500 |
|:--------------------|:--------------------|:--------------------|
| luminosity_distance | 0.00010 +/- 0.00003 | 0.00020 +/- 0.00003 |
| cos_tilt_2 | 0.00031 +/- 0.00010 | 0.00020 +/- 0.00003 |
| dec | 0.00064 +/- 0.00009 | 0.00028 +/- 0.00005 |
| mass_2 | 0.00027 +/- 0.00005 | 0.00022 +/- 0.00003 |
| a_1 | 0.00061 +/- 0.00012 | 0.00093 +/- 0.00009 |
| mass_1 | 0.00029 +/- 0.00006 | 0.00027 +/- 0.00005 |
| mass_ratio | 0.00015 +/- 0.00006 | 0.00009 +/- 0.00002 |
| a_2 | 0.00043 +/- 0.00007 | 0.00006 +/- 0.00001 |
| cos_tilt_1 | 0.00058 +/- 0.00008 | 0.00043 +/- 0.00006 |
| chi_eff | 0.00085 +/- 0.00015 | 0.00061 +/- 0.00010 |
| ra | 0.00035 +/- 0.00007 | 0.00034 +/- 0.00006 |
| chirp_mass | 0.00069 +/- 0.00011 | 0.00075 +/- 0.00006 |
Note: the JS-Divergence values are median +- std values obtained with bootstrapping ([see JS-div script](https://git.ligo.org/avi.vajpeyi/pbilby_o4_review/-/blob/main/review_utils/js_test.py#L20)).
</details>
## GW190425
We analysed this event twice with different `nlive` settings (2500, 3000).
<details><summary>Click to expand</summary>
| | JS Divergence |
|:--------------------|:--------------------|
| dec | 0.00061 +/- 0.00017 |
| a_1 | 0.00028 +/- 0.00005 |
| a_2 | 0.00155 +/- 0.00014 |
| luminosity_distance | 0.00092 +/- 0.00021 |
| chi_eff | 0.00229 +/- 0.00020 |
| mass_ratio | 0.00034 +/- 0.00011 |
| cos_tilt_1 | 0.00111 +/- 0.00009 |
| mass_1 | 0.00056 +/- 0.00013 |
| chirp_mass | 0.00217 +/- 0.00020 |
| ra | 0.00209 +/- 0.00022 |
| cos_tilt_2 | 0.00239 +/- 0.00027 |
| mass_2 | 0.00042 +/- 0.00009 |
| |
| ------ |
| ![GW190425_effective_spin] |
| ![GW190425_mass] |
| ![GW190425_skypos]|
| ![GW190425_spin]|
[GW190425_effective_spin]:uploads/5a974ba13f2263f51ba56985f150d663/GW190425_effective_spin.png
[GW190425_mass]:uploads/d4eeb75db9d0db0ee5d64aabb24d534c/GW190425_mass.png
[GW190425_skypos]:uploads/cffef926e5b2f0b4fa46504d77927d66/GW190425_skypos.png
[GW190425_spin]:uploads/2d7461ec8cebbd3e5c45dbf896d3a7ea/GW190425_spin.png
</details>
## GW191109
## GW200208
We analysed this event twice with different `nlive` settings (2500, 3000).
<details><summary>Click to expand</summary>
| | JS Divergence |
|:--------------------|:--------------------|
| a_1 | 0.00053 +/- 0.00004 |
| chirp_mass | 0.00103 +/- 0.00006 |
| luminosity_distance | 0.00033 +/- 0.00003 |
| mass_2 | 0.00095 +/- 0.00006 |
| cos_tilt_2 | 0.00006 +/- 0.00001 |
| chi_eff | 0.00159 +/- 0.00009 |
| mass_ratio | 0.00063 +/- 0.00002 |
| mass_1 | 0.00245 +/- 0.00012 |
| cos_tilt_1 | 0.00046 +/- 0.00002 |
| a_2 | 0.00004 +/- 0.00001 |
| dec | 0.00026 +/- 0.00004 |
| ra | 0.00015 +/- 0.00002 |
| |
| ------ |
| ![GW200208_effective_spin] |
| ![GW200208_mass] |
| ![GW200208_skypos]|
| ![GW200208_spin]|
</details>
[GW200208_effective_spin]:uploads/c8f36d2ca2fda235525db268db78f9ae/GW200208_effective_spin.png
[GW200208_mass]:uploads/f9b79e87a27f9564d2dbfdfe6d20c673/GW200208_mass.png
[GW200208_skypos]:uploads/9b26f90ca0993bb950b888c5c1c61c98/GW200208_skypos.png
[GW200208_spin]:uploads/89e9a4bde46becb2673d5d5e06747b3d/GW200208_spin.png
## Fiducial BBH Inj
......@@ -300,6 +589,155 @@ Ran the above [Fiducial 4s BBH Inj](#fiducial-bbh-inj) with nCPU 32, 64, 128.
| <img src="uploads/81612c80fde5285764e850f795fd0e84/scaling_1d_histograms.png" height="1000"> |
</details>
## 4s BBH PP Test
<details><summary>Configs Contents</summary>
[Link to config](https://git.ligo.org/avi.vajpeyi/pbilby_o4_review/-/blob/main/pp_tests/bbh_4s/4s_bbh_0.ini)
```ini
prior-file = 4s_bbh.prior
injection-file = injections.dat
gaussian-noise = True
maximum-frequency = 1024
minimum-frequency = 20
trigger-time = 0
injection = True
detectors = [H1, L1]
psd_dict = None
duration = 4
distance-marginalization = True
phase-marginalization = False
time-marginalization = True
jitter-time = True
reference-frame = H1L1
time-reference = geocent
waveform-approximant = IMRPhenomXPHM
catch-waveform-errors = True
enforce-signal-duration = False
nlive = 1000
nact = 50
```
Prior
```python
chirp_mass = bilby.gw.prior.UniformInComponentsChirpMass(minimum=25, maximum=50, name='chirp_mass', latex_label='$\\mathcal{M}$', unit=None, boundary=None)
mass_ratio = bilby.gw.prior.UniformInComponentsMassRatio(minimum=0.05, maximum=1, name='mass_ratio', latex_label='$q$', unit=None, boundary=None)
mass_1 = Constraint(minimum=1, maximum=1000, name=None, latex_label=None, unit=None)
mass_2 = Constraint(minimum=1, maximum=1000, name=None, latex_label=None, unit=None)
# total_mass = Constraint(minimum=150, maximum=1000)
a_1 = Uniform(minimum=0, maximum=0.99, name=None, latex_label=None, unit=None, boundary=None)
a_2 = Uniform(minimum=0, maximum=0.99, name=None, latex_label=None, unit=None, boundary=None)
tilt_1 = Sine(minimum=0, maximum=3.141592653589793, name='tilt_1', latex_label='$\\theta_1$', unit=None, boundary=None)
tilt_2 = Sine(minimum=0, maximum=3.141592653589793, name='tilt_2', latex_label='$\\theta_2$', unit=None, boundary=None)
phi_12 = Uniform(minimum=0, maximum=6.283185307179586, name='phi_12', latex_label='$\\Delta\\phi$', unit=None, boundary='periodic')
phi_jl = Uniform(minimum=0, maximum=6.283185307179586, name='phi_jl', latex_label='$\\phi_{JL}$', unit=None, boundary='periodic')
luminosity_distance = bilby.gw.prior.UniformSourceFrame(minimum=1.0, maximum=1000.0, name='luminosity_distance', latex_label='$d_L$', unit="Mpc", boundary=None)
dec = Cosine(minimum=-1.5707963267948966, maximum=1.5707963267948966, name='dec', latex_label='$\\mathrm{DEC}$', unit=None, boundary=None)
ra = Uniform(minimum=0, maximum=6.283185307179586, name='ra', latex_label='$\\mathrm{RA}$', unit=None, boundary='periodic')
theta_jn = Sine(minimum=0, maximum=3.141592653589793, name='theta_jn', latex_label='$\\theta_{JN}$', unit=None, boundary=None)
psi = Uniform(minimum=0, maximum=3.141592653589793, name='psi', latex_label='$\\psi$', unit=None, boundary='periodic')
phase = Uniform(minimum=0, maximum=6.283185307179586, name='phase', latex_label='$\\phi$', unit=None, boundary='periodic')
geocent_time = Uniform(minimum=-0.1, maximum=0.1, name=None, latex_label=None, unit=None, boundary=None)
```
</details>
<details><summary>Results</summary>
| Plot | KS-test p-value |
| ------ | ------ |
| <img src="uploads/6e6f97f853bdd518a1657b4a2edb6f88/pp.png" width="400"> | tilt_1: 0.2110473235496638<br>ra: 0.4553802594360755<br>a_1: 0.16749876930052376<br>phi_12: 0.9312673273519888<br>tilt_2: 0.6359606676409679<br>chirp_mass: 0.5084285364847729<br>phi_jl: 0.7645417984623673<br>phase: 0.3009719585091924<br>geocent_time: 0.5782924423635027<br>theta_jn: 0.3038257852818535<br>mass_ratio: 0.8624046637946167<br>psi: 0.8546023833772283<br>dec: 0.957635050095067<br>a_2: 0.4764282474559757<br>luminosity_distance: 0.5814225969546702<br>Combined p-value: 0.9074045216633773 |
</details>
## 128s BBH PP Test
<details><summary>Results</summary>
| header | header |
| ------ | ------ |
| <img src="uploads/8e3002cde9bc5ff0a370500928d78361/bns_pp.png" width="400"> | phase: 0.7744099724749578<br>tilt_1: 0.7182219359634205<br>luminosity_distance: 0.9507364884241358<br>dec: 0.41521188748901317<br>theta_jn: 0.10384747690124049<br>lambda_1: 0.9262111416109838<br>a_1: 0.38581424471865616<br>phi_12: 0.5786998901358622<br>mass_ratio: 0.3096047054469295<br>lambda_2: 0.7678925193667797<br>phi_jl: 0.13388897336031386<br>chirp_mass: 0.08629093787462656<br>ra: 0.19870218727649275<br>a_2: 0.6730985889626475<br>geocent_time: 0.357524434774315<br>tilt_2: 0.3650129484179962<br>psi: 0.10390012639717033<br>Combined p-value: 0.4142167739142971 |
</details>
## Energy/carbon footprint of review
| Job | Walltime [hr] | CPU Runtime [hr] | kg CO2^ |
|----------------------|---------------|------------------|---------|
| GW150914 (run a) | 3.7 | 908.7 | 245.4 |
| GW150914 (run b) | 6 | 1635.1 | 441.5 |
| GW190425 | | | |
| GW191109 | | | |
| GW200208 | | | |
| Fiducial BBH Inj | 4.7 | 1223.7 | 330.4 |
| Scaling Tests (BBH) | 16.1 | 3321.2 | 896.7 |
| Scaling Tests (BNS) | | | |
| 4s BBH PP | 22.6 | 9564.3 | 2582.3 |
| 16s BBH PP | | | |
| 64s NSBH PP | | | |
| 128s BNS PP | | | |
| Failed/Test runs | | 12973.4 | 3502.8 |
**Total CO2: ~8 tCO2**
For the capture of 1 tonne of CO2 emissions ~50 trees must grow for one year [[1]]
^using 0.905 kg of CO2 per kilowatt hour [[2]], and 0.3kWh/hr per CPU.
Failed/Test runs include 1 GW150914 run, 2 GW170817 runs (w + w/o ROQ), 1 biased BBH pp test.
[1]: https://www.climateneutralgroup.com/en/news/what-exactly-is-1-tonne-of-co2/
[2]: http://www.environment.gov.au/climate-change/climate-science-data/greenhouse-gas-measurement/publications/national-greenhouse-accounts-factors-july-2018
1414886be9c0b/inj_spins2.png" width="400"> |
[inj_masses]:uploads/a40d073630b115acbcd88d100c326b00/inj_masses.png
[inj_sky]:uploads/c227b1c2ee25ce25222d5b23c58329bb/inj_sky.png
[inj_spins]:uploads/89d74184c10d984b11d300bdce5b0c55/inj_spins.png
[inj_spins2]:uploads/c9b5a555b919b82d5521414886be9c0b/inj_spins2.png
</details>
## Scaling test
Ran the above [Fiducial 4s BBH Inj](#fiducial-bbh-inj) with nCPU 32, 64, 128.
<details><summary>Results</summary>
| ncpu | time[s] | nsamples | log_evidence | log_evidence_err |
|-------:|--------:|-----------:|---------------:|-------------------:|
| 32 | 57307.8 | 14587 | -16332.8 | 0.108911 |
| 32 | 58007 | 14534 | -16332.8 | 0.108911 |
| 64 | 30241.6 | 15305 | -16332.8 | 0.109552 |
| 64 | 30356.7 | 15144 | -16332.8 | 0.109552 |
| 128 | 17208.8 | 15196 | -16332.7 | 0.109678 |
| 128 | 17072.3 | 15278 | -16332.7 | 0.109678 |
| Plots |
| ------ |
| <img src="uploads/c7e58df9f02943cc674e51337e018c61/scaling_metadata.png" height="500"> |
| <img src="uploads/81612c80fde5285764e850f795fd0e84/scaling_1d_histograms.png" height="1000"> |
......