Changes

Page history
Update O4 Code Review authored by Mick Wright's avatar Mick Wright
Hide whitespace changes
Inline Side-by-side
O4-Code-Review.md
View page @ 21183daf
......@@ -21,33 +21,32 @@ Note the failure of convergence of the nessai run. These settings have proved to
| Run Type | Run Directory | Corner Plot | Sign-off (Harsh) | Sign-off (Suvodip) |
|----------|---------------|-------------|------------------|--------------------|
| Microlensed GW150914-like as Microlensing | `CIT:/home/mick.wright/gravelamps-o4-review/micro-as-micro` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/micro-as-micro-corner.png) | | |
| Microlensed GW150914-like as Millilensing | `CIT:/home/mick.wright/gravelamps-o4-review/micro-as-milli` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/micro-as-milli-corner.png) | | |
| Microlensed GW150914-like as Unlensed | `CIT:/home/mick.wright/gravelamps-o4-review/micro-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/micro-as-unlensed-corner.png) | | |
| Millilensed GW150914-like as Microlensing | `CIT:/home/mick.wright/gravelamps-o4-review/milli-as-micro` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/milli-as-micro-corner.png) | | |
| Millilensed GW150914-like as Millilensing | `CIT:/home/mick.wright/gravelamps-o4-review/milli-as-milli` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/milli-as-milli-corner.png) | | |
| Millilensed GW150914-like as Unlensed | `CIT:/home/mick.wright/gravelamps-o4-review/milli-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/milli-as-unlensed-corner.png) | | |
| Microlensed GW150914-like as Microlensing | `CIT:/home/mick.wright/gravelamps-o4-review/micro-as-micro` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/micro-as-micro-corner.png) | | |
| Microlensed GW150914-like as Millilensing | `CIT:/home/mick.wright/gravelamps-o4-review/micro-as-milli` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/micro-as-milli-corner.png) | | |
| Microlensed GW150914-like as Unlensed | `CIT:/home/mick.wright/gravelamps-o4-review/micro-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/micro-as-unlensed-corner.png) | | |
| Millilensed GW150914-like as Microlensing | `CIT:/home/mick.wright/gravelamps-o4-review/milli-as-micro` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/milli-as-micro-corner.png) | | |
| Millilensed GW150914-like as Millilensing | `CIT:/home/mick.wright/gravelamps-o4-review/milli-as-milli` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/milli-as-milli-corner.png) | | |
| Millilensed GW150914-like as Unlensed | `CIT:/home/mick.wright/gravelamps-o4-review/milli-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/milli-as-unlensed-corner.png) | | |
#### Bayes Factors
#### Bayes Factors
| Run Type | Bayes Factor (Other Lensing) | Bayes Factor (Non-Lensing) |
| -------- | ---------------------------- | -------------------------- |
|----------|------------------------------|----------------------------|
| Microlensing | 3707.914 | 3707.9814 |
| Millilensing | -201.403 | 8.794 |
| Millilensing | -201.403 | 8.794 |
Runs indicate error in the millilensing setup. Microlensed runs behaving as expected so far.
Relatively minor difference between the millilensing and non-lensing as compared to the microlensing cases is curious.
Runs indicate error in the millilensing setup. Microlensed runs behaving as expected so far. Relatively minor difference between the millilensing and non-lensing as compared to the microlensing cases is curious.
#### Individual Chain Plots for Microlensing Test Run
| Chain Title | Corner Plot |
| ----------- | ----------- |
| Chain 0 | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/chain_0.png) |
| Chain 1 | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/chain_1.png) |
| Chain 2 | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/chain_2.png) |
| Chain 3 | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/chain_3.png) |
|-------------|-------------|
| Chain 0 | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/chain_0.png) |
| Chain 1 | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/chain_1.png) |
| Chain 2 | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/chain_2.png) |
| Chain 3 | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/chain_3.png) |
It would seem that the odd plotting behaviour of the combined samples is present in Chain 2, but all chains are in agreement.
It would seem that the odd plotting behaviour of the combined samples is present in Chain 2, but all chains are in agreement.
#### Millilensing update
......@@ -56,15 +55,15 @@ After some bugs were identified, a rerun of the millilensing run has yielded the
- Run Directory: `UWM:/home/mick.wright/gravelamps-o4-review/test-set-rerun/milli-as-milli-update`
- Status: Complete
- Corners:
- [Continuous Parameters](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/milli-update.png)
- [Signal Numbers](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/milli-discrete.png)
- [Signal 0 Phase](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/milli-discrete-phase-0.png)
- [Signal 1 Phase](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/milli-discrete-phase-1.png)
- [Continuous Parameters](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/milli-update.png)
- [Signal Numbers](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/milli-discrete.png)
- [Signal 0 Phase](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/milli-discrete-phase-0.png)
- [Signal 1 Phase](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/milli-discrete-phase-1.png)
- Parameters are largely as expected, signal 1 phase may bear some further investigation since it does not seem to yield confident constraint.
### Candidate O4 Sampling Review Tests
These runs use the configuration from the microlensing true run with the candidate sampler settings from the O4 Sampler Review.
These runs use the configuration from the microlensing true run with the candidate sampler settings from the O4 Sampler Review.
- Candidate O4 Bilby Version: 2.1.0
- Candidate O4 Bilby Pipe Verison: 1.0.10
......@@ -73,19 +72,19 @@ These runs use the configuration from the microlensing true run with the candida
- Run Directory: `UWM:/home/mick.wright/gravelamps-o4-review/sampler-settings/dynesty-review-candidate`
- Status: Complete
- Corner: [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/dynesty-candidate.png)
- Corner: [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/dynesty-candidate.png)
- Parameters recovered as expected
Sign offs:
- [ ] Harsh
- [ ] Suvodip
* [ ] Harsh
* [ ] Suvodip
#### Bilby MCMC
- Run Directory: `UWM:/home/mick.wright/gravlemaps-o4-review/sampler-settings/bilby-mcmc-review-candidate`
- Status: Complete
- Corner: [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/bilby-mcmc-candidate.png)
- Corner: [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/bilby-mcmc-candidate.png)
- Run did not converge, chains are very inconsistent
#### Nessai
......@@ -96,10 +95,10 @@ Sign offs:
## Lens Generation
| Lens Model | Run Status | Run Directory | Amplification Plot | Sign Off (Harsh) | Sign Off (Suvodip) |
| ---------- | ---------- | ------------- | ------------------ | ---------------- | ------------------ |
| Point | Complete | UWM:`/home/mick.wright/gravelmaps-o4-review/generation-runs/point-generation` | [Plot](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/point-amplification.png) | | |
| SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/generation-runs/light-test/sis-generation` | [Plot](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/sis-amplification.png) | | |
| NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/generation-runs/light-test/nfw-generation-lighter` | [Plot](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/nfw-amplification.png) | | |
|------------|------------|---------------|--------------------|------------------|--------------------|
| Point | Complete | UWM:`/home/mick.wright/gravelmaps-o4-review/generation-runs/point-generation` | [Plot](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/point-amplification.png) | | |
| SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/generation-runs/light-test/sis-generation` | [Plot](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/sis-amplification.png) | | |
| NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/generation-runs/light-test/nfw-generation-lighter` | [Plot](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/nfw-amplification.png) | | |
NFW feature in y = 0.01 line supported by higher accuracy run
......@@ -108,29 +107,30 @@ NFW feature in y = 0.01 line supported by higher accuracy run
All of these are being performed with the O4 candidate bilby, bilby_pipe, and dynesty and using the above settings for generation/the result files from these for SIS and NFW (to avoid duplication of lengthy work)
| Run | Status | Directory | Corner | Sign Off (Harsh) | Sign Off (Suvodip) |
| --- | ------ | --------- | ------ | ---------------- | ------------------ |
| Point as Point | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/point-as-point` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/point-as-point.pdf) | | |
| Point as SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/point-as-sis` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/point-as-sis.pdf) | | |
| Point as NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/point-as-nfw` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/point-as-nfw.pdf) | | |
| Point as Unlensed | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/point-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/point-as-unlensed.pdf) | | |
| SIS as Point | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/sis-as-point` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/sis-as-point.pdf) | | |
| SIS as SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/sis-as-sis` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/sis-as-sis.pdf) | | |
| SIS as NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comaprison/sis-as-nfw` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/sis-as-nfw.pdf) | | |
| SIS as Unlensed | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/sis-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/sis-as-unlensed.pdf) | | |
| NFW as Point | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/nfw-as-point` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/nfw-as-point.pdf) | | |
| NFW as SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/nfw-as-sis` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/nfw-as-sis.pdf) | | |
| NFW as NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/nfw-as-nfw` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/nfw-as-nfw.pdf) | | |
| NFW as Unlensed | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/nfw-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/nfw-as-unlensed.pdf) | | |
| Unlensed as Point | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/unlensed-as-point` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/unlensed-as-point.pdf) | | |
| Unlensed as SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/unlensed-as-sis` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/unlensed-as-sis.pdf) | | |
| Unlensed as NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/unlensed-as-nfw` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/unlensed-as-nfw.pdf) | | |
| Unlensed as Unlensed | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/unlensed-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/unlensed-as-unlensed.pdf) | | |
|-----|--------|-----------|--------|------------------|--------------------|
| Point as Point | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/point-as-point` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/point-as-point.pdf) | | |
| Point as SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/point-as-sis` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/point-as-sis.pdf) | | |
| Point as NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/point-as-nfw` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/point-as-nfw.pdf) | | |
| Point as Unlensed | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/point-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/point-as-unlensed.pdf) | | |
| SIS as Point | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/sis-as-point` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/sis-as-point.pdf) | | |
| SIS as SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/sis-as-sis` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/sis-as-sis.pdf) | | |
| SIS as NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comaprison/sis-as-nfw` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/sis-as-nfw.pdf) | | |
| SIS as Unlensed | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/sis-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/sis-as-unlensed.pdf) | | |
| NFW as Point | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/nfw-as-point` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/nfw-as-point.pdf) | | |
| NFW as SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/nfw-as-sis` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/nfw-as-sis.pdf) | | |
| NFW as NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/nfw-as-nfw` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/nfw-as-nfw.pdf) | | |
| NFW as Unlensed | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/nfw-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/nfw-as-unlensed.pdf) | | |
| Unlensed as Point | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/unlensed-as-point` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/unlensed-as-point.pdf) | | |
| Unlensed as SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/unlensed-as-sis` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/unlensed-as-sis.pdf) | | |
| Unlensed as NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/unlensed-as-nfw` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/unlensed-as-nfw.pdf) | | |
| Unlensed as Unlensed | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/unlensed-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/unlensed-as-unlensed.pdf) | | |
### Log Bayes Factor Table
| Model | B^{Model}_{Point} | B^{Model}_{SIS} | B^{Model}_{NFW} | B^{Model}_{Unlensed} |
| ----- | ----------------- | --------------- | --------------- | -------------------- |
| Model | B^{Model}\_{Point} | B^{Model}\_{SIS} | B^{Model}\_{NFW} | B^{Model}\_{Unlensed} |
|-------|--------------------|------------------|------------------|-----------------------|
| Point | N/A | 13.25 | 49.26 | 3729.28 |
| SIS | 49.93 | N/A | 476.14 | 4016.60 |
| NFW | 37.01 | 27.00 | N/A | 113.80 |
| Unlensed | 6.06 | -0.43 | -0.82 | N/A |
| Unlensed | 6.06 | -0.43 | -0.82 | N/A |