Changes

Page history
Update O4 Code Review authored by Mick Wright's avatar Mick Wright
Hide whitespace changes
Inline Side-by-side
O4-Code-Review.md
View page @ 21183daf
...@@ -21,33 +21,32 @@ Note the failure of convergence of the nessai run. These settings have proved to ...@@ -21,33 +21,32 @@ Note the failure of convergence of the nessai run. These settings have proved to
| Run Type | Run Directory | Corner Plot | Sign-off (Harsh) | Sign-off (Suvodip) | | Run Type | Run Directory | Corner Plot | Sign-off (Harsh) | Sign-off (Suvodip) |
|----------|---------------|-------------|------------------|--------------------| |----------|---------------|-------------|------------------|--------------------|
| Microlensed GW150914-like as Microlensing | `CIT:/home/mick.wright/gravelamps-o4-review/micro-as-micro` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/micro-as-micro-corner.png) | | | | Microlensed GW150914-like as Microlensing | `CIT:/home/mick.wright/gravelamps-o4-review/micro-as-micro` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/micro-as-micro-corner.png) | | |
| Microlensed GW150914-like as Millilensing | `CIT:/home/mick.wright/gravelamps-o4-review/micro-as-milli` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/micro-as-milli-corner.png) | | | | Microlensed GW150914-like as Millilensing | `CIT:/home/mick.wright/gravelamps-o4-review/micro-as-milli` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/micro-as-milli-corner.png) | | |
| Microlensed GW150914-like as Unlensed | `CIT:/home/mick.wright/gravelamps-o4-review/micro-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/micro-as-unlensed-corner.png) | | | | Microlensed GW150914-like as Unlensed | `CIT:/home/mick.wright/gravelamps-o4-review/micro-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/micro-as-unlensed-corner.png) | | |
| Millilensed GW150914-like as Microlensing | `CIT:/home/mick.wright/gravelamps-o4-review/milli-as-micro` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/milli-as-micro-corner.png) | | | | Millilensed GW150914-like as Microlensing | `CIT:/home/mick.wright/gravelamps-o4-review/milli-as-micro` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/milli-as-micro-corner.png) | | |
| Millilensed GW150914-like as Millilensing | `CIT:/home/mick.wright/gravelamps-o4-review/milli-as-milli` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/milli-as-milli-corner.png) | | | | Millilensed GW150914-like as Millilensing | `CIT:/home/mick.wright/gravelamps-o4-review/milli-as-milli` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/milli-as-milli-corner.png) | | |
| Millilensed GW150914-like as Unlensed | `CIT:/home/mick.wright/gravelamps-o4-review/milli-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/milli-as-unlensed-corner.png) | | | | Millilensed GW150914-like as Unlensed | `CIT:/home/mick.wright/gravelamps-o4-review/milli-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/milli-as-unlensed-corner.png) | | |
#### Bayes Factors #### Bayes Factors
| Run Type | Bayes Factor (Other Lensing) | Bayes Factor (Non-Lensing) | | Run Type | Bayes Factor (Other Lensing) | Bayes Factor (Non-Lensing) |
| -------- | ---------------------------- | -------------------------- | |----------|------------------------------|----------------------------|
| Microlensing | 3707.914 | 3707.9814 | | Microlensing | 3707.914 | 3707.9814 |
| Millilensing | -201.403 | 8.794 | | Millilensing | -201.403 | 8.794 |
Runs indicate error in the millilensing setup. Microlensed runs behaving as expected so far. Runs indicate error in the millilensing setup. Microlensed runs behaving as expected so far. Relatively minor difference between the millilensing and non-lensing as compared to the microlensing cases is curious.
Relatively minor difference between the millilensing and non-lensing as compared to the microlensing cases is curious.
#### Individual Chain Plots for Microlensing Test Run #### Individual Chain Plots for Microlensing Test Run
| Chain Title | Corner Plot | | Chain Title | Corner Plot |
| ----------- | ----------- | |-------------|-------------|
| Chain 0 | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/chain_0.png) | | Chain 0 | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/chain_0.png) |
| Chain 1 | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/chain_1.png) | | Chain 1 | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/chain_1.png) |
| Chain 2 | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/chain_2.png) | | Chain 2 | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/chain_2.png) |
| Chain 3 | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/chain_3.png) | | Chain 3 | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/chain_3.png) |
It would seem that the odd plotting behaviour of the combined samples is present in Chain 2, but all chains are in agreement. It would seem that the odd plotting behaviour of the combined samples is present in Chain 2, but all chains are in agreement.
#### Millilensing update #### Millilensing update
...@@ -56,15 +55,15 @@ After some bugs were identified, a rerun of the millilensing run has yielded the ...@@ -56,15 +55,15 @@ After some bugs were identified, a rerun of the millilensing run has yielded the
- Run Directory: `UWM:/home/mick.wright/gravelamps-o4-review/test-set-rerun/milli-as-milli-update` - Run Directory: `UWM:/home/mick.wright/gravelamps-o4-review/test-set-rerun/milli-as-milli-update`
- Status: Complete - Status: Complete
- Corners: - Corners:
- [Continuous Parameters](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/milli-update.png) - [Continuous Parameters](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/milli-update.png)
- [Signal Numbers](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/milli-discrete.png) - [Signal Numbers](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/milli-discrete.png)
- [Signal 0 Phase](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/milli-discrete-phase-0.png) - [Signal 0 Phase](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/milli-discrete-phase-0.png)
- [Signal 1 Phase](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/milli-discrete-phase-1.png) - [Signal 1 Phase](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/milli-discrete-phase-1.png)
- Parameters are largely as expected, signal 1 phase may bear some further investigation since it does not seem to yield confident constraint. - Parameters are largely as expected, signal 1 phase may bear some further investigation since it does not seem to yield confident constraint.
### Candidate O4 Sampling Review Tests ### Candidate O4 Sampling Review Tests
These runs use the configuration from the microlensing true run with the candidate sampler settings from the O4 Sampler Review. These runs use the configuration from the microlensing true run with the candidate sampler settings from the O4 Sampler Review.
- Candidate O4 Bilby Version: 2.1.0 - Candidate O4 Bilby Version: 2.1.0
- Candidate O4 Bilby Pipe Verison: 1.0.10 - Candidate O4 Bilby Pipe Verison: 1.0.10
...@@ -73,19 +72,19 @@ These runs use the configuration from the microlensing true run with the candida ...@@ -73,19 +72,19 @@ These runs use the configuration from the microlensing true run with the candida
- Run Directory: `UWM:/home/mick.wright/gravelamps-o4-review/sampler-settings/dynesty-review-candidate` - Run Directory: `UWM:/home/mick.wright/gravelamps-o4-review/sampler-settings/dynesty-review-candidate`
- Status: Complete - Status: Complete
- Corner: [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/dynesty-candidate.png) - Corner: [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/dynesty-candidate.png)
- Parameters recovered as expected - Parameters recovered as expected
Sign offs: Sign offs:
- [ ] Harsh * [ ] Harsh
- [ ] Suvodip * [ ] Suvodip
#### Bilby MCMC #### Bilby MCMC
- Run Directory: `UWM:/home/mick.wright/gravlemaps-o4-review/sampler-settings/bilby-mcmc-review-candidate` - Run Directory: `UWM:/home/mick.wright/gravlemaps-o4-review/sampler-settings/bilby-mcmc-review-candidate`
- Status: Complete - Status: Complete
- Corner: [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/bilby-mcmc-candidate.png) - Corner: [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/bilby-mcmc-candidate.png)
- Run did not converge, chains are very inconsistent - Run did not converge, chains are very inconsistent
#### Nessai #### Nessai
...@@ -96,10 +95,10 @@ Sign offs: ...@@ -96,10 +95,10 @@ Sign offs:
## Lens Generation ## Lens Generation
| Lens Model | Run Status | Run Directory | Amplification Plot | Sign Off (Harsh) | Sign Off (Suvodip) | | Lens Model | Run Status | Run Directory | Amplification Plot | Sign Off (Harsh) | Sign Off (Suvodip) |
| ---------- | ---------- | ------------- | ------------------ | ---------------- | ------------------ | |------------|------------|---------------|--------------------|------------------|--------------------|
| Point | Complete | UWM:`/home/mick.wright/gravelmaps-o4-review/generation-runs/point-generation` | [Plot](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/point-amplification.png) | | | | Point | Complete | UWM:`/home/mick.wright/gravelmaps-o4-review/generation-runs/point-generation` | [Plot](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/point-amplification.png) | | |
| SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/generation-runs/light-test/sis-generation` | [Plot](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/sis-amplification.png) | | | | SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/generation-runs/light-test/sis-generation` | [Plot](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/sis-amplification.png) | | |
| NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/generation-runs/light-test/nfw-generation-lighter` | [Plot](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/nfw-amplification.png) | | | | NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/generation-runs/light-test/nfw-generation-lighter` | [Plot](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/nfw-amplification.png) | | |
NFW feature in y = 0.01 line supported by higher accuracy run NFW feature in y = 0.01 line supported by higher accuracy run
...@@ -108,29 +107,30 @@ NFW feature in y = 0.01 line supported by higher accuracy run ...@@ -108,29 +107,30 @@ NFW feature in y = 0.01 line supported by higher accuracy run
All of these are being performed with the O4 candidate bilby, bilby_pipe, and dynesty and using the above settings for generation/the result files from these for SIS and NFW (to avoid duplication of lengthy work) All of these are being performed with the O4 candidate bilby, bilby_pipe, and dynesty and using the above settings for generation/the result files from these for SIS and NFW (to avoid duplication of lengthy work)
| Run | Status | Directory | Corner | Sign Off (Harsh) | Sign Off (Suvodip) | | Run | Status | Directory | Corner | Sign Off (Harsh) | Sign Off (Suvodip) |
| --- | ------ | --------- | ------ | ---------------- | ------------------ | |-----|--------|-----------|--------|------------------|--------------------|
| Point as Point | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/point-as-point` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/point-as-point.pdf) | | | | Point as Point | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/point-as-point` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/point-as-point.pdf) | | |
| Point as SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/point-as-sis` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/point-as-sis.pdf) | | | | Point as SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/point-as-sis` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/point-as-sis.pdf) | | |
| Point as NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/point-as-nfw` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/point-as-nfw.pdf) | | | | Point as NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/point-as-nfw` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/point-as-nfw.pdf) | | |
| Point as Unlensed | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/point-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/point-as-unlensed.pdf) | | | | Point as Unlensed | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/point-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/point-as-unlensed.pdf) | | |
| SIS as Point | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/sis-as-point` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/sis-as-point.pdf) | | | | SIS as Point | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/sis-as-point` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/sis-as-point.pdf) | | |
| SIS as SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/sis-as-sis` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/sis-as-sis.pdf) | | | | SIS as SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/sis-as-sis` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/sis-as-sis.pdf) | | |
| SIS as NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comaprison/sis-as-nfw` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/sis-as-nfw.pdf) | | | | SIS as NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comaprison/sis-as-nfw` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/sis-as-nfw.pdf) | | |
| SIS as Unlensed | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/sis-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/sis-as-unlensed.pdf) | | | | SIS as Unlensed | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/sis-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/sis-as-unlensed.pdf) | | |
| NFW as Point | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/nfw-as-point` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/nfw-as-point.pdf) | | | | NFW as Point | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/nfw-as-point` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/nfw-as-point.pdf) | | |
| NFW as SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/nfw-as-sis` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/nfw-as-sis.pdf) | | | | NFW as SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/nfw-as-sis` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/nfw-as-sis.pdf) | | |
| NFW as NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/nfw-as-nfw` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/nfw-as-nfw.pdf) | | | | NFW as NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/nfw-as-nfw` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/nfw-as-nfw.pdf) | | |
| NFW as Unlensed | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/nfw-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/nfw-as-unlensed.pdf) | | | | NFW as Unlensed | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/nfw-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/nfw-as-unlensed.pdf) | | |
| Unlensed as Point | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/unlensed-as-point` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/unlensed-as-point.pdf) | | | | Unlensed as Point | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/unlensed-as-point` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/unlensed-as-point.pdf) | | |
| Unlensed as SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/unlensed-as-sis` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/unlensed-as-sis.pdf) | | | | Unlensed as SIS | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/unlensed-as-sis` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/unlensed-as-sis.pdf) | | |
| Unlensed as NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/unlensed-as-nfw` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/unlensed-as-nfw.pdf) | | | | Unlensed as NFW | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/unlensed-as-nfw` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/unlensed-as-nfw.pdf) | | |
| Unlensed as Unlensed | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/unlensed-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/~mick.wright/gravelamps-o4-review/corners/unlensed-as-unlensed.pdf) | | | | Unlensed as Unlensed | Complete | UWM:`/home/mick.wright/gravelamps-o4-review/model-comparison/unlensed-as-unlensed` | [Corner](https://ldas-jobs.ligo.caltech.edu/\~mick.wright/gravelamps-o4-review/corners/unlensed-as-unlensed.pdf) | | |
### Log Bayes Factor Table ### Log Bayes Factor Table
| Model | B^{Model}_{Point} | B^{Model}_{SIS} | B^{Model}_{NFW} | B^{Model}_{Unlensed} | | Model | B^{Model}\_{Point} | B^{Model}\_{SIS} | B^{Model}\_{NFW} | B^{Model}\_{Unlensed} |
| ----- | ----------------- | --------------- | --------------- | -------------------- | |-------|--------------------|------------------|------------------|-----------------------|
| Point | N/A | 13.25 | 49.26 | 3729.28 | | Point | N/A | 13.25 | 49.26 | 3729.28 |
| SIS | 49.93 | N/A | 476.14 | 4016.60 | | SIS | 49.93 | N/A | 476.14 | 4016.60 |
| NFW | 37.01 | 27.00 | N/A | 113.80 | | NFW | 37.01 | 27.00 | N/A | 113.80 |
| Unlensed | 6.06 | -0.43 | -0.82 | N/A | | Unlensed | 6.06 | -0.43 | -0.82 | N/A |