Changes

Page history
Update Result Review O4a authored by Srashti Goyal's avatar Srashti Goyal
Hide whitespace changes
Inline Side-by-side
Result-Review-O4a.md
View page @ 09e849c0
...@@ -24,7 +24,9 @@ O4a repo: [lensid-ml-o4](https://git.ligo.org/srashti.goyal/lensid-ml-o4) ...@@ -24,7 +24,9 @@ O4a repo: [lensid-ml-o4](https://git.ligo.org/srashti.goyal/lensid-ml-o4)
# Final statement # Final statement
### Reviewer: Jean-Rene Cudell ### Reviewer: Jean-Rene Cudell [@jean-rene.cudell](https://git.ligo.org/jean-rene.cudell) , Sourabh Magare [@sourabh.magare](https://git.ligo.org/sourabh.magare)
Analysts: Srashti Goyal @srashti.goyal
Final Sign-off: Date: Final Sign-off: Date:
...@@ -57,7 +59,9 @@ Script for downloading events info from CBCFlow (path loaded on CIT), skymaps(.f ...@@ -57,7 +59,9 @@ Script for downloading events info from CBCFlow (path loaded on CIT), skymaps(.f
<td>Reviewed</td> <td>Reviewed</td>
<td></td> <td></td>
<td>include a link to the instructions for generating the LIGO key path in comments.</td> <td>include a link to the instructions for generating the LIGO key path in comments.</td>
<td>:heavy_check_mark: :heavy_check_mark: <td>
:heavy_check_mark: :heavy_check_mark:
</td> </td>
</tr> </tr>
<tr> <tr>
...@@ -88,37 +92,50 @@ Prepare cartesian skymaps and qtransforms for O4a the real events. Also filters ...@@ -88,37 +92,50 @@ Prepare cartesian skymaps and qtransforms for O4a the real events. Also filters
:heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
</td> </td>
</tr> </tr>
<tr>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
</table> </table>
ML Models: ML Models:
| File(s) | Short description | Status | Comment | Git hash | Sign-off | | File(s) | Short description | Status | Comment | Git hash | Sign-off |
|---------|-------------------|--------|---------|----------|----------| |---------|-------------------|--------|---------|----------|----------|
| [pop_datasets.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/retraining_for_O4/pop_datasets.ipynb) | Notebook having plots of injection parameters for training and testing. | Done | -------------- | | | | [pop_datasets.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/retraining_for_O4/pop_datasets.ipynb) | Notebook having plots of injection parameters for training and testing. | Done | \-------------- | | |
| [PSD_plots.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/retraining_for_O4/PSD_plots.ipynb) | Comparison of the PSDs used for training and testing in O3 and O4 etc. | Reviewed | OK | 5f525a652d0374f11a207bc8158f7c75d37de884 | :heavy_check_mark: :heavy_check_mark: | | [PSD_plots.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/retraining_for_O4/PSD_plots.ipynb) | Comparison of the PSDs used for training and testing in O3 and O4 etc. | Reviewed | Reviewed | 5f525a652d0374f11a207bc8158f7c75d37de884 | :heavy_check_mark: :heavy_check_mark: |
| ML QTs [L1](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/O4a_training/L1/uniform_config_lr_0.01_ep_15_bs_500/), [H1](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/O4a_training/uniform_config_lr_0.01_ep_15_bs_500/) | Production ML QTs models train directories. Uniform in masses for H1 and L1 | Done | | -------------- | | | ML QTs [L1](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/O4a_training/L1/uniform_config_lr_0.01_ep_15_bs_500/), [H1](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/O4a_training/uniform_config_lr_0.01_ep_15_bs_500/) | Production ML QTs models train directories. Uniform in masses for H1 and L1 | Done | | \-------------- | |
| [2vs3det.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/retraining_for_O4/2vs3det.ipynb) | Train and Test ML Skymaps. Benchmark performance for HL v/s HLV | Reviewed | Why some values are NaN in DataFrame?, Why no orange curve in plots?, Choice of hyperparameters for training? - Sourabh | 5f525a652d0374f11a207bc8158f7c75d37de884 | :heavy_check_mark: | | [2vs3det.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/retraining_for_O4/2vs3det.ipynb) | Train and Test ML Skymaps. Benchmark performance for HL v/s HLV | Reviewed | Why some values are NaN in DataFrame?, Why no orange curve in plots?, Choice of hyperparameters for training? - Sourabh | 5f525a652d0374f11a207bc8158f7c75d37de884 | :heavy_check_mark: |
| [make_predictions.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/make_predictions.ipynb?ref_type=heads) |Demo LensID O4a, ML models benchmark performance and background estimations. | Done | | -------------- | | | [make_predictions.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/make_predictions.ipynb?ref_type=heads) | Demo LensID O4a, ML models benchmark performance and background estimations. | Need to clean up | | \-------------- | |
O4a Predictions: O4a Predictions:
| File(s) | Short description | Status | Comment | Git hash | Sign-off | | File(s) | Short description | Status | Comment | Git hash | Sign-off |
|---------|-------------------|--------|---------|----------|----------| |---------|-------------------|--------|---------|----------|----------|
| [Config](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/config_o4a.yaml?ref_type=heads) | ML models and config for production runs | Done| | -------------- | | | [Config](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/config_o4a.yaml?ref_type=heads) | ML models and config for production runs | Done | | \-------------- | |
| [O4a Production Run](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/lensid_runs/O4a_alensid/result/) | ML Production run on CIT | Done | | -------------- | | | [O4a Production Run](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/lensid_runs/O4a_alensid/result/) | ML Production run on CIT | Done | | \-------------- | |
| [investigations_visualisations.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/investigations_visualisations.ipynb?ref_type=heads) | ML pipeline run through a notebook with Investigations/Eyeballing pairs and comparing the performances with the other pipelines | Done | | -------------- | | | [investigations_visualisations.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/investigations_visualisations.ipynb?ref_type=heads) | ML pipeline run through a notebook with Investigations/Eyeballing pairs and comparing the performances with the other pipelines | Done | | \-------------- | |
Publication results: Publication results:
| File(s) | Short description | Status | Comment | Git hash | Sign-off | | File(s) | Short description | Status | Comment | Git hash | Sign-off |
|---------|-------------------|--------|---------|----------|----------| |---------|-------------------|--------|---------|----------|----------|
| [Final Results](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/LensID_results_5May25.csv) | O4a Pairs analysed by LensID | Done | | b3a5135e0f03ad7587e53b2814feb2de10b94507| | | [Final Results](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/LensID_results_5May25.csv) | O4a Pairs analysed by LensID | Done | | b3a5135e0f03ad7587e53b2814feb2de10b94507 | |
| [Interesting Pairs](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/interesting_pairs_5May25.csv?ref_type=heads) | Pairs passed for follow-up analysis | Done | | d20a4c29ffa623e771d664709747a57bb68ae5ce| | | [Interesting Pairs](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/interesting_pairs_5May25.csv?ref_type=heads) | Pairs passed for follow-up analysis | Done | | d20a4c29ffa623e771d664709747a57bb68ae5ce | |
| [final_results_plots.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/final_results_plots.ipynb) | Plot of rejected and selected pairs | needs PWT approval | | -------------- | | | [final_results_plots.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/final_results_plots.ipynb) | Plot of rejected and selected pairs | needs PWT approval | | \-------------- | |
| [Lensing paper plot](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/LensID_O4a.pdf) | Plot of rejected and selected pairs | needs PWT approval | | -------------- | | | [Lensing paper plot](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/LensID_O4a.pdf) | Plot of rejected and selected pairs | needs PWT approval | | \-------------- | |
# Review Calls # Review Calls
...@@ -194,7 +211,7 @@ The review call happen on Wednesdays 1 PM CEST/ 4:30 PM IST virtual IFPA room: h ...@@ -194,7 +211,7 @@ The review call happen on Wednesdays 1 PM CEST/ 4:30 PM IST virtual IFPA room: h
## 8 November 2023 ## 8 November 2023
* We discussed the integration of lensid with the lensing flow and visited the new package : https://git.ligo.org/srashti.goyal/alensidforlensingflow * We discussed the integration of lensid with the lensing flow and visited the new package : https://git.ligo.org/srashti.goyal/alensidforlensingflow
* JR suggested to use O4a real noise PSDs for the training and testing of the final ML model for the production runs. (detchar)\[https://ldas-jobs.ligo-wa.caltech.edu/~detchar/summary\] * JR suggested to use O4a real noise PSDs for the training and testing of the final ML model for the production runs. (detchar)\[https://ldas-jobs.ligo-wa.caltech.edu/~detchar/summary\\]
* We are still unsure about the inclusion of the population and the time delay lensing priors for the follow-up strategy. * We are still unsure about the inclusion of the population and the time delay lensing priors for the follow-up strategy.
## Action items ## Action items
... ...
......