Changes

Page history
Update Result Review O4a authored by Jean-Rene Cudell's avatar Jean-Rene Cudell
Show whitespace changes
Inline Side-by-side
Result-Review-O4a.md
View page @ 5f7c140f
...@@ -8,6 +8,10 @@ LensID O4 MDC : [lensing_mdc_o4](https://git.ligo.org/srashti.goyal/lensing_mdc_ ...@@ -8,6 +8,10 @@ LensID O4 MDC : [lensing_mdc_o4](https://git.ligo.org/srashti.goyal/lensing_mdc_
O4a repo: [lensid-ml-o4](https://git.ligo.org/srashti.goyal/lensid-ml-o4) O4a repo: [lensid-ml-o4](https://git.ligo.org/srashti.goyal/lensid-ml-o4)
**The input data for O4a and MDC is saved on CIT:**
`/home/srashti.goyal/lensid_O4/data_download_preparation` and `/home/srashti.goyal/lensing_MDC_O4/data_prep/data`
## Useful links ## Useful links
- slides [Till O3 method, review, results](https://docs.google.com/presentation/d/10bIhtFae5RIJ3WBJg1Lcy7PueSKwxh1m2APRDN0w0PA/edit?usp=sharing) - slides [Till O3 method, review, results](https://docs.google.com/presentation/d/10bIhtFae5RIJ3WBJg1Lcy7PueSKwxh1m2APRDN0w0PA/edit?usp=sharing)
...@@ -94,29 +98,19 @@ ML Models: ...@@ -94,29 +98,19 @@ ML Models:
| File(s) | Short description | Status | Comment | Git hash | Sign-off | | File(s) | Short description | Status | Comment | Git hash | Sign-off |
|---------|-------------------|--------|---------|----------|----------| |---------|-------------------|--------|---------|----------|----------|
| [pop_datasets.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/retraining_for_O4/pop_datasets.ipynb) | Notebook having plots of injection parameters for training and testing. | Done | \-------------- | | | | [pop_datasets.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/retraining_for_O4/pop_datasets.ipynb) | Notebook having plots of injection parameters for training and testing. | Done | \-------------- | :heavy_check_mark: | |
| [PSD_plots.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/retraining_for_O4/PSD_plots.ipynb) | Comparison of the PSDs used for training and testing in O3 and O4 etc. | Reviewed | Reviewed | 5f525a652d0374f11a207bc8158f7c75d37de884 | :heavy_check_mark: :heavy_check_mark: | | [PSD_plots.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/retraining_for_O4/PSD_plots.ipynb) | Comparison of the PSDs used for training and testing in O3 and O4 etc. | Reviewed | Reviewed | 5f525a652d0374f11a207bc8158f7c75d37de884 | :heavy_check_mark: :heavy_check_mark: |
| ML QTs [L1](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/O4a_training/L1/uniform_config_lr_0.01_ep_15_bs_500/), [H1](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/O4a_training/uniform_config_lr_0.01_ep_15_bs_500/) | Production ML QTs models train directories. Uniform in masses for H1 and L1 | Done | | \-------------- | | | ML QTs [L1](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/O4a_training/L1/uniform_config_lr_0.01_ep_15_bs_500/), [H1](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/O4a_training/uniform_config_lr_0.01_ep_15_bs_500/) | Production ML QTs models train directories. Uniform in masses for H1 and L1 | Done | | \-------------- | |
| [2vs3det.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/retraining_for_O4/2vs3det.ipynb) | Train and Test ML Skymaps. Benchmark performance for HL v/s HLV | Reviewed | Why some values are NaN in DataFrame?, Why no orange curve in plots?, Choice of hyperparameters for training? - Sourabh | 5f525a652d0374f11a207bc8158f7c75d37de884 | :heavy_check_mark: | | [2vs3det.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/retraining_for_O4/2vs3det.ipynb) | Train and Test ML Skymaps. Benchmark performance for HL v/s HLV | Reviewed | Why some values are NaN in DataFrame?, Why no orange curve in plots?, Choice of hyperparameters for training? - Sourabh | 5f525a652d0374f11a207bc8158f7c75d37de884 | :heavy_check_mark: |
| [make_predictions.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/O4a_make_predictions.ipynb) | Demo LensID O4a, ML models benchmark performance and background estimations. | Done | | \-------------- | | | [make_predictions.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/O4a_make_predictions.ipynb) | Demo LensID O4a, ML models benchmark performance and background estimations. | Need to clean up | | \-------------- | |
O4a Predictions: O4a Predictions:
**The input data for O4a and MDC is saved on CIT:**
`/home/srashti.goyal/lensid_O4/data_download_preparation` and `/home/srashti.goyal/lensing_MDC_O4/data_prep/data`
**To reproduce results on CIT:**
`conda activate /home/srashti.goyal/.conda/envs/tensorflowenvpy39`
`alensid_compute_compatibility --outdir /home/srashti.goyal/public_html/lensid_runs/O4a_alensid --label May25 --config /home/srashti.goyal/alensidforlensingflow/Examples/SimpleExample/config_o4a.yaml --event1 S230606d --event2 S231226av --singlepair 0`
| File(s) | Short description | Status | Comment | Git hash | Sign-off | | File(s) | Short description | Status | Comment | Git hash | Sign-off |
|---------|-------------------|--------|---------|----------|----------| |---------|-------------------|--------|---------|----------|----------|
| [Config](https://git.ligo.org/srashti.goyal/alensidforlensingflow/-/blob/main/Examples/SimpleExample/config_o4a.yaml) | ML models and config for production runs, | Done | | \-------------- | | | [Config](https://git.ligo.org/srashti.goyal/alensidforlensingflow/-/blob/main/Examples/SimpleExample/config_o4a.yaml) | ML models and config for production runs | Done | | \-------------- | |
| [O4a Production Run](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/lensid_runs/O4a_alensid/result/) | ML Production run on CIT | Done | | \-------------- | | | [O4a Production Run](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/lensid_runs/O4a_alensid/result/) | ML Production run on CIT | Done | | \-------------- | |
| [investigations_visualisations.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/investigations_visualisations.ipynb?ref_type=heads) | Investigations with BD cuts, Eyeballing pairs, and comparing the performances with the other pipelines | Done | | \-------------- | | | [investigations_visualisations.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/investigations_visualisations.ipynb?ref_type=heads) | ML pipeline run through a notebook with Investigations/Eyeballing pairs and comparing the performances with the other pipelines | Done | | \-------------- | |
Publication results: Publication results:
...@@ -201,7 +195,7 @@ The review call happen on Wednesdays 1 PM CEST/ 4:30 PM IST virtual IFPA room: h ...@@ -201,7 +195,7 @@ The review call happen on Wednesdays 1 PM CEST/ 4:30 PM IST virtual IFPA room: h
## 8 November 2023 ## 8 November 2023
* We discussed the integration of lensid with the lensing flow and visited the new package : https://git.ligo.org/srashti.goyal/alensidforlensingflow * We discussed the integration of lensid with the lensing flow and visited the new package : https://git.ligo.org/srashti.goyal/alensidforlensingflow
* JR suggested to use O4a real noise PSDs for the training and testing of the final ML model for the production runs. (detchar)\[https://ldas-jobs.ligo-wa.caltech.edu/~detchar/summary\\\\\\\] * JR suggested to use O4a real noise PSDs for the training and testing of the final ML model for the production runs. (detchar)\[https://ldas-jobs.ligo-wa.caltech.edu/~detchar/summary\\\\\]
* We are still unsure about the inclusion of the population and the time delay lensing priors for the follow-up strategy. * We are still unsure about the inclusion of the population and the time delay lensing priors for the follow-up strategy.
## Action items ## Action items
...@@ -282,13 +276,3 @@ We also want to prepare a document to quantify all the things. One main issue is ...@@ -282,13 +276,3 @@ We also want to prepare a document to quantify all the things. One main issue is
* [x] Prepare the scripts for result review. * [x] Prepare the scripts for result review.
* [x] Train a final ML and background while optimising. * [x] Train a final ML and background while optimising.
\ No newline at end of file
## 22 May 2025
* We discussed the LensID results and the things that need to be reviewed.
* The results are produced manually and not with LensingFlow, as requested by the group.
* The Result review page is updated with final results, and JR and Sourabh will go through the.
### Action items
* [x] Update result review page.
\ No newline at end of file