Changes

Page history
Update Result Review O4a authored by Srashti Goyal's avatar Srashti Goyal
Show whitespace changes
Inline Side-by-side
Result-Review-O4a.md
View page @ aae87688
...@@ -38,27 +38,44 @@ Git hash: ...@@ -38,27 +38,44 @@ Git hash:
O4a Data preparation scripts: O4a Data preparation scripts:
| Script | Short description | Status | git hash | Comment | final sign-off |
|--------|-------------------|--------|----------|---------|----------------|
| [data_download_cbcflow.py](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/data_download_preparation/data_download_cbcflow.py) | Script for downloading events info from CBCFlow (path loaded on CIT), skymaps(.fits) from GraceDB and strain data from LIGO servers using GWpy. Note: needs valid ligo key path for accessing non-public data in Gracedb., eg: `/tmp/x509up\\\\\\\*` Raw data on CIT is in `/home/srashti.goyal/lensid_O4/data_download_preparation/O4a_events_data` | Reviewed | d20a4c29ffa623e771d664709747a57bb68ae5ce | include a link to the instructions for generating the LIGO key path in comments. | :heavy_check_mark: :heavy_check_mark: |
| [data_prepare.py](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/data_download_preparation/data_prepare.py) | Prepare cartesian skymaps and qtransforms for O4a the real events. Also filters events compatible with lensid (BBHs) and prepares a dataframe. Note: The prepared inputs are saved on CIT in `/home/srashti.goyal/lensid_O4/data_download_preparation/O4a_lensid_inputs` | Reviewed | 1871c895a6a8b1e9d5d77854df1c0c38ba2a0ea0 | \--- | :heavy_check_mark: :heavy_check_mark: |
ML Models:
| File(s) | Short description | Status | Comment | Git hash | Sign-off |
|---------|-------------------|--------|---------|----------|----------|
| [pop_datasets.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/retraining_for_O4/pop_datasets.ipynb) | Notebook having plots of injection parameters for training and testing. | Done | JR-What are the columns in Out \[9\] | | :heavy_check_mark: :heavy_check_mark: |
| [PSD_plots.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/retraining_for_O4/PSD_plots.ipynb) | Comparison of the PSDs used for training and testing in O3 and O4 etc. | Reviewed | Reviewed | 5f525a652d0374f11a207bc8158f7c75d37de884 | :heavy_check_mark: :heavy_check_mark: |
| ML QTs [L1](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/O4a_training/L1/uniform_config_lr_0.01_ep_15_bs_500/), [H1](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/O4a_training/uniform_config_lr_0.01_ep_15_bs_500/) | Production ML QTs models train directories. Uniform in masses for H1 and L1 | Reviewed | OK | \-------------- | :heavy_check_mark: :heavy_check_mark: |
| [2vs3det.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/retraining_for_O4/2vs3det.ipynb) | Train and Test ML Skymaps. Benchmark performance for HL v/s HLV | Reviewed | Why some values are NaN in DataFrame?, Why no orange curve in plots?, Choice of hyperparameters for training? - Sourabh | 5f525a652d0374f11a207bc8158f7c75d37de884 | :heavy_check_mark: |
| [make_predictions.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/O4a_make_predictions.ipynb) | Demo LensID O4a, ML models benchmark performance and background estimations. | Done | | \-------------- | |
O4a Predictions:
<table> <table>
<tr> <tr>
<th>Script</th> <th>File(s)</th>
<th>Short description</th> <th>Short description</th>
<th>Status</th> <th>Status</th>
<th>git hash</th>
<th>Comment</th> <th>Comment</th>
<th>final sign-off</th> <th>Git hash</th>
<th>Sign-off</th>
</tr> </tr>
<tr> <tr>
<td> <td>
[data_download_cbcflow.py](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/data_download_preparation/data_download_cbcflow.py) [Config](https://git.ligo.org/srashti.goyal/alensidforlensingflow/-/blob/main/Examples/SimpleExample/config_o4a.yaml)
</td> </td>
<td>ML models and config for production runs</td>
<td>Reviewed</td>
<td></td>
<td> <td>
Script for downloading events info from CBCFlow (path loaded on CIT), skymaps(.fits) from GraceDB and strain data from LIGO servers using GWpy. Note: needs valid ligo key path for accessing non-public data in Gracedb., eg: `/tmp/x509up\\\\\\\*` Raw data on CIT is in `/home/srashti.goyal/lensid_O4/data_download_preparation/O4a_events_data` \--------------
</td> </td>
<td>Reviewed</td>
<td></td>
<td>include a link to the instructions for generating the LIGO key path in comments.</td>
<td> <td>
:heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
...@@ -67,51 +84,36 @@ Script for downloading events info from CBCFlow (path loaded on CIT), skymaps(.f ...@@ -67,51 +84,36 @@ Script for downloading events info from CBCFlow (path loaded on CIT), skymaps(.f
<tr> <tr>
<td> <td>
[data_prepare.py](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/data_download_preparation/data_prepare.py) [O4a Production Run](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/lensid_runs/O4a_alensid/result/)
</td> </td>
<td>ML Production run on CIT</td>
<td>Reviewed</td>
<td></td>
<td> <td>
Prepare cartesian skymaps and qtransforms for O4a the real events. Also filters events compatible with lensid (BBHs) and prepares a dataframe. Note: The prepared inputs are saved on CIT in `/home/srashti.goyal/lensid_O4/data_download_preparation/O4a_lensid_inputs` \--------------
</td> </td>
<td> <td>
<table> :heavy_check_mark: :heavy_check_mark:
<tr>
<td>Reviewed</td>
</tr>
</table>
</td> </td>
<td>1871c895a6a8b1e9d5d77854df1c0c38ba2a0ea0</td> </tr>
<tr>
<td> <td>
\--- [investigations_visualisations.html](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/investigations_lensidO4a.html)
</td> </td>
<td>ML pipeline run through a notebook with Investigations/Eyeballing pairs and comparing the performances with the other pipelines</td>
<td>Done</td>
<td></td>
<td> <td>
:heavy_check_mark: :heavy_check_mark: \--------------
</td> </td>
<td></td>
</tr> </tr>
</table> </table>
ML Models:
| File(s) | Short description | Status | Comment | Git hash | Sign-off |
|---------|-------------------|--------|---------|----------|----------|
| [pop_datasets.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/retraining_for_O4/pop_datasets.ipynb) | Notebook having plots of injection parameters for training and testing. | Done | JR-What are the columns in Out \[9\] | | :heavy_check_mark: :heavy_check_mark: |
| [PSD_plots.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/retraining_for_O4/PSD_plots.ipynb) | Comparison of the PSDs used for training and testing in O3 and O4 etc. | Reviewed | Reviewed | 5f525a652d0374f11a207bc8158f7c75d37de884 | :heavy_check_mark: :heavy_check_mark: |
| ML QTs [L1](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/O4a_training/L1/uniform_config_lr_0.01_ep_15_bs_500/), [H1](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/O4a_training/uniform_config_lr_0.01_ep_15_bs_500/) | Production ML QTs models train directories. Uniform in masses for H1 and L1 | Done | JR-Should one worry about the missing npz files? What is the difference between ML and dense? Which curve is QT+skymaps? | \-------------- | |
| [2vs3det.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/retraining_for_O4/2vs3det.ipynb) | Train and Test ML Skymaps. Benchmark performance for HL v/s HLV | Reviewed | Why some values are NaN in DataFrame?, Why no orange curve in plots?, Choice of hyperparameters for training? - Sourabh | 5f525a652d0374f11a207bc8158f7c75d37de884 | :heavy_check_mark: |
| [make_predictions.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/O4a_make_predictions.ipynb) | Demo LensID O4a, ML models benchmark performance and background estimations. | Done | | \-------------- | |
O4a Predictions:
| File(s) | Short description | Status | Comment | Git hash | Sign-off |
|---------|-------------------|--------|---------|----------|----------|
| [Config](https://git.ligo.org/srashti.goyal/alensidforlensingflow/-/blob/main/Examples/SimpleExample/config_o4a.yaml) | ML models and config for production runs | Done | | \-------------- | :heavy_check_mark: :heavy_check_mark: |
| [O4a Production Run](https://ldas-jobs.ligo.caltech.edu/~srashti.goyal/lensid_runs/O4a_alensid/result/) | ML Production run on CIT | Done | | \-------------- | :heavy_check_mark: :heavy_check_mark: |
| [investigations_visualisations.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o4/-/blob/master/investigations_visualisations.ipynb?ref_type=heads) | ML pipeline run through a notebook with Investigations/Eyeballing pairs and comparing the performances with the other pipelines | Done | JR-The file could not be displayed because it is too large. | \-------------- | |
Publication results: Publication results:
| File(s) | Short description | Status | Comment | Git hash | Sign-off | | File(s) | Short description | Status | Comment | Git hash | Sign-off |
...@@ -195,7 +197,7 @@ The review call happens on Wednesdays 1 PM CEST/ 4:30 PM IST virtual IFPA room: ...@@ -195,7 +197,7 @@ The review call happens on Wednesdays 1 PM CEST/ 4:30 PM IST virtual IFPA room:
## 8 November 2023 ## 8 November 2023
* We discussed the integration of lensid with the lensing flow and visited the new package: https://git.ligo.org/srashti.goyal/alensidforlensingflow * We discussed the integration of lensid with the lensing flow and visited the new package: https://git.ligo.org/srashti.goyal/alensidforlensingflow
* JR suggested to use O4a real noise PSDs for the training and testing of the final ML model for the production runs. (detchar)\[https://ldas-jobs.ligo-wa.caltech.edu/~detchar/summary\\\\\\\\] * JR suggested to use O4a real noise PSDs for the training and testing of the final ML model for the production runs. (detchar)\[https://ldas-jobs.ligo-wa.caltech.edu/~detchar/summary\\\\\\\\\]
* We are still unsure about the inclusion of the population and the time delay lensing priors for the follow-up strategy. * We are still unsure about the inclusion of the population and the time delay lensing priors for the follow-up strategy.
## Action items ## Action items
... ...
......