... | ... | @@ -42,7 +42,7 @@ Git hash: 09e01c8972074ee4f44120afb9095ea76c458c9f |
|
|
## Scripts/Configs
|
|
|
| Script | Short description | Status | git hash | Comment | final sign-off |
|
|
|
|--------|-------------------|--------|----------|---------|----------------|
|
|
|
| [data_download.py](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/data_download_preparation/data_download.py) | Script for downloading events info, skymaps(.fits) from GraceDB and strain data from ligo servers using GWpy. Note: needs valid ligo key path(line 26) for accessing non-public data in Gracedb., eg: `/tmp/x509up\*` | DC: OK | 4e81d2bb94c38960e6e5d2e4104c94679a78bb67 | ------- | JRC------------ |
|
|
|
| [data_download.py](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/data_download_preparation/data_download.py) | Script for downloading events info, skymaps(.fits) from GraceDB and strain data from ligo servers using GWpy. Note: needs valid ligo key path(line 26) for accessing non-public data in Gracedb., eg: `/tmp/x509up\\\*` | DC: OK | 4e81d2bb94c38960e6e5d2e4104c94679a78bb67 | ------- | JRC------------ |
|
|
|
| [data_prepare.py](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/data_download_preparation/data_prepare.py) | Preparing Qtransform images, dataframes and skymaps for O3 real events given the event list and raw data downloaded from LIGO servers. Note: raw data on CIT is in `/home/srashti.goyal/lensid-ml-o3/data_download_preparation/O3` | DC: OK | 4e81d2bb94c38960e6e5d2e4104c94679a78bb67 | --- | -JRC------------- |
|
|
|
| [ml_predict_workflow.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/ml_predict_workflow.py), [config_O3_events.yaml](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/config_O3_events.yaml) | Script for computing ML predictions for a given event pairs in dataframe, their skymaps, Qtransforms and the trained ML models. Optionally computes the False Positive Probabilities given the background. It requires config file as input. Eg: `lensid_make_predictions -config /home/srashti.goyal/lensid-ml-O3/config_O3_events.yaml` Note: change `odir` in **config** file. | JR: OK | ---------- | DC: Should be considered in the results review | -JRC--------------- |
|
|
|
| [get_candidates_compare_to_blu_tagged.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/O3_ML_gwtc3/get_candidates_compare_to_blu_tagged.ipynb) | Notebook for comparing ML and BLU results for the full O3 catalogue of BBHs. | JRC: OK | 09e01c8972074ee4f44120afb9095ea76c458c9f | --------- | -JRC--------------- |
|
... | ... | @@ -80,7 +80,12 @@ PO and LensID results tracking [page](https://git.ligo.org/srashti.goyal/lensid- |
|
|
</td>
|
|
|
<td>List of events considered for the O3 lensing analysis</td>
|
|
|
<td>OK-JRC--------</td>
|
|
|
<td>4e81d2bb94c38960e6e5d2e4104c94679a78bb67</td>
|
|
|
<td>
|
|
|
|
|
|
4e81d2bb94c38960e6e5d2e4104c94679a78bb67
|
|
|
|
|
|
09e01c8972074ee4f44120afb9095ea76c458c9f
|
|
|
</td>
|
|
|
<td>---------</td>
|
|
|
<td>
|
|
|
|
... | ... | @@ -94,7 +99,12 @@ PO and LensID results tracking [page](https://git.ligo.org/srashti.goyal/lensid- |
|
|
</td>
|
|
|
<td>Contains paths to the Qtransforms, Skymaps, background, machines used for producing the final results</td>
|
|
|
<td>OK-JRC--------</td>
|
|
|
<td>4e81d2bb94c38960e6e5d2e4104c94679a78bb67</td>
|
|
|
<td>
|
|
|
|
|
|
4e81d2bb94c38960e6e5d2e4104c94679a78bb67
|
|
|
|
|
|
09e01c8972074ee4f44120afb9095ea76c458c9f
|
|
|
</td>
|
|
|
<td>---------</td>
|
|
|
<td>
|
|
|
|
... | ... | @@ -128,6 +138,8 @@ PO statistics results used for FPP computations and making the plot. md5sum: `03 |
|
|
<td>
|
|
|
|
|
|
45d7bde987481f175e065ee3665ec03d3ab7ca5e
|
|
|
|
|
|
09e01c8972074ee4f44120afb9095ea76c458c9f
|
|
|
</td>
|
|
|
<td>---------</td>
|
|
|
<td>----------------</td>
|
... | ... | @@ -141,8 +153,13 @@ PO statistics results used for FPP computations and making the plot. md5sum: `03 |
|
|
|
|
|
PO statistics background injection results used for FPP computations and making the plot. md5sum: `a85bbc0327a128ca29323fe697a7b3ef`
|
|
|
</td>
|
|
|
<td>--------</td>
|
|
|
<td>45d7bde987481f175e065ee3665ec03d3ab7ca5e</td>
|
|
|
<td>OK-JRC--------</td>
|
|
|
<td>
|
|
|
|
|
|
45d7bde987481f175e065ee3665ec03d3ab7ca5e
|
|
|
|
|
|
09e01c8972074ee4f44120afb9095ea76c458c9f
|
|
|
</td>
|
|
|
<td>---------</td>
|
|
|
<td>----------------</td>
|
|
|
</tr>
|
... | ... | @@ -169,8 +186,13 @@ PO statistics background injection results used for FPP computations and making |
|
|
[Plot notebook](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/O3_ML_gwtc3/plot_ml_blu_predictions.ipynb)
|
|
|
</td>
|
|
|
<td>Notebook to plot the fig. 1 from the ML and PO analysis results, and preparing cleaned up results for data release.</td>
|
|
|
<td>--------</td>
|
|
|
<td>45d7bde987481f175e065ee3665ec03d3ab7ca5e</td>
|
|
|
<td>OK-JRC--------</td>
|
|
|
<td>
|
|
|
|
|
|
45d7bde987481f175e065ee3665ec03d3ab7ca5e
|
|
|
|
|
|
09e01c8972074ee4f44120afb9095ea76c458c9f
|
|
|
</td>
|
|
|
<td>---------</td>
|
|
|
<td>
|
|
|
|
... | ... | @@ -182,8 +204,13 @@ PO statistics background injection results used for FPP computations and making |
|
|
[Data release](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/O3_ML_gwtc3/ML_PO_FPPs_data_release.csv)
|
|
|
</td>
|
|
|
<td>Cleaned up version of final results</td>
|
|
|
<td>--------</td>
|
|
|
<td>45d7bde987481f175e065ee3665ec03d3ab7ca5e</td>
|
|
|
<td>OK-JRC--------</td>
|
|
|
<td>
|
|
|
|
|
|
45d7bde987481f175e065ee3665ec03d3ab7ca5e
|
|
|
|
|
|
09e01c8972074ee4f44120afb9095ea76c458c9f
|
|
|
</td>
|
|
|
<td>---------</td>
|
|
|
<td>
|
|
|
|
... | ... | @@ -245,7 +272,7 @@ PO statistics background injection results used for FPP computations and making |
|
|
## 5 January 2022
|
|
|
|
|
|
* We discussed the performance of XGBoost with QTs for the 2 detector real events.
|
|
|
* Imputing the dense outputs(i.e. input features of XGBoost) for the missing data help but is not logically understood. (notebook)\[https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/review/missing_strain_ML_qts.ipynb\]
|
|
|
* Imputing the dense outputs(i.e. input features of XGBoost) for the missing data help but is not logically understood. (notebook)\[https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/review/missing_strain_ML_qts.ipynb\\\]
|
|
|
* We converged on the training configurations for the final ML results for O3.
|
|
|
|
|
|
### Action items:
|
... | ... | |