Changes

Page history
Update Result Review O3b lensing authored by Jean-Rene Cudell's avatar Jean-Rene Cudell
Hide whitespace changes
Inline Side-by-side
Result-Review-O3b-lensing.md
View page @ f80bbaee
...@@ -16,9 +16,10 @@ This repository deals with the result review of the machine learning based pipel ...@@ -16,9 +16,10 @@ This repository deals with the result review of the machine learning based pipel
# Final statement # Final statement
### Can be found [here](https://git.ligo.org/srashti.goyal/lensid/-/wikis/Result-Review-Final-Statement) ### Can be found [here](https://git.ligo.org/srashti.goyal/lensid/-/wikis/Result-Review-Final-Statement)
### Reviewer: Jean-Rene Cudell ### Reviewer: Jean-Rene Cudell
Final Sign-off: :heavy_check_mark: Final Sign-off:
Date: 14th April 2022 Date: 14th April 2022
...@@ -30,14 +31,11 @@ Git hash: 09e01c8972074ee4f44120afb9095ea76c458c9f ...@@ -30,14 +31,11 @@ Git hash: 09e01c8972074ee4f44120afb9095ea76c458c9f
### Reviewer: Jean-Rene Cudell ### Reviewer: Jean-Rene Cudell
Final Sign-off: Final Sign-off: :heavy_check_mark:
Date:
Git hash:
Date: May 11, 2022
Git hash: 09e01c8972074ee4f44120afb9095ea76c458c9f
# Result review # Result review
...@@ -57,9 +55,9 @@ Git hash: ...@@ -57,9 +55,9 @@ Git hash:
| [ML_blu_FPPs_inj_pars_investigate.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/review/ML_blu_FPPs_inj_pars_investigate.ipynb) | Notebook comparing the ML and BLU FPPs for each pair in test set, and also investigationg correlations with the injection parameters. Also contain statistics of input sky features for lensed and unlensed test pairs. | JRC: OK | 09e01c8972074ee4f44120afb9095ea76c458c9f | -------------- | | [ML_blu_FPPs_inj_pars_investigate.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/review/ML_blu_FPPs_inj_pars_investigate.ipynb) | Notebook comparing the ML and BLU FPPs for each pair in test set, and also investigationg correlations with the injection parameters. Also contain statistics of input sky features for lensed and unlensed test pairs. | JRC: OK | 09e01c8972074ee4f44120afb9095ea76c458c9f | -------------- |
| [PSD_plots.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/review/PSD_plots.ipynb) | Notebook having plots of PSDs that are used. | JRC: OK | 09e01c8972074ee4f44120afb9095ea76c458c9f | -------------- | | [PSD_plots.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/review/PSD_plots.ipynb) | Notebook having plots of PSDs that are used. | JRC: OK | 09e01c8972074ee4f44120afb9095ea76c458c9f | -------------- |
| [hp_cartview.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/review/hp_cartview.ipynb) | Sanity check for bayestar skymaps to see overall probability and the overlap calculation for cartesian projection v/s the flattened fits. | JRC: OK | 09e01c8972074ee4f44120afb9095ea76c458c9f | ---------------- | | [hp_cartview.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/review/hp_cartview.ipynb) | Sanity check for bayestar skymaps to see overall probability and the overlap calculation for cartesian projection v/s the flattened fits. | JRC: OK | 09e01c8972074ee4f44120afb9095ea76c458c9f | ---------------- |
| [background_injections_ML_blu.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/notebooks/O3a_events/background_injections_ML_blu.ipynb) | Notebook showing ML and BLU outputs for the background unlensed injections as simulated by Haris during O3a analysis. | JRC: OK |09e01c8972074ee4f44120afb9095ea76c458c9f | -------------- | | [background_injections_ML_blu.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/notebooks/O3a_events/background_injections_ML_blu.ipynb) | Notebook showing ML and BLU outputs for the background unlensed injections as simulated by Haris during O3a analysis. | JRC: OK | 09e01c8972074ee4f44120afb9095ea76c458c9f | -------------- |
| [investigations_visualisations.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/review/investigations_visualisations.ipynb) | Notebook to visualise qtransforms and skymaps of the interesting candidate real event pairs. | JRC: OK | --------- | ---------------- | | [investigations_visualisations.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/review/investigations_visualisations.ipynb) | Notebook to visualise qtransforms and skymaps of the interesting candidate real event pairs. | JRC: OK | --------- | ---------------- |
| [optimise_densenets.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/development/optimise_densenets.py) | Optimise densenet learning rates, with and without whitening of strain | JRC: OK |09e01c8972074ee4f44120afb9095ea76c458c9f | ---------------- | | [optimise_densenets.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/development/optimise_densenets.py) | Optimise densenet learning rates, with and without whitening of strain | JRC: OK | 09e01c8972074ee4f44120afb9095ea76c458c9f | ---------------- |
| [missing_strain_ML_qts.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/review/missing_strain_ML_qts.ipynb) | Implement XGBoost with QTs using imputed values instead of 1s(default) for the single/double detector real events. Additionally compare the results of PO and ML to Golum for the selected candidates. | JRC: 0K | 09e01c8972074ee4f44120afb9095ea76c458c9f | ---------------- | | [missing_strain_ML_qts.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/review/missing_strain_ML_qts.ipynb) | Implement XGBoost with QTs using imputed values instead of 1s(default) for the single/double detector real events. Additionally compare the results of PO and ML to Golum for the selected candidates. | JRC: 0K | 09e01c8972074ee4f44120afb9095ea76c458c9f | ---------------- |
## Results ## Results
...@@ -70,20 +68,20 @@ In progress: [page](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/wikis/O3-r ...@@ -70,20 +68,20 @@ In progress: [page](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/wikis/O3-r
| Item | Short description | Status | git hash | Comment | final sign-off | | Item | Short description | Status | git hash | Comment | final sign-off |
|------|-------------------|--------|----------|---------|----------------| |------|-------------------|--------|----------|---------|----------------|
| [Event list](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/a0c855f18e7435d977f1fed5a2e806368fde15d8/data_download_preparation/O3_events_updated_1feb2022.txt) | List of events considered for the O3 lensing analysis | OK-JRC-------- | 4e81d2bb94c38960e6e5d2e4104c94679a78bb67 | --------- | :heavy_check_mark: | | [Event list](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/a0c855f18e7435d977f1fed5a2e806368fde15d8/data_download_preparation/O3_events_updated_1feb2022.txt) | List of events considered for the O3 lensing analysis | OK-JRC-------- | 4e81d2bb94c38960e6e5d2e4104c94679a78bb67 | --------- | :heavy_check_mark: |
| [Final config](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/a0c855f18e7435d977f1fed5a2e806368fde15d8/config_O3_events_18022022.yaml) | Contains paths to the Qtransforms, Skymaps, background, machines used for producing the final results | OK-JRC-------- | 4e81d2bb94c38960e6e5d2e4104c94679a78bb67 | --------- |:heavy_check_mark: | | [Final config](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/a0c855f18e7435d977f1fed5a2e806368fde15d8/config_O3_events_18022022.yaml) | Contains paths to the Qtransforms, Skymaps, background, machines used for producing the final results | OK-JRC-------- | 4e81d2bb94c38960e6e5d2e4104c94679a78bb67 | --------- | :heavy_check_mark: |
| [Final results](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/09e01c8972074ee4f44120afb9095ea76c458c9f/O3_ML_gwtc3/results_21april_kaggle_re_pesky_18022022/ML_BLU_FPPs_imputed_missing_strains.csv) | Total 2415 pairs, 79 selected for Joint-PE follow up analysis by combine ML and PO pipelines | OK-JRC-------- | 09e01c8972074ee4f44120afb9095ea76c458c9f | --------- | :heavy_check_mark:| | [Final results](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/09e01c8972074ee4f44120afb9095ea76c458c9f/O3_ML_gwtc3/results_21april_kaggle_re_pesky_18022022/ML_BLU_FPPs_imputed_missing_strains.csv) | Total 2415 pairs, 79 selected for Joint-PE follow up analysis by combine ML and PO pipelines | OK-JRC-------- | 09e01c8972074ee4f44120afb9095ea76c458c9f | --------- | :heavy_check_mark: |
| [PO and ML plot](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/09e01c8972074ee4f44120afb9095ea76c458c9f/O3_ML_gwtc3/results_21april_kaggle_re_pesky_18022022/ML_BLU_scatter_hist_FPPs.pdf) | ML and PO False positive probabilities for all the O3 pairs. Threshold of FPP = 0.01 signifies the pairs considered for joint-pe analysis. | OK-JRC-------- | 09e01c8972074ee4f44120afb9095ea76c458c9f | --------- | :heavy_check_mark:| | [PO and ML plot](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/09e01c8972074ee4f44120afb9095ea76c458c9f/O3_ML_gwtc3/results_21april_kaggle_re_pesky_18022022/ML_BLU_scatter_hist_FPPs.pdf) | ML and PO False positive probabilities for all the O3 pairs. Threshold of FPP = 0.01 signifies the pairs considered for joint-pe analysis. | OK-JRC-------- | 09e01c8972074ee4f44120afb9095ea76c458c9f | --------- | :heavy_check_mark: |
## Additional: Targeted Sub-threshold Search ## Additional: Targeted Sub-threshold Search
| Script/Notebook | Short description | Status | git hash | Comment | final sign-off | | Script/Notebook | Short description | Status | git hash | Comment | final sign-off |
|-----------------|-------------------|--------|----------|---------|----------------| |-----------------|-------------------|--------|----------|---------|----------------|
| [search_24mar.dat](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/subthreshold/search_24mar.dat) | Subthreshold triggers found by GSTLAL targeted subthreshold search pipeline. These are analysed by ML for finding out the most interesting pairs. | JRC:OK |09e01c8972074ee4f44120afb9095ea76c458c9f| | :heavy_check_mark:| | [search_24mar.dat](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/subthreshold/search_24mar.dat) | Subthreshold triggers found by GSTLAL targeted subthreshold search pipeline. These are analysed by ML for finding out the most interesting pairs. | JRC:OK | 09e01c8972074ee4f44120afb9095ea76c458c9f | | :heavy_check_mark: |
| [sub_data_download_prep.py](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/subthreshold/sub_data_download_prep.py) | Subthreshold candidates data downloading and preparing Qtransforms and input sky features for ML analysis | JRC: OK |09e01c8972074ee4f44120afb9095ea76c458c9f | --------- | :heavy_check_mark: | | [sub_data_download_prep.py](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/subthreshold/sub_data_download_prep.py) | Subthreshold candidates data downloading and preparing Qtransforms and input sky features for ML analysis | JRC: OK | 09e01c8972074ee4f44120afb9095ea76c458c9f | --------- | :heavy_check_mark: |
| [config_O3_super_sub_events_24022022.yaml](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/subthreshold/configs/config_O3_super_sub_events_24022022.yaml) | Config used for producing the preliminary results with the `lensid_make_predictions` command line. | JRC: OK | 09e01c8972074ee4f44120afb9095ea76c458c9f | --------- | :heavy_check_mark: | | [config_O3_super_sub_events_24022022.yaml](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/subthreshold/configs/config_O3_super_sub_events_24022022.yaml) | Config used for producing the preliminary results with the `lensid_make_predictions` command line. | JRC: OK | 09e01c8972074ee4f44120afb9095ea76c458c9f | --------- | :heavy_check_mark: |
| [get_candidates_super_sub.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/subthreshold/get_candidates_super_sub.ipynb) | Notebook for loading the results and sorting candidates | JRC: OK |09e01c8972074ee4f44120afb9095ea76c458c9f | --------- | :heavy_check_mark: | | [get_candidates_super_sub.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/subthreshold/get_candidates_super_sub.ipynb) | Notebook for loading the results and sorting candidates | JRC: OK | 09e01c8972074ee4f44120afb9095ea76c458c9f | --------- | :heavy_check_mark: |
| [test_super_sub_pairs.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/subthreshold/test_super_sub_pairs.ipynb) | Notebook for testing the machines that are trained on super-super pairs on the super-sub pairs. The performance is okay, can be improved by retraining on super-sub dataset(after generating). | JRC: OK- | 09e01c8972074ee4f44120afb9095ea76c458c9f | --| :heavy_check_mark: | | [test_super_sub_pairs.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/subthreshold/test_super_sub_pairs.ipynb) | Notebook for testing the machines that are trained on super-super pairs on the super-sub pairs. The performance is okay, can be improved by retraining on super-sub dataset(after generating). | JRC: OK- | 09e01c8972074ee4f44120afb9095ea76c458c9f | -- | :heavy_check_mark: |
| [real_events_bhattu_comp.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/subthreshold/pe_match_mchirp/real_events_bhattu_comp.ipynb) | Notebook for checking the chirp mass mean and variance in PE v/s match filter using the Caliskan et.al formula for the superthreshold O3 events. | JRC: OK- | 09e01c8972074ee4f44120afb9095ea76c458c9f | --| :heavy_check_mark: | | [real_events_bhattu_comp.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/subthreshold/pe_match_mchirp/real_events_bhattu_comp.ipynb) | Notebook for checking the chirp mass mean and variance in PE v/s match filter using the Caliskan et.al formula for the superthreshold O3 events. | JRC: OK- | 09e01c8972074ee4f44120afb9095ea76c458c9f | -- | :heavy_check_mark: |
# Meetings # Meetings
## 3 December 2021 ## 3 December 2021
... ...
......