| ... | ... | @@ -32,45 +32,49 @@ The ML model predictions are compared to posterior overlap statistic results als |
|
|
|
### Data preparation
|
|
|
|
| Script | Short description | Status | git hash | Comment | final sign-off |
|
|
|
|
| ------ | ----------------- | ------ | -------- | ------- | -------------- |
|
|
|
|
| [qt_utils.py](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/package/strong_lensing_ml/qt_utils.py) | helper script for injecting gaussian noise given a psd and waveform. Also plots and saves Qtransforms. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [create_qts_lensed_gaussian_noise_dataset.py](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/package/scripts/create_qts_lensed_gaussian_noise_dataset.py) | generates waveforms and q-transforms for simulated lensed events given a set of injection parameters, using analytical/O3a PSDs. Eg: `create_qts_lensed_gaussian_noise_dataset.py -odir check -start 10 -n 3 -infile ~/strong-lensing-ml/data/injection_pars/haris-et-al/lensed_inj_data.npz -psd_mode 1 -qrange 2 -mode 2`| ------ | -------- | ------- | -------------- |
|
|
|
|
| [create_qts_unlensed_gaussian_noise_dataset.py](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/package/scripts/create_qts_unlensed_gaussian_noise_dataset.py) | generates waveforms and q-transforms for simulated unlensed events given a set of injection parameters, using analytical/O3a PSDs. Eg: `create_qts_unlensed_gaussian_noise_dataset.py -odir check -start 10 -n 3 -infile ~/strong-lensing-ml/data/injection_pars/haris-et-al/unlensed_inj_data.npz -psd_mode 1 -qrange 2 -mode 2`| ------ | -------- | ------- | -------------- |
|
|
|
|
| [create_lensed_df.py](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/package/scripts/create_lensed_df.py) | generates dataframe containing tags for lensed simulated event pairs, with columns as img_0, img_1 and Lensing(=1). Eg: `create_lensed_df.py -odir check -outfile lensed.csv -start 10 -n 3 -infile ~/strong-lensing-ml/data/injection_pars/haris-et-al/lensed_inj_data.npz`| ------ | -------- | ------- | -------------- |
|
|
|
|
| [create_unlensed_df.py](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/package/scripts/create_unlensed_df.py) | generates dataframe containing tags for pairs of unlensed simulated events, with columns as img_0, img_1 and Lensing(=0). Eg: `create_unlensed_df.py -odir check -outfile unlensed.csv -start 10 -n 3 -infile ~/strong-lensing-ml/data/injection_pars/haris-et-al/unlensed_inj_data.npz` | ------ | -------- | ------- | -------------- |
|
|
|
|
| [create_lensed_inj_xmls.py](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/package/scripts/bayestar/create_lensed_inj_xmls.py) | helper script that outputs LAL inj.xml file for lensed simulated events given the injection parameters for bayestar. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [create_unlensed_inj_xmls.py](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/package/scripts/bayestar/create_unlensed_inj_xmls.py) | helper script that outputs LAL inj.xml file for unlensed simulated events given the injection parameters for bayestar. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [create_bayestar_sky_lensed_dataset.sh](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/package/scripts/bayestar/create_bayestar_sky_lensed_dataset.sh) | generates bayestar skymaps(.fits) for lensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `create_bayestar_sky_lensed_dataset.sh -o check -s 10 -n 3 -i ~/strong-lensing-ml/data/injection_pars/haris-et-al/lensed_inj_data.npz -p ~/strong-lensing-ml/data/PSDs/aligo_virgo_psd.xml`| ------ | -------- | ------- | -------------- |
|
|
|
|
| [create_bayestar_sky_unlensed_dataset.sh](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/package/scripts/bayestar/create_bayestar_sky_unlensed_dataset.sh) | generates bayestar skymaps(.fits) for unlensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `create_bayestar_sky_unlensed_dataset.sh -o check -s 10 -n 3 -i ~/strong-lensing-ml/data/injection_pars/haris-et-al/unlensed_inj_data.npz -p ~/strong-lensing-ml/data/PSDs/aligo_virgo_psd.xml`| ------ | -------- | ------- | -------------- |
|
|
|
|
| [fits_to_cart.py](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/package/scripts/bayestar/fits_to_cart.py) | helper script for converting HealPix skymap format(.fits) to cartesian. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [sky_injs_cart.py](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/package/scripts/bayestar/sky_injs_cart.py) | helper script for managing IO of fits_to_cart.py script for injection study | ------ | -------- | ------- | -------------- |
|
|
|
|
| [qt_utils.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/qt_utils.py) | helper script for injecting gaussian noise given a psd and waveform. Also plots and saves Qtransforms. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [lensid_create_qts_lensed_injs.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_qts_lensed_injs.py) | generates waveforms and q-transforms for simulated lensed events given a set of injection parameters, using analytical/O3a PSDs. Eg: `lensid_create_qts_lensed_injs -odir check -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz -psd_mode 1 -qrange 2 -mode 2`| ------ | -------- | ------- | -------------- |
|
|
|
|
| [lensid_create_qts_unlensed_injs.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_qts_unlensed_injs.py)| generates waveforms and q-transforms for simulated unlensed events given a set of injection parameters, using analytical/O3a PSDs. Eg: `lensid_create_qts_unlensed_injs -odir check -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz -psd_mode 1 -qrange 2 -mode 2`| ------ | -------- | ------- | -------------- |
|
|
|
|
| [lensid_create_lensed_df.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_lensed_df.py) | generates dataframe containing tags for lensed simulated event pairs, with columns as img_0, img_1 and Lensing(=1). Eg: `lensid_create_lensed_df -odir check -outfile lensed.csv -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz`| ------ | -------- | ------- | -------------- |
|
|
|
|
| [lensid_create_unlensed_df.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_unlensed_df.py) | generates dataframe containing tags for pairs of unlensed simulated events, with columns as img_0, img_1 and Lensing(=0). Eg: `lensid_create_unlensed_df -odir check -outfile unlensed.csv -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz` | ------ | -------- | ------- | -------------- |
|
|
|
|
| [lensid_create_lensed_inj_xmls.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_lensed_inj_xmls.py) | helper script that outputs LAL inj.xml file for lensed simulated events given the injection parameters for bayestar. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [lensid_create_unlensed_inj_xmls.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_unlensed_inj_xmls.py) | helper script that outputs LAL inj.xml file for unlensed simulated events given the injection parameters for bayestar. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [lensid_create_bayestar_sky_lensed_injs.sh](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/scripts/lensid_create_bayestar_sky_lensed_injs.sh) | generates bayestar skymaps(.fits) for lensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `lensid_create_bayestar_sky_lensed_injs.sh -o check -s 10 -n 3 -i ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz -p ~/lensid/data/PSDs/aligo_virgo_psd.xml` Note: if this does not work try running this before `export PATH=$HOME/.local/bin:$PATH`| ------ | -------- | ------- | -------------- |
|
|
|
|
| [lensid_create_bayestar_sky_unlensed_injs.sh](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/scripts/lensid_create_bayestar_sky_unlensed_injs.sh) | generates bayestar skymaps(.fits) for unlensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `lensid_create_bayestar_sky_unlensed_injs.sh -o check -s 10 -n 3 -i ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz -p ~/lensid/data/PSDs/aligo_virgo_psd.xml`| ------ | -------- | ------- | -------------- |
|
|
|
|
| [lensid_fits_to_cart.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/lensid_fits_to_cart.py) | helper script for converting HealPix skymap format(.fits) to cartesian. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [lensid_sky_injs_cart.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_sky_injs_cart.py) | helper script for managing IO of fits_to_cart.py script for injection study | ------ | -------- | ------- | -------------- |
|
|
|
|
|
|
|
|
### Features extraction, Train/test/predict utilities
|
|
|
|
| Script | Short description | Status | git hash | Comment | final sign-off |
|
|
|
|
| ------ | ----------------- | ------ | -------- | ------- | -------------- |
|
|
|
|
| [get_features_QTs_ML.py](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/package/scripts/get_features_QTs_ML.py) | Script for calculating densenets output and other features from the Q-transforms images which go as input to "XGBoost with QTs model". Requires trained denset models for three detectors. Eg: `get_features_QTs_ML.py -infile check/lensed.csv -outfile check/lensed_QTs.csv -dense_models_dir ~/strong-lensing-ml/saved_models/ -data_dir check/` | ------ | -------- | ------- | -------------- |
|
|
|
|
| [get_features_skymaps_ML.py](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/package/scripts/get_features_skymaps_ML.py) | Script for calculating features from the bayestar skymaps which go as input to "XGBoost with Skymaps" model. Eg: `get_features_skymaps_ML.py -infile check/lensed.csv -outfile check/lensed_sky.csv -data_dir check/` | ------ | -------- | ------- | -------------- |
|
|
|
|
| [ML_utils.py](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/package/strong_lensing_ml/ML_utils.py) | utility script containing all machine learning model functions for training, FAP computation, predictions etc. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [lensid_get_features_qts_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_qts_ml.py) | Script for calculating densenets output and other features from the Q-transforms images which go as input to "XGBoost with QTs model". Requires trained denset models for three detectors. Eg: `lensid_get_features_qts_ml -infile check/lensed.csv -outfile check/lensed_QTs.csv -dense_models_dir ~/lensid/saved_models/ -data_dir check` | ------ | -------- | ------- | -------------- |
|
|
|
|
| [lensid_get_features_sky_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_sky_ml.py) | Script for calculating features from the bayestar skymaps which go as input to "XGBoost with Skymaps" model. Eg: `lensid_get_features_sky_ml -infile check/lensed.csv -outfile check/lensed_sky.csv -data_dir check` | ------ | -------- | ------- | -------------- |
|
|
|
|
| [ml_utils.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/ml_utils.py) | utility script containing all machine learning model functions for training, FAP computation, predictions etc. | ------ | -------- | ------- | -------------- |
|
|
|
|
|
|
|
|
## ML models: Training, Cross-validation, Optimisation, Testing, Comparison with BLU, Investigations.
|
|
|
|
|
|
|
|
| Notebook | Short description | Status | git hash | Comment | final sign-off |
|
|
|
|
| ------ | ----------------- | ------ | -------- | ------- | -------------- |
|
|
|
|
| [train_densenets_QTs.ipynb](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/scripts/training_cv/train_densenets_QTs.ipynb) | Notebook that trains 3 DenseNets(CNN) for 3 detector Q-tranforms. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [train_crossvalidate_XGB_QTs.ipynb](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/scripts/training_cv/train_crossvalidate_XGB_QTs.ipynb) | Notebook that trains and cross-validates "XGBoost with QTs" model. Requires dataframe that already has the input features calculated from the Qtransform images and trained DenseNets. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [train_crossvalidate_XGB_sky.ipynb](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/scripts/training_cv/train_crossvalidate_XGB_sky.ipynb) | Notebook that trains and cross-validates "XGBoost with Skymaps" model. Requires dataframe that already has the input features calculated from the Bayestar/PE skymaps | ------ | -------- | ------- | -------------- |
|
|
|
|
| [test_QTs_ML.ipynb](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/scripts/testing_cv_calc_fpp/test_QTs_ML.ipynb) | Notebook that tests ML with Qtransforms and compare ROCs with $B^L_U$ statistic for the haris-et-al dataset. Requires trained "XGBoost with QTs" model and its input features, also requires $B^L_U$ values for comparison. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [test_skymaps_ML.ipynb](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/scripts/testing_cv_calc_fpp/test_skymaps_ML.ipynb) | Notebook that tests ML with Skymaps and compare ROCs with $B^L_U$ statistic for the haris-et-al dataset. Requires trained "XGBoost with Skymaps" model and its input features, also requires $B^L_U$ values for comparison. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [test_combined_ML_results.ipynb](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/scripts/testing_cv_calc_fpp/test_combined_ML_results.ipynb) | Notebook that combines individual ML outputs for Qtransforms and Skymaps, and compares the ROCs with $B^L_U$ statistic for the haris-et-al dataset. Also requires $B^L_U$ values for comparison. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [train_test_pars.ipynb](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/review/train_test_pars.ipynb) | Notebook having plots of injection parameters for training, testing and O3a sets that are used. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [ML_blu_FPPs_inj_pars_investigate.ipynb](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/review/ML_blu_FPPs_inj_pars_investigate.ipynb) | Notebook comparing the ML and BLU FPPs for each pair in test set, and also investigationg correlations with the injection parameters. Also contain statistics of input sky features for lensed and unlensed test pairs. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [PSD_plots.ipynb](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/review/PSD_plots.ipynb) | Notebook having plots of PSDs that are used. | ------ | -------- | ------- | -------------- |
|
|
|
|
|
|
|
|
## ML Predictions: Real events, Data preparation, FAP computation, Comparison with BLU
|
|
|
|
| [train_densenets_QTs.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/notebooks/training_cv/train_densenets_QTs.ipynb) | Notebook that trains 3 DenseNets(CNN) for 3 detector Q-tranforms. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [train_crossvalidate_XGB_QTs.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/notebooks/training_cv/train_crossvalidate_XGB_QTs.ipynb) | Notebook that trains and cross-validates "XGBoost with QTs" model. Requires dataframe that already has the input features calculated from the Qtransform images and trained DenseNets. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [train_crossvalidate_XGB_sky.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/notebooks/training_cv/train_crossvalidate_XGB_sky.ipynb) | Notebook that trains and cross-validates "XGBoost with Skymaps" model. Requires dataframe that already has the input features calculated from the Bayestar/PE skymaps | ------ | -------- | ------- | -------------- |
|
|
|
|
| [test_QTs_ML.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/notebooks/testing_cv_calc_fpp/test_QTs_ML.ipynb) | Notebook that tests ML with Qtransforms and compare ROCs with $B^L_U$ statistic for the haris-et-al dataset. Requires trained "XGBoost with QTs" model and its input features, also requires $B^L_U$ values for comparison. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [test_skymaps_ML.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/notebooks/testing_cv_calc_fpp/test_skymaps_ML.ipynb) | Notebook that tests ML with Skymaps and compare ROCs with $B^L_U$ statistic for the haris-et-al dataset. Requires trained "XGBoost with Skymaps" model and its input features, also requires $B^L_U$ values for comparison. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [test_combined_ML_results.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/notebooks/testing_cv_calc_fpp/test_combined_ML_results.ipynb) | Notebook that combines individual ML outputs for Qtransforms and Skymaps, and compares the ROCs with $B^L_U$ statistic for the haris-et-al dataset. Also requires $B^L_U$ values for comparison. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [train_test_pars.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/review/train_test_pars.ipynb) | Notebook having plots of injection parameters for training, testing and O3a sets that are used. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [ML_blu_FPPs_inj_pars_investigate.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/review/ML_blu_FPPs_inj_pars_investigate.ipynb) | Notebook comparing the ML and BLU FPPs for each pair in test set, and also investigationg correlations with the injection parameters. Also contain statistics of input sky features for lensed and unlensed test pairs. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [PSD_plots.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/review/PSD_plots.ipynb) | Notebook having plots of PSDs that are used. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [background_injections_ML_blu.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/scripts/O3a_events/background_injections_ML_blu.ipynb) | Notebook showing ML and BLU outputs for the background unlensed injections as simulated by Haris during O3a analysis. | ------ | -------- | ------- | -------------- |
|
|
|
|
|
|
|
|
## ML Predictions: O3 Real events, Data preparation, FAP computation, Comparison with BLU
|
|
|
|
### O3 analysis in git repo: [lensid-ml-o3](https://git.ligo.org/srashti.goyal/lensid-ml-o3)
|
|
|
|
|
|
|
|
| Notebook | Short description | Status | git hash | Comment | final sign-off |
|
|
|
|
| ------ | ----------------- | ------ | -------- | ------- | -------------- |
|
|
|
|
| [background_injections_ML_blu.ipynb](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/scripts/O3a_events/background_injections_ML_blu.ipynb) | Notebook showing ML and BLU outputs for the background unlensed injections as simulated by Haris during O3a analysis. | ------ | -------- | ------- | -------------- |
|
|
|
|
| [real_events_ML_BLU_FPP_results.ipynb](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/scripts/O3a_events/real_events_ML_BLU_FPP_results.ipynb) | Notebook showing ML and BLU False positive probabilities for the O3a real events. | ------ | -------- | ------- | -------------- |
|
|
|
|
| (download_data.ipynb)[https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/data_download_preparation/download_data.ipynb] | Notebook for downloading skymaps(.fits) from GraceDB and strain data from ligo servers using GWpy. | ------ | -------- | ------- | -------------- |
|
|
|
|
| (data_preparation.ipynb)[https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/data_download_preparation/data_preparation.ipynb] | Notebook for preparing Qtransform images, dataframes and skymaps for O3 real events. | ------ | -------- | ------- | -------------- |
|
|
|
|
|
|
|
|
| (ML_pred_O3.ipynb)[https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/O3_ML/ML_pred_O3.ipynb] | Notebook showing ML and BLU False positive probabilities for the O3 real events and comparison with BLU for O3a events. | ------ | -------- | ------- | -------------- |
|
|
|
|
|
|
|
|
|
|
|
|
|
| ... | ... | @@ -114,9 +118,9 @@ Password: 001303 |
|
|
|
- [x] diagnostics on training/testing set (lensed and unlensed) , ref: https://git.ligo.org/shaon.ghosh/EM_Bright_ML/-/tree/master/O2-HL-rates-injections
|
|
|
|
|
|
|
|
## 28 May 2021
|
|
|
|
- Discussed [Data_generation_process.ipynb](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/review/Data_generation_process.ipynb)
|
|
|
|
- Discussed [Data_generation_process.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/review/lensid_data_generation_process.ipynb)
|
|
|
|
|
|
|
|
- Discussed [train_test_pars.ipynb](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/review/train_test_pars.ipynb)
|
|
|
|
- Discussed [train_test_pars.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/review/train_test_pars.ipynb)
|
|
|
|
|
|
|
|
- We went through the data generation scripts along with the diagnostic plots for training and testing sets. We check the analytical PSDs used in Bayestar and in PyCBC are the same.
|
|
|
|
|
| ... | ... | @@ -193,9 +197,9 @@ Password: 001303 |
|
|
|
|
|
|
|
## 2 July 2021
|
|
|
|
|
|
|
|
- Discussed **O3a injections** with ML and BLU [notebook](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/scripts/O3a_events/background_injections_ML_blu.ipynb)
|
|
|
|
- Discussed **O3a injections** with ML and BLU [notebook](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/notebooks/O3a_events/background_injections_ML_blu.ipynb)
|
|
|
|
|
|
|
|
- Discussed **O3a real events** with ML and BLU [notebook](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/scripts/O3a_events/real_events_ML_BLU_FPP_results.ipynb)
|
|
|
|
- Discussed **O3a real events** with ML and BLU [notebook](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/scripts/O3a_events/real_events_ML_BLU_FPP_results.ipynb)
|
|
|
|
|
|
|
|
- Discussed O3a real events data download from GWOSC, bayestar, scripts, events selection and data preparation.
|
|
|
|
|
| ... | ... | @@ -218,13 +222,13 @@ Password: 001303 |
|
|
|
|
|
|
|
## 23 July 2021
|
|
|
|
|
|
|
|
- Installation instructions: [wiki](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/wikis/Installation-instructions)
|
|
|
|
- Installation instructions: [wiki](https://git.ligo.org/srashti.goyal/lensid/-/wikis/Installation-instructions)
|
|
|
|
|
|
|
|
- Recap and Discuss **O3a real events** with ML and BLU [notebook](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/scripts/O3a_events/real_events_ML_BLU_FPP_results.ipynb)
|
|
|
|
|
|
|
|
- Discussed O3a real events data download from GWOSC, bayestar, scripts, events selection and data preparation.
|
|
|
|
|
|
|
|
- Discuss **ML and BLU FPPs for test set** [notebook](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/blob/master/review/ML_blu_FPPs_inj_pars_investigate.ipynb)
|
|
|
|
- Discuss **ML and BLU FPPs for test set** [notebook](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/review/ML_blu_FPPs_inj_pars_investigate.ipynb)
|
|
|
|
|
|
|
|
- Discuss about training with O3a psd, optimising training size etc.
|
|
|
|
|
| ... | ... | @@ -245,7 +249,7 @@ Password: 001303 |
|
|
|
|
|
|
|
- Organisation of the git repositories and packaging stuff. A saparate repository for O3b/O3 events will be created and the current repository would be published as a strong lensing pipeline.
|
|
|
|
|
|
|
|
- Went through data download and preparation scripts for O3b/O3 events using gwpy and gracedb
|
|
|
|
- Went through data download and preparation scripts for O3b/O3 events using gwpy and gracedb. Notebooks can be found [here](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/data_download_preparation)
|
|
|
|
|
|
|
|
- We decided that for skymaps LAL> bayestar> subthreshold is the order of preference as per availability. Also we should eliminated cWB only events.
|
|
|
|
|
| ... | ... | @@ -261,9 +265,9 @@ Password: 001303 |
|
|
|
## 13 August 2021
|
|
|
|
- JR Tested the installation and added fix an issue of cloning. The installation seems to work fine.
|
|
|
|
|
|
|
|
- We went through the preliminary results of O3 event pairs.
|
|
|
|
- We went through the preliminary results of O3 event pairs. Notebook [here](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/O3_ML/ML_pred_O3.ipynb).
|
|
|
|
|
|
|
|
- JR suggested to have a name for the analysis.
|
|
|
|
- JR suggested to have a name for the analysis: LENSID is the consensus.
|
|
|
|
|
|
|
|
- JR and Deep agrees for trying out for super-sub pairs the analysis with current ML first and then check the performance. Also adding the figure of BLU FPP v/s ML FPP in the appendix of O3b lensing paper, seems reasonable.
|
|
|
|
|
| ... | ... | @@ -297,10 +301,10 @@ Password: 001303 |
|
|
|
|
|
|
|
- [ ] Investigate extreme events in the test set for the BLU & ML, by seeing their QTs and skymaps to understand the independent behaviour of ML & BLU.
|
|
|
|
|
|
|
|
- [ ] Use entry points, add unit tests etc. as mentioned (here)[https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/issues/1].
|
|
|
|
- [x] Use entry points as mentioned (here)[https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/issues/1].
|
|
|
|
|
|
|
|
- [ ] Think about integrating with igwn-py37.
|
|
|
|
|
|
|
|
- [ ] Try out the installation on igwn-py38.
|
|
|
|
|
|
|
|
- [ ] Create new repo for the package and package data, and probably move this review page also over there. |
|
|
\ No newline at end of file |
|
|
|
- [x] Create new repo for the package and package data, and probably move this review page also over there. |
|
|
\ No newline at end of file |
