Skip to content

Changes

Page history
Update Code Review authored by Deep Chatterjee's avatar Deep Chatterjee
Show whitespace changes
Inline Side-by-side
Code-Review.md
View page @ 538b70bb
...@@ -90,25 +90,25 @@ The ML model predictions are compared to posterior overlap statistic results als ...@@ -90,25 +90,25 @@ The ML model predictions are compared to posterior overlap statistic results als
| [lensid_create_unlensed_inj_xmls.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_unlensed_inj_xmls.py) | helper script that outputs LAL inj.xml file for unlensed simulated events given the injection parameters for bayestar. | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- | | [lensid_create_unlensed_inj_xmls.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_unlensed_inj_xmls.py) | helper script that outputs LAL inj.xml file for unlensed simulated events given the injection parameters for bayestar. | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- |
| [lensid_create_bayestar_sky_lensed_injs.sh](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/scripts/lensid_create_bayestar_sky_lensed_injs.sh) | generates bayestar skymaps(.fits) for lensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `lensid_create_bayestar_sky_lensed_injs.sh -o check -s 10 -n 3 -i ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz -p ~/lensid/data/PSDs/analytical_psd.xml` Note: if this does not work try running this before `export PATH=$HOME/.local/bin:$PATH` | Ongoing | -------- | Issue created | -------------- | | [lensid_create_bayestar_sky_lensed_injs.sh](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/scripts/lensid_create_bayestar_sky_lensed_injs.sh) | generates bayestar skymaps(.fits) for lensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `lensid_create_bayestar_sky_lensed_injs.sh -o check -s 10 -n 3 -i ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz -p ~/lensid/data/PSDs/analytical_psd.xml` Note: if this does not work try running this before `export PATH=$HOME/.local/bin:$PATH` | Ongoing | -------- | Issue created | -------------- |
| [lensid_create_bayestar_sky_unlensed_injs.sh](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/scripts/lensid_create_bayestar_sky_unlensed_injs.sh) | generates bayestar skymaps(.fits) for unlensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `lensid_create_bayestar_sky_unlensed_injs.sh -o check -s 10 -n 3 -i ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz -p ~/lensid/data/PSDs/analytical_psd.xml` | Ongoing | -------- | Issue created | -------------- | | [lensid_create_bayestar_sky_unlensed_injs.sh](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/scripts/lensid_create_bayestar_sky_unlensed_injs.sh) | generates bayestar skymaps(.fits) for unlensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `lensid_create_bayestar_sky_unlensed_injs.sh -o check -s 10 -n 3 -i ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz -p ~/lensid/data/PSDs/analytical_psd.xml` | Ongoing | -------- | Issue created | -------------- |
| [lensid_fits_to_cart.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/lensid_fits_to_cart.py) | helper script for converting HealPix skymap format(.fits) to cartesian. | Ongoing | -------- | Issue created | -------------- | | [lensid_fits_to_cart.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/lensid_fits_to_cart.py) | helper script for converting HealPix skymap format(.fits) to cartesian. | Ongoing; need sanity check for hp.cartview | -------- | Issue created | -------------- |
| [lensid_sky_injs_cart.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_sky_injs_cart.py) | helper script for managing IO of fits_to_cart.py script for injection study | Ongoing | -------- | Issue created | -------------- | | [lensid_sky_injs_cart.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_sky_injs_cart.py) | helper script for managing IO of fits_to_cart.py script for injection study | OK | ac95f97e0c7e8d584b68ed364f353a5ed4bbb12d | Issue created | -------------- |
### Features extraction, Train/test/predict utilities ### Features extraction, Train/test/predict utilities
| Script | Short description | Status | git hash | Comment | final sign-off | | Script | Short description | Status | git hash | Comment | final sign-off |
|--------|-------------------|--------|----------|---------|----------------| |--------|-------------------|--------|----------|---------|----------------|
| [lensid_get_features_qts_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_qts_ml.py) | Script for calculating densenets output and other features from the Q-transforms images which go as input to "XGBoost with QTs model". Requires trained denset models for three detectors. Eg: `lensid_get_features_qts_ml -infile check/lensed.csv -outfile check/lensed_QTs.csv -dense_models_dir ~/lensid/saved_models/ -data_dir check` | -OK, OK-jrc--- | -------- | ------- | -------------- | | [lensid_get_features_qts_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_qts_ml.py) | Script for calculating densenets output and other features from the Q-transforms images which go as input to "XGBoost with QTs model". Requires trained denset models for three detectors. Eg: `lensid_get_features_qts_ml -infile check/lensed.csv -outfile check/lensed_QTs.csv -dense_models_dir ~/lensid/saved_models/ -data_dir check` | -OK-DC, OK-jrc--- | f9b7075d0e6ca8db211a0c3e43299af1eb428410 | ------- | -------------- |
| [lensid_get_features_sky_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_sky_ml.py) | Script for calculating features from the bayestar skymaps which go as input to "XGBoost with Skymaps" model. Eg: `lensid_get_features_sky_ml -infile check/lensed.csv -outfile check/lensed_sky.csv -data_dir check` | -OK- , OK-jrc | -------- | ------- | -------------- | | [lensid_get_features_sky_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_sky_ml.py) | Script for calculating features from the bayestar skymaps which go as input to "XGBoost with Skymaps" model. Eg: `lensid_get_features_sky_ml -infile check/lensed.csv -outfile check/lensed_sky.csv -data_dir check` | -OK-DC , OK-jrc | f9b7075d0e6ca8db211a0c3e43299af1eb428410 | ------- | -------------- |
| [ml_utils.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/ml_utils.py) | utility script containing all machine learning model functions for training, FAP computation, predictions etc. | in progress-jrc | -------- | Why is leakyrelu loaded? The logic of rfn(epoch) could be simplified.------- | -------------- | | [ml_utils.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/ml_utils.py) | utility script containing all machine learning model functions for training, FAP computation, predictions etc. | in progress-jrc | -------- | Why is leakyrelu loaded? The logic of rfn(epoch) could be simplified.------- | -------------- |
### ML models: Training, Cross-validation, Optimisation, Testing, Comparison with BLU , Predictions ### ML models: Training, Cross-validation, Optimisation, Testing, Comparison with BLU , Predictions
| Scripts | Short description | Status | git hash | Comment | final sign-off | | Scripts | Short description | Status | git hash | Comment | final sign-off |
|---------|-------------------|--------|----------|---------|----------------| |---------|-------------------|--------|----------|---------|----------------|
| [train_densenets_qts.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/train_test/train_densenets_qts.py) | Train densenet with qtransform for a given detector. Eg: `python train_densenets_qts.py -lensed_df ~/strong-lensing-ml/data/dataframes/train/lensed.csv -unlensed_df ~/strong-lensing-ml/data/dataframes/train/unlensed_half.csv -odir dense_out/cit/ -epochs 10 -data_dir ~/alice_data_lensid/qts/train/`. Note: tensorflowenv on pcdev12@CIT has GPU | --OK-jrc | -------- | jrc: is it only on pcdev12? | ---------------- | | [train_densenets_qts.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/train_test/train_densenets_qts.py) | Train densenet with qtransform for a given detector. Eg: `python train_densenets_qts.py -lensed_df ~/strong-lensing-ml/data/dataframes/train/lensed.csv -unlensed_df ~/strong-lensing-ml/data/dataframes/train/unlensed_half.csv -odir dense_out/cit/ -epochs 10 -data_dir ~/alice_data_lensid/qts/train/`. Note: tensorflowenv on pcdev12@CIT has GPU | --OK-jrc | -------- | jrc: is it only on pcdev12? | ---------------- |
| [train_crossvalidate_test_XGB_qts.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/train_test/train_crossvalidate_test_XGB_qts.py) | Trains, cross-validate and compare to BLU "XGBoost with QTs" model. Requires dataframe that already has the input features calculated from the Qtransform images and trained DenseNets. `python train_crossvalidate_test_XGB_qts.py -help` | ------ | -------- | ------- | -------------- | | [train_crossvalidate_test_XGB_qts.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/train_test/train_crossvalidate_test_XGB_qts.py) | Trains, cross-validate and compare to BLU "XGBoost with QTs" model. Requires dataframe that already has the input features calculated from the Qtransform images and trained DenseNets. `python train_crossvalidate_test_XGB_qts.py -help` | ------ | -------- | reason for using `.sample(frac=1,...)` in various places? | -------------- |
| [train_crossvalidate_test_XGB_sky.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/train_test/train_crossvalidate_test_XGB_sky.py) | Train, cross-validates and compare to BLU "XGBoost with Skymaps" model. Requires dataframe that already has the input features calculated from the Bayestar/PE skymaps. `python train_crossvalidate_test_XGB_sky.py -help` | ------ | -------- | ------- | -------------- | | [train_crossvalidate_test_XGB_sky.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/train_test/train_crossvalidate_test_XGB_sky.py) | Train, cross-validates and compare to BLU "XGBoost with Skymaps" model. Requires dataframe that already has the input features calculated from the Bayestar/PE skymaps. `python train_crossvalidate_test_XGB_sky.py -help` | ------ | -------- | Cannot follow: <https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/train_test/test_combined_ML_results.py#L95-101> | -------------- |
| [test_combined_ML_results.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/train_test/test_combined_ML_results.py) | Test and compare to BLU overall ML model. Requires dataframes that already has the ML predictions calculated from the qts and skymaps. `python test_combined_ML_results.py -help` | ------ | -------- | ------- | -------------- | | [test_combined_ML_results.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/train_test/test_combined_ML_results.py) | Test and compare to BLU overall ML model. Requires dataframes that already has the ML predictions calculated from the qts and skymaps. `python test_combined_ML_results.py -help` | ------ | -------- | Same snippet as above | -------------- |
| [ml_predict_workflow.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/ml_predict_workflow.py), [config_O3_events.yaml](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/config_O3_events.yaml) | Script for computing ML predictions for a given event pairs in dataframe, their skymaps, Qtransforms and the trained ML models. Optionally computes the False Positive Probabilities given the background. It requires config file as input. Eg: `lensid_make_predictions -config /home/srashti.goyal/lensid-ml-O3/config_O3_events.yaml` Note: change `odir` in **config** file. | -------- | ---------- | --------- | ---------------- | | [ml_predict_workflow.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/ml_predict_workflow.py), [config_O3_events.yaml](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/config_O3_events.yaml) | Script for computing ML predictions for a given event pairs in dataframe, their skymaps, Qtransforms and the trained ML models. Optionally computes the False Positive Probabilities given the background. It requires config file as input. Eg: `lensid_make_predictions -config /home/srashti.goyal/lensid-ml-O3/config_O3_events.yaml` Note: change `odir` in **config** file. | -------- | ---------- | --------- | ---------------- |
## Investigations: Injection parameters, features statistics, PSDs etc. ## Investigations: Injection parameters, features statistics, PSDs etc.
| Notebook | Short description | Status | git hash | Comment | final sign-off | | Notebook | Short description | Status | git hash | Comment | final sign-off |
|----------|-------------------|--------|----------|---------|----------------| |----------|-------------------|--------|----------|---------|----------------|
...@@ -118,7 +118,6 @@ The ML model predictions are compared to posterior overlap statistic results als ...@@ -118,7 +118,6 @@ The ML model predictions are compared to posterior overlap statistic results als
| [background_injections_ML_blu.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/notebooks/O3a_events/background_injections_ML_blu.ipynb) | Notebook showing ML and BLU outputs for the background unlensed injections as simulated by Haris during O3a analysis. | ------ | -------- | ------- | -------------- | | [background_injections_ML_blu.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/notebooks/O3a_events/background_injections_ML_blu.ipynb) | Notebook showing ML and BLU outputs for the background unlensed injections as simulated by Haris during O3a analysis. | ------ | -------- | ------- | -------------- |
## ML Predictions: O3 Real events, Data preparation, FAP computation, Comparison with BLU ## ML Predictions: O3 Real events, Data preparation, FAP computation, Comparison with BLU
| Notebook/Scripts | Short description | Status | git hash | Comment | final sign-off | | Notebook/Scripts | Short description | Status | git hash | Comment | final sign-off |
|------------------|-------------------|--------|----------|---------|----------------| |------------------|-------------------|--------|----------|---------|----------------|
| [download_data.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/data_download_preparation/download_data.ipynb) | Notebook for downloading skymaps(.fits) from GraceDB and strain data from ligo servers using GWpy. | ------ | -------- | ------- | -------------- | | [download_data.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/data_download_preparation/download_data.ipynb) | Notebook for downloading skymaps(.fits) from GraceDB and strain data from ligo servers using GWpy. | ------ | -------- | ------- | -------------- |
...@@ -128,7 +127,6 @@ The ML model predictions are compared to posterior overlap statistic results als ...@@ -128,7 +127,6 @@ The ML model predictions are compared to posterior overlap statistic results als
| [investigations_visualisations.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/review/investigations_visualisations.ipynb) | Notebook to visualise qtransforms and skymaps of the interesting candidate real event pairs. | -------- | ---------- | --------- | ---------------- | | [investigations_visualisations.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/review/investigations_visualisations.ipynb) | Notebook to visualise qtransforms and skymaps of the interesting candidate real event pairs. | -------- | ---------- | --------- | ---------------- |
## Top-level scripts: Training and testing workflow. ## Top-level scripts: Training and testing workflow.
| Scripts | Short description | Status | git hash | Comment | final sign-off | | Scripts | Short description | Status | git hash | Comment | final sign-off |
|---------|-------------------|--------|----------|---------|----------------| |---------|-------------------|--------|----------|---------|----------------|
| [condor_data_gen_train_test.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/train_test_workflow/condor_data_gen_train_test.py) | Generate Qtransforms, Dataframes, Bayestar skymaps for training and testing given the injection parameters using condor dag jobs submission. Note: change `exec_file_loc` in the script according to your installation and `base_out_dir` as desired. | -------- | ---------- | --------- | ---------------- | | [condor_data_gen_train_test.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/train_test_workflow/condor_data_gen_train_test.py) | Generate Qtransforms, Dataframes, Bayestar skymaps for training and testing given the injection parameters using condor dag jobs submission. Note: change `exec_file_loc` in the script according to your installation and `base_out_dir` as desired. | -------- | ---------- | --------- | ---------------- |
...@@ -322,7 +320,6 @@ Meeting ID: 860 7262 9011 Password: 001303 ...@@ -322,7 +320,6 @@ Meeting ID: 860 7262 9011 Password: 001303
* [x] Prepare the pipeline to get the first set of results with the accepted event list. update event list. * [x] Prepare the pipeline to get the first set of results with the accepted event list. update event list.
* [x] Convert training/testing notebooks to scripts format wherever possible. * [x] Convert training/testing notebooks to scripts format wherever possible.
## 7 September 2021 ## 7 September 2021
* We discussed the data preparation process for real events and also which events would be finally selected based on O3b catalogue. Deep suggested to look for offline events to make sure that the info doesn't change. * We discussed the data preparation process for real events and also which events would be finally selected based on O3b catalogue. Deep suggested to look for offline events to make sure that the info doesn't change.
...@@ -347,7 +344,6 @@ Meeting ID: 860 7262 9011 Password: 001303 ...@@ -347,7 +344,6 @@ Meeting ID: 860 7262 9011 Password: 001303
* [x] Write a script and config for full ML predictions. * [x] Write a script and config for full ML predictions.
* [x] Update event list for O3 events and send to the lensing channel. * [x] Update event list for O3 events and send to the lensing channel.
## 5 October 2021 ## 5 October 2021
* We discussed about the code review and the scripts. * We discussed about the code review and the scripts.
...@@ -375,4 +371,5 @@ Meeting ID: 860 7262 9011 Password: 001303 ...@@ -375,4 +371,5 @@ Meeting ID: 860 7262 9011 Password: 001303
* From now on any changes to the code should go through a merge request. * From now on any changes to the code should go through a merge request.
### Action items: ### Action items:
* [x] Update code review page for the sign-offs of the updated scripts. * [x] Update code review page for the sign-offs of the updated scripts.
\ No newline at end of file