Changes

Page history
Update Code Review authored by Srashti Goyal's avatar Srashti Goyal
Show whitespace changes
Inline Side-by-side
Code-Review.md
View page @ a2b85203
...@@ -57,14 +57,19 @@ The ML model predictions are compared to posterior overlap statistic results als ...@@ -57,14 +57,19 @@ The ML model predictions are compared to posterior overlap statistic results als
[**Installation instructions**](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/wikis/Installation-instructions) [**Installation instructions**](https://git.ligo.org/srashti.goyal/strong-lensing-ml/-/wikis/Installation-instructions)
# Statement of Code Review : # Final Statement of Code Review
(to be added by Deep.)
## Final Sign-off (to be added by Deep)
### Reviewers ## Sign-off
(please put checkmarks and dates)
Deep Chatterjee: Jean-Rene: Git Hash: 493ea099f42fc50d2cc081754d5395f57fafae76
(please put :heavy_check_mark: ) and dates)
* [ ] Deep Chatterjee:
* [ ] Jean-Rene:
# Code review plan # Code review plan
...@@ -97,17 +102,17 @@ Deep Chatterjee: Jean-Rene: ...@@ -97,17 +102,17 @@ Deep Chatterjee: Jean-Rene:
| [lensid_create_unlensed_df.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_unlensed_df.py) | generates dataframe containing tags for pairs of unlensed simulated events, with columns as img_0, img_1 and Lensing(=0). Eg: `lensid_create_unlensed_df -odir check -outfile unlensed.csv -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz` | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- | | [lensid_create_unlensed_df.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_unlensed_df.py) | generates dataframe containing tags for pairs of unlensed simulated events, with columns as img_0, img_1 and Lensing(=0). Eg: `lensid_create_unlensed_df -odir check -outfile unlensed.csv -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz` | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- |
| [lensid_create_lensed_inj_xmls.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_lensed_inj_xmls.py) | helper script that outputs LAL inj.xml file for lensed simulated events given the injection parameters for bayestar. | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- | | [lensid_create_lensed_inj_xmls.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_lensed_inj_xmls.py) | helper script that outputs LAL inj.xml file for lensed simulated events given the injection parameters for bayestar. | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- |
| [lensid_create_unlensed_inj_xmls.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_unlensed_inj_xmls.py) | helper script that outputs LAL inj.xml file for unlensed simulated events given the injection parameters for bayestar. | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- | | [lensid_create_unlensed_inj_xmls.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_unlensed_inj_xmls.py) | helper script that outputs LAL inj.xml file for unlensed simulated events given the injection parameters for bayestar. | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- |
| [lensid_create_bayestar_sky_lensed_injs.sh](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/scripts/lensid_create_bayestar_sky_lensed_injs.sh) | generates bayestar skymaps(.fits) for lensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `lensid_create_bayestar_sky_lensed_injs.sh -o check -s 10 -n 3 -i ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz -p ~/lensid/data/PSDs/analytical_psd.xml` Note: if this does not work try running this before `export PATH=$HOME/.local/bin:$PATH` | OK-DC | cd113eaddc4a44336361af982852498aa605e7b0 | ------- | -------------- | | [lensid_create_bayestar_sky_lensed_injs.sh](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/scripts/lensid_create_bayestar_sky_lensed_injs.sh) | generates bayestar skymaps(.fits) for lensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `lensid_create_bayestar_sky_lensed_injs.sh -o check -s 10 -n 3 -i ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz -p ~/lensid/data/PSDs/analytical_psd.xml` Note: if this does not work try running this before `export PATH=$HOME/.local/bin:$PATH` | OK-DC | 493ea099f42fc50d2cc081754d5395f57fafae76 | ------- | -------------- |
| [lensid_create_bayestar_sky_unlensed_injs.sh](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/scripts/lensid_create_bayestar_sky_unlensed_injs.sh) | generates bayestar skymaps(.fits) for unlensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `lensid_create_bayestar_sky_unlensed_injs.sh -o check -s 10 -n 3 -i ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz -p ~/lensid/data/PSDs/analytical_psd.xml` | OK-DC | cd113eaddc4a44336361af982852498aa605e7b0 | ------- | -------------- | | [lensid_create_bayestar_sky_unlensed_injs.sh](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/scripts/lensid_create_bayestar_sky_unlensed_injs.sh) | generates bayestar skymaps(.fits) for unlensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `lensid_create_bayestar_sky_unlensed_injs.sh -o check -s 10 -n 3 -i ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz -p ~/lensid/data/PSDs/analytical_psd.xml` | OK-DC | 493ea099f42fc50d2cc081754d5395f57fafae76 | ------- | -------------- |
| [lensid_fits_to_cart.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/lensid_fits_to_cart.py) | helper script for converting HealPix skymap format(.fits) to cartesian. | OK-DC | ac95f97e0c7e8d584b68ed364f353a5ed4bbb12d | need sanity check for hp.cartview during results review | | | [lensid_fits_to_cart.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/lensid_fits_to_cart.py) | helper script for converting HealPix skymap format(.fits) to cartesian. | OK-DC | ac95f97e0c7e8d584b68ed364f353a5ed4bbb12d | need sanity check for hp.cartview during results review | |
| [lensid_sky_injs_cart.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_sky_injs_cart.py) | helper script for managing IO of fits_to_cart.py script for injection study | OK | ac95f97e0c7e8d584b68ed364f353a5ed4bbb12d | Issue created | -------------- | | [lensid_sky_injs_cart.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_sky_injs_cart.py) | helper script for managing IO of fits_to_cart.py script for injection study | OK | 493ea099f42fc50d2cc081754d5395f57fafae76 | OK | -------------- |
### Features extraction, Train/test/predict utilities ### Features extraction, Train/test/predict utilities
| Script | Short description | Status | git hash | Comment | final sign-off | | Script | Short description | Status | git hash | Comment | final sign-off |
|--------|-------------------|--------|----------|---------|----------------| |--------|-------------------|--------|----------|---------|----------------|
| [lensid_get_features_qts_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_qts_ml.py) | Script for calculating densenets output and other features from the Q-transforms images which go as input to "XGBoost with QTs model". Requires trained denset models for three detectors. Eg: `lensid_get_features_qts_ml -infile check/lensed.csv -outfile check/lensed_QTs.csv -dense_models_dir ~/lensid/saved_models/ -data_dir check` | -OK-DC, OK-jrc--- | f9b7075d0e6ca8db211a0c3e43299af1eb428410 | -------------- | -------------- | | [lensid_get_features_qts_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_qts_ml.py) | Script for calculating densenets output and other features from the Q-transforms images which go as input to "XGBoost with QTs model". Requires trained denset models for three detectors. Eg: `lensid_get_features_qts_ml -infile check/lensed.csv -outfile check/lensed_QTs.csv -dense_models_dir ~/lensid/saved_models/ -data_dir check` | -OK-DC, OK-jrc--- | f9b7075d0e6ca8db211a0c3e43299af1eb428410 | -------------- | -------------- |
| [lensid_get_features_sky_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_sky_ml.py) | Script for calculating features from the bayestar skymaps which go as input to "XGBoost with Skymaps" model. Eg: `lensid_get_features_sky_ml -infile check/lensed.csv -outfile check/lensed_sky.csv -data_dir check` | -OK-DC , OK-jrc | f9b7075d0e6ca8db211a0c3e43299af1eb428410 | -------------- | -------------- | | [lensid_get_features_sky_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_sky_ml.py) | Script for calculating features from the bayestar skymaps which go as input to "XGBoost with Skymaps" model. Eg: `lensid_get_features_sky_ml -infile check/lensed.csv -outfile check/lensed_sky.csv -data_dir check` | -OK-DC , OK-jrc | f9b7075d0e6ca8db211a0c3e43299af1eb428410 | -------------- | -------------- |
| [ml_utils.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/ml_utils.py) | utility script containing all machine learning model functions for training, FAP computation, predictions etc. | in progress-jrc | -------- | Why is leakyrelu loaded? The logic of rfn(epoch) could be simplified.------- | -------------- | | [ml_utils.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/ml_utils.py) | utility script containing all machine learning model functions for training, FAP computation, predictions etc. | OK | 493ea099f42fc50d2cc081754d5395f57fafae76 | Why is leakyrelu loaded? The logic of rfn(epoch) could be simplified.------- | -------------- |
### ML models: Training, Cross-validation, Optimisation, Testing, Comparison with BLU , Predictions ### ML models: Training, Cross-validation, Optimisation, Testing, Comparison with BLU , Predictions
| Scripts | Short description | Status | git hash | Comment | final sign-off | | Scripts | Short description | Status | git hash | Comment | final sign-off |
...@@ -138,10 +143,10 @@ Deep Chatterjee: Jean-Rene: ...@@ -138,10 +143,10 @@ Deep Chatterjee: Jean-Rene:
## Top-level scripts: Training and testing workflow. ## Top-level scripts: Training and testing workflow.
| Scripts | Short description | Status | git hash | Comment | final sign-off | | Scripts | Short description | Status | git hash | Comment | final sign-off |
|---------|-------------------|--------|----------|---------|----------------| |---------|-------------------|--------|----------|---------|----------------|
| [condor_data_gen_train_test.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/train_test_workflow/condor_data_gen_train_test.py) | Generate Qtransforms, Dataframes, Bayestar skymaps for training and testing given the injection parameters using condor dag jobs submission. Note: change `exec_file_loc` in the script according to your installation and `base_out_dir` as desired. | OK-jrc; OK-DC | a60740bb5a0cccb2be8e8184f16c0c7c93f8150b | --------- | ---------------- | | [condor_data_gen_train_test_config.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/train_test_workflow/condor_data_gen_train_test_config.py) | Generate Qtransforms, Dataframes, Bayestar skymaps for training and testing given the injection parameters using condor dag jobs submission. Note: change `exec_file_loc` in the script according to your installation and `base_out_dir` as desired. | OK-jrc; OK-DC | 493ea099f42fc50d2cc081754d5395f57fafae76 | --------- | ---------------- |
| [config_train_test_workflow.yaml](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/train_test_workflow/config_train_test_workflow.yaml) | config file for training and testing ML models. Note: change `base_out_dir` as desired. | OK-DC; question-jrc | | | | | [config_train_test_workflow.yaml](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/train_test_workflow/config_train_test_workflow.yaml) | config file for training and testing ML models. Note: change `base_out_dir` as desired. | OK-DC; question-jrc | | | |
| [train_three_densenets.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/train_test_workflow/train_three_densenets.py) | Trains the three densenets, needs **config** file as input. Runs very fast on GPU systems. Optionally one can use condor to submit it as job. `python train_three_densenets.py` | OK-DC; OK-jrc | a60740bb5a0cccb2be8e8184f16c0c7c93f8150b | | ---------------- | | [train_three_densenets.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/train_test_workflow/train_three_densenets.py) | Trains the three densenets, needs **config** file as input. Runs very fast on GPU systems. Optionally one can use condor to submit it as job. `python train_three_densenets.py` | OK-DC; OK-jrc | a60740bb5a0cccb2be8e8184f16c0c7c93f8150b | | ---------------- |
| [condor_train_test_features_extraction.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/train_test_workflow/condor_train_test_features_extraction.py) | Extract sky and qts features of the training and testing dataset by condor dag jobs submission. Needs **config** file as input. | OK-DC; OK-jrc | a60740bb5a0cccb2be8e8184f16c0c7c93f8150b | --------- | ---------------- | | [condor_train_test_features_extraction.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/train_test_workflow/condor_train_test_features_extraction.py) | Extract sky and qts features of the training and testing dataset by condor dag jobs submission. Needs **config** file as input. | OK-DC; OK-jrc | 493ea099f42fc50d2cc081754d5395f57fafae76 | --------- | ---------------- |
| [train_test_XGBs.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/train_test_workflow/train_test_XGBs.py) | Train and test the QTs and Skymaps machine learning models, optionally compare to BLU. Needs **config** as input. Runs in <5 mins, optionally can be submitted using condor. | OK-jrc; OK-DC | a60740bb5a0cccb2be8e8184f16c0c7c93f8150b | | ---------------- | | [train_test_XGBs.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/train_test_workflow/train_test_XGBs.py) | Train and test the QTs and Skymaps machine learning models, optionally compare to BLU. Needs **config** as input. Runs in <5 mins, optionally can be submitted using condor. | OK-jrc; OK-DC | a60740bb5a0cccb2be8e8184f16c0c7c93f8150b | | ---------------- |
# Meetings # Meetings
...@@ -417,5 +422,13 @@ Meeting ID: 860 7262 9011 Password: 001303 ...@@ -417,5 +422,13 @@ Meeting ID: 860 7262 9011 Password: 001303
* [ ] Add a section for the Code review sign off and send across an email to the committee heads. * [ ] Add a section for the Code review sign off and send across an email to the committee heads.
* [x] Create a google link for the future meetings, with JR. * [x] Create a google link for the future meetings, with JR.
## 17 November 2021
* Merge request approved by Deep and JR. Development branch merged.
* JR and Deep prepared the code review report, final code- review signoff is awaited.
### Action items:
* [ ] Look into skymap calculations, theta to declination conversion as JR pointed out.
* [ ] Produce a better version of results as soon as possible.
* [ ] Wrap up the code review.