Changes

Page history
Update Code Review authored by Srashti Goyal's avatar Srashti Goyal
Show whitespace changes
Inline Side-by-side
Code-Review.md
View page @ bf3e2fd1
...@@ -85,7 +85,7 @@ The ML model predictions are compared to posterior overlap statistic results als ...@@ -85,7 +85,7 @@ The ML model predictions are compared to posterior overlap statistic results als
### Data preparation ### Data preparation
| Script | Short description | Status | git hash | Comment | final sign-off | | Script | Short description | Status | git hash | Comment | final sign-off |
|--------|-------------------|--------|----------|---------|----------------| |--------|-------------------|--------|----------|---------|----------------|
| [qt_utils.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/qt_utils.py) | helper script for injecting gaussian noise given a psd and waveform. Also plots and saves Qtransforms. | Ongoing, Ongoing-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | Remove hard-coded numbers | -------------- | | [qt_utils.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/qt_utils.py) | helper script for injecting gaussian noise given a psd and waveform. Also plots and saves Qtransforms. | Ongoing, Ongoing-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | OK, OK-jrc | -------------- |
| [lensid_create_qts_lensed_injs.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_qts_lensed_injs.py) | generates waveforms and q-transforms for simulated lensed events given a set of injection parameters, using analytical/O3a PSDs. Eg: `lensid_create_qts_lensed_injs -odir check -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz -psd_mode 1 -qrange 2 -mode 2` | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- | | [lensid_create_qts_lensed_injs.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_qts_lensed_injs.py) | generates waveforms and q-transforms for simulated lensed events given a set of injection parameters, using analytical/O3a PSDs. Eg: `lensid_create_qts_lensed_injs -odir check -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz -psd_mode 1 -qrange 2 -mode 2` | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- |
| [lensid_create_qts_unlensed_injs.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_qts_unlensed_injs.py) | generates waveforms and q-transforms for simulated unlensed events given a set of injection parameters, using analytical/O3a PSDs. Eg: `lensid_create_qts_unlensed_injs -odir check -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz -psd_mode 1 -qrange 2 -mode 2` | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- | | [lensid_create_qts_unlensed_injs.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_qts_unlensed_injs.py) | generates waveforms and q-transforms for simulated unlensed events given a set of injection parameters, using analytical/O3a PSDs. Eg: `lensid_create_qts_unlensed_injs -odir check -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz -psd_mode 1 -qrange 2 -mode 2` | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- |
| [lensid_create_lensed_df.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_lensed_df.py) | generates dataframe containing tags for lensed simulated event pairs, with columns as img_0, img_1 and Lensing(=1). Eg: `lensid_create_lensed_df -odir check -outfile lensed.csv -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz` | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- | | [lensid_create_lensed_df.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_lensed_df.py) | generates dataframe containing tags for lensed simulated event pairs, with columns as img_0, img_1 and Lensing(=1). Eg: `lensid_create_lensed_df -odir check -outfile lensed.csv -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz` | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- |
...@@ -102,7 +102,7 @@ The ML model predictions are compared to posterior overlap statistic results als ...@@ -102,7 +102,7 @@ The ML model predictions are compared to posterior overlap statistic results als
|--------|-------------------|--------|----------|---------|----------------| |--------|-------------------|--------|----------|---------|----------------|
| [lensid_get_features_qts_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_qts_ml.py) | Script for calculating densenets output and other features from the Q-transforms images which go as input to "XGBoost with QTs model". Requires trained denset models for three detectors. Eg: `lensid_get_features_qts_ml -infile check/lensed.csv -outfile check/lensed_QTs.csv -dense_models_dir ~/lensid/saved_models/ -data_dir check` | -OK, OK-jrc--- | -------- | ------- | -------------- | | [lensid_get_features_qts_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_qts_ml.py) | Script for calculating densenets output and other features from the Q-transforms images which go as input to "XGBoost with QTs model". Requires trained denset models for three detectors. Eg: `lensid_get_features_qts_ml -infile check/lensed.csv -outfile check/lensed_QTs.csv -dense_models_dir ~/lensid/saved_models/ -data_dir check` | -OK, OK-jrc--- | -------- | ------- | -------------- |
| [lensid_get_features_sky_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_sky_ml.py) | Script for calculating features from the bayestar skymaps which go as input to "XGBoost with Skymaps" model. Eg: `lensid_get_features_sky_ml -infile check/lensed.csv -outfile check/lensed_sky.csv -data_dir check` | -OK- , OK-jrc | -------- | ------- | -------------- | | [lensid_get_features_sky_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_sky_ml.py) | Script for calculating features from the bayestar skymaps which go as input to "XGBoost with Skymaps" model. Eg: `lensid_get_features_sky_ml -infile check/lensed.csv -outfile check/lensed_sky.csv -data_dir check` | -OK- , OK-jrc | -------- | ------- | -------------- |
| [ml_utils.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/ml_utils.py) | utility script containing all machine learning model functions for training, FAP computation, predictions etc. | in progress-jrc | -------- | arc: the fixed directory in densebet_input_matrix should be user-defined. Why is leakyrelu loaded? The logic of rfn(epoch) could be simplified.------- | -------------- | | [ml_utils.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/ml_utils.py) | utility script containing all machine learning model functions for training, FAP computation, predictions etc. | in progress-jrc | -------- | Why is leakyrelu loaded? The logic of rfn(epoch) could be simplified.------- | -------------- |
## ML models: Training, Cross-validation, Optimisation, Testing, Comparison with BLU. ## ML models: Training, Cross-validation, Optimisation, Testing, Comparison with BLU.
| Scripts | Short description | Status | git hash | Comment | final sign-off | | Scripts | Short description | Status | git hash | Comment | final sign-off |
...@@ -237,7 +237,6 @@ Meeting ID: 860 7262 9011 Password: 001303 ...@@ -237,7 +237,6 @@ Meeting ID: 860 7262 9011 Password: 001303
* [x] Investigate high FPP region for ML v/s BLU, discard high FPP in QT ROC plots. * [x] Investigate high FPP region for ML v/s BLU, discard high FPP in QT ROC plots.
* [ ] Compare O3a psd trained and analytical PSD trained machines with O3a injections test set. Also do whitening. * [ ] Compare O3a psd trained and analytical PSD trained machines with O3a injections test set. Also do whitening.
* [x] Test ML with Densenets trained on Kaggle. * [x] Test ML with Densenets trained on Kaggle.
* [ ] Write separate wiki pages for running data generation, training, testing, downloading O3a/O3b real events, predicting scripts.
## 2 July 2021 ## 2 July 2021
...@@ -325,9 +324,7 @@ Meeting ID: 860 7262 9011 Password: 001303 ...@@ -325,9 +324,7 @@ Meeting ID: 860 7262 9011 Password: 001303
* [x] Prepare the pipeline to get the first set of results with the accepted event list. update event list. * [x] Prepare the pipeline to get the first set of results with the accepted event list. update event list.
* [x] Convert training/testing notebooks to scripts format wherever possible. * [x] Convert training/testing notebooks to scripts format wherever possible.
* [ ] retrain densenets using whitened QTs, wider qrange etc, XGB with QTs with missing data etc.
* [ ] increase training set size, check convergence.
* [ ] Prepare injection set for super-sub pairs. Lensed and Unlensed.
## 7 September 2021 ## 7 September 2021
...@@ -340,7 +337,6 @@ Meeting ID: 860 7262 9011 Password: 001303 ...@@ -340,7 +337,6 @@ Meeting ID: 860 7262 9011 Password: 001303
* [ ] Ask/check if S190413ac made it to the catalogue being an High far event(1e-4). * [ ] Ask/check if S190413ac made it to the catalogue being an High far event(1e-4).
* [x] Organise and send the event list to Justin etc. as a preliminary set of results. * [x] Organise and send the event list to Justin etc. as a preliminary set of results.
* [x] Write notebook to visually inspect the candidate real event pairs in O3. * [x] Write notebook to visually inspect the candidate real event pairs in O3.
* [ ] Gather injection parameters for O3b background from Apratim.
## 14 September 2021 ## 14 September 2021
...@@ -353,6 +349,23 @@ Meeting ID: 860 7262 9011 Password: 001303 ...@@ -353,6 +349,23 @@ Meeting ID: 860 7262 9011 Password: 001303
* [x] Generate dataset from scratch on CIT. * [x] Generate dataset from scratch on CIT.
* [x] Write a script and config for full ML predictions. * [x] Write a script and config for full ML predictions.
* [x] Update event list for O3 events and send to the lensing channel. * [x] Update event list for O3 events and send to the lensing channel.
## 5 October 2021
* We discussed about the code review and the scripts.
* Deep and JR both suggested to have a condor dag submission routine for the overall workflow.
* We hope to have a sign-off in a week or two.
* Deep suggested to have a merge request to change the reviewed scripts.
* JR suggested to meet every week for efficient review.
### Action items:
* [ ] Check off all the comments and issues in the review table.
* [ ] retrain densenets using whitened QTs, wider qrange etc, XGB with QTs with missing data etc.
* [ ] increase training set size, check convergence.
* [ ] Prepare injection set for super-sub pairs. Lensed and Unlensed.
* [ ] Write scripts for submitting condor dags.
* [ ] get posterior paths and plot corners for the event pairs. * [ ] get posterior paths and plot corners for the event pairs.
* [ ] Ask lensing group for the O3b representative psd. * [ ] Ask lensing group for the O3b representative psd.
* [ ] Work on ML 2.0 * [ ] Gather injection parameters for O3b background from Apratim.
\ No newline at end of file