Skip to content

Changes

Page history
Update Code Review authored by Srashti Goyal's avatar Srashti Goyal
Show whitespace changes
Inline Side-by-side
Code-Review.md
View page @ c4cace18
...@@ -85,20 +85,13 @@ The ML model predictions are compared to posterior overlap statistic results als ...@@ -85,20 +85,13 @@ The ML model predictions are compared to posterior overlap statistic results als
### Data preparation ### Data preparation
| Script | Short description | Status | git hash | Comment | final sign-off | | Script | Short description | Status | git hash | Comment | final sign-off |
|--------|-------------------|--------|----------|---------|----------------| |--------|-------------------|--------|----------|---------|----------------|
| [qt_utils.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/qt_utils.py) | helper script for injecting gaussian noise given a psd and waveform. Also plots and saves Qtransforms. | Ongoing\ | [qt_utils.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/qt_utils.py) | helper script for injecting gaussian noise given a psd and waveform. Also plots and saves Qtransforms. | Ongoing, Ongoing-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | Remove hard-coded numbers | -------------- |
Ongoing-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | Remove hard-coded numbers | -------------- | | [lensid_create_qts_lensed_injs.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_qts_lensed_injs.py) | generates waveforms and q-transforms for simulated lensed events given a set of injection parameters, using analytical/O3a PSDs. Eg: `lensid_create_qts_lensed_injs -odir check -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz -psd_mode 1 -qrange 2 -mode 2` | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- |
| [lensid_create_qts_lensed_injs.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_qts_lensed_injs.py) | generates waveforms and q-transforms for simulated lensed events given a set of injection parameters, using analytical/O3a PSDs. Eg: `lensid_create_qts_lensed_injs -odir check -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz -psd_mode 1 -qrange 2 -mode 2` | OK\ | [lensid_create_qts_unlensed_injs.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_qts_unlensed_injs.py) | generates waveforms and q-transforms for simulated unlensed events given a set of injection parameters, using analytical/O3a PSDs. Eg: `lensid_create_qts_unlensed_injs -odir check -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz -psd_mode 1 -qrange 2 -mode 2` | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- |
OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- | | [lensid_create_lensed_df.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_lensed_df.py) | generates dataframe containing tags for lensed simulated event pairs, with columns as img_0, img_1 and Lensing(=1). Eg: `lensid_create_lensed_df -odir check -outfile lensed.csv -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz` | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- |
| [lensid_create_qts_unlensed_injs.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_qts_unlensed_injs.py) | generates waveforms and q-transforms for simulated unlensed events given a set of injection parameters, using analytical/O3a PSDs. Eg: `lensid_create_qts_unlensed_injs -odir check -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz -psd_mode 1 -qrange 2 -mode 2` | OK\ | [lensid_create_unlensed_df.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_unlensed_df.py) | generates dataframe containing tags for pairs of unlensed simulated events, with columns as img_0, img_1 and Lensing(=0). Eg: `lensid_create_unlensed_df -odir check -outfile unlensed.csv -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz` | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- |
OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- | | [lensid_create_lensed_inj_xmls.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_lensed_inj_xmls.py) | helper script that outputs LAL inj.xml file for lensed simulated events given the injection parameters for bayestar. | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- |
| [lensid_create_lensed_df.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_lensed_df.py) | generates dataframe containing tags for lensed simulated event pairs, with columns as img_0, img_1 and Lensing(=1). Eg: `lensid_create_lensed_df -odir check -outfile lensed.csv -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz` | OK\ | [lensid_create_unlensed_inj_xmls.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_unlensed_inj_xmls.py) | helper script that outputs LAL inj.xml file for unlensed simulated events given the injection parameters for bayestar. | OK, OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- |
OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- |
| [lensid_create_unlensed_df.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_unlensed_df.py) | generates dataframe containing tags for pairs of unlensed simulated events, with columns as img_0, img_1 and Lensing(=0). Eg: `lensid_create_unlensed_df -odir check -outfile unlensed.csv -start 10 -n 3 -infile ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz` | OK\
OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- |
| [lensid_create_lensed_inj_xmls.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_lensed_inj_xmls.py) | helper script that outputs LAL inj.xml file for lensed simulated events given the injection parameters for bayestar. | OK\
OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- |
| [lensid_create_unlensed_inj_xmls.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_unlensed_inj_xmls.py) | helper script that outputs LAL inj.xml file for unlensed simulated events given the injection parameters for bayestar. | OK\
OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- |
| [lensid_create_bayestar_sky_lensed_injs.sh](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/scripts/lensid_create_bayestar_sky_lensed_injs.sh) | generates bayestar skymaps(.fits) for lensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `lensid_create_bayestar_sky_lensed_injs.sh -o check -s 10 -n 3 -i ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz -p ~/lensid/data/PSDs/aligo_virgo_psd.xml` Note: if this does not work try running this before `export PATH=$HOME/.local/bin:$PATH` | ------ | -------- | ------- | -------------- | | [lensid_create_bayestar_sky_lensed_injs.sh](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/scripts/lensid_create_bayestar_sky_lensed_injs.sh) | generates bayestar skymaps(.fits) for lensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `lensid_create_bayestar_sky_lensed_injs.sh -o check -s 10 -n 3 -i ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz -p ~/lensid/data/PSDs/aligo_virgo_psd.xml` Note: if this does not work try running this before `export PATH=$HOME/.local/bin:$PATH` | ------ | -------- | ------- | -------------- |
| [lensid_create_bayestar_sky_unlensed_injs.sh](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/scripts/lensid_create_bayestar_sky_unlensed_injs.sh) | generates bayestar skymaps(.fits) for unlensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `lensid_create_bayestar_sky_unlensed_injs.sh -o check -s 10 -n 3 -i ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz -p ~/lensid/data/PSDs/aligo_virgo_psd.xml` | ------ | -------- | ------- | -------------- | | [lensid_create_bayestar_sky_unlensed_injs.sh](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/scripts/lensid_create_bayestar_sky_unlensed_injs.sh) | generates bayestar skymaps(.fits) for unlensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `lensid_create_bayestar_sky_unlensed_injs.sh -o check -s 10 -n 3 -i ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz -p ~/lensid/data/PSDs/aligo_virgo_psd.xml` | ------ | -------- | ------- | -------------- |
| [lensid_fits_to_cart.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/lensid_fits_to_cart.py) | helper script for converting HealPix skymap format(.fits) to cartesian. | ------ | -------- | ------- | -------------- | | [lensid_fits_to_cart.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/lensid_fits_to_cart.py) | helper script for converting HealPix skymap format(.fits) to cartesian. | ------ | -------- | ------- | -------------- |
...@@ -107,11 +100,9 @@ OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- | ...@@ -107,11 +100,9 @@ OK-jrc | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | ------- | -------------- |
### Features extraction, Train/test/predict utilities ### Features extraction, Train/test/predict utilities
| Script | Short description | Status | git hash | Comment | final sign-off | | Script | Short description | Status | git hash | Comment | final sign-off |
|--------|-------------------|--------|----------|---------|----------------| |--------|-------------------|--------|----------|---------|----------------|
| [lensid_get_features_qts_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_qts_ml.py) | Script for calculating densenets output and other features from the Q-transforms images which go as input to "XGBoost with QTs model". Requires trained denset models for three detectors. Eg: `lensid_get_features_qts_ml -infile check/lensed.csv -outfile check/lensed_QTs.csv -dense_models_dir ~/lensid/saved_models/ -data_dir check` | -OK\ | [lensid_get_features_qts_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_qts_ml.py) | Script for calculating densenets output and other features from the Q-transforms images which go as input to "XGBoost with QTs model". Requires trained denset models for three detectors. Eg: `lensid_get_features_qts_ml -infile check/lensed.csv -outfile check/lensed_QTs.csv -dense_models_dir ~/lensid/saved_models/ -data_dir check` | -OK, OK-jrc--- | -------- | ------- | -------------- |
OK-jrc--- | -------- | ------- | -------------- | | [lensid_get_features_sky_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_sky_ml.py) | Script for calculating features from the bayestar skymaps which go as input to "XGBoost with Skymaps" model. Eg: `lensid_get_features_sky_ml -infile check/lensed.csv -outfile check/lensed_sky.csv -data_dir check` | -OK- , OK-jrc | -------- | ------- | -------------- |
| [lensid_get_features_sky_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_sky_ml.py) | Script for calculating features from the bayestar skymaps which go as input to "XGBoost with Skymaps" model. Eg: `lensid_get_features_sky_ml -infile check/lensed.csv -outfile check/lensed_sky.csv -data_dir check` | -OK-\ | [ml_utils.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/ml_utils.py) | utility script containing all machine learning model functions for training, FAP computation, predictions etc. | in progress-jrc | -------- | arc: the fixed directory in densebet_input_matrix should be user-defined. Why is leakyrelu loaded? The logic of rfn(epoch) could be simplified.------- | -------------- |
OK-jrc | -------- | ------- | -------------- |
| [ml_utils.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/ml_utils.py) | utility script containing all machine learning model functions for training, FAP computation, predictions etc. | in progress-jrc----- | -------- | arc: the fixed directory in densebet_input_matrix should be user-defined. Why is leakyrelu loaded? The logic of rfn(epoch) could be simplified.------- | -------------- |
## ML models: Training, Cross-validation, Optimisation, Testing, Comparison with BLU. ## ML models: Training, Cross-validation, Optimisation, Testing, Comparison with BLU.
| Scripts | Short description | Status | git hash | Comment | final sign-off | | Scripts | Short description | Status | git hash | Comment | final sign-off |
...@@ -138,6 +129,11 @@ OK-jrc | -------- | ------- | -------------- | ...@@ -138,6 +129,11 @@ OK-jrc | -------- | ------- | -------------- |
| [data_prepare.py](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/data_download_preparation/data_prepare.py) | Preparing Qtransform images, dataframes and skymaps for O3 real events given the event list and raw data downloaded from LIGO servers. Note: raw data on CIT is in `/home/srashti.goyal/lensid-ml-o3/data_download_preparation/O3` | ------ | -------- | ------- | -------------- | | [data_prepare.py](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/data_download_preparation/data_prepare.py) | Preparing Qtransform images, dataframes and skymaps for O3 real events given the event list and raw data downloaded from LIGO servers. Note: raw data on CIT is in `/home/srashti.goyal/lensid-ml-o3/data_download_preparation/O3` | ------ | -------- | ------- | -------------- |
| [ml_predict_workflow.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/scripts/ml_predict_workflow.py), [config_O3_events.yaml](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/scripts/config_O3_events.yaml) | Script for computing ML predictions for a given event pairs in dataframe, their skymaps, Qtransforms and the trained ML models. Optionally computes the False Positive Probabilities given the background. It requires config file as input. Eg: `python ml_predict_workflow.py -config config_O3_events.yaml` Note: change `odir` in config file. | -------- | ---------- | --------- | ---------------- | | [ml_predict_workflow.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/scripts/ml_predict_workflow.py), [config_O3_events.yaml](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/scripts/config_O3_events.yaml) | Script for computing ML predictions for a given event pairs in dataframe, their skymaps, Qtransforms and the trained ML models. Optionally computes the False Positive Probabilities given the background. It requires config file as input. Eg: `python ml_predict_workflow.py -config config_O3_events.yaml` Note: change `odir` in config file. | -------- | ---------- | --------- | ---------------- |
| [get_candidates_compare_to_blu.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/O3_ML_gwtc3/get_candidates_compare_to_blu.ipynb) | Notebook for comparing ML and BLU results for the full O3 catalogue of BBHs. | -------- | ---------- | --------- | ---------------- | | [get_candidates_compare_to_blu.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/O3_ML_gwtc3/get_candidates_compare_to_blu.ipynb) | Notebook for comparing ML and BLU results for the full O3 catalogue of BBHs. | -------- | ---------- | --------- | ---------------- |
|[investigations_visualisations.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/review/investigations_visualisations.ipynb)|Notebook to visualise qtransforms and skymaps of the interesting candidate real event pairs.|--------|----------|---------|----------------|
## Top-level scripts: Training, Cross-validation, Optimisation, Testing, Comparison with BLU.
| Scripts | Short description | Status | git hash | Comment | final sign-off |
|---------|-------------------|--------|----------|---------|----------------|
# Meetings # Meetings
...@@ -324,7 +320,6 @@ Meeting ID: 860 7262 9011 Password: 001303 ...@@ -324,7 +320,6 @@ Meeting ID: 860 7262 9011 Password: 001303
* [x] Prepare the pipeline to get the first set of results with the accepted event list. update event list. * [x] Prepare the pipeline to get the first set of results with the accepted event list. update event list.
* [x] Convert training/testing notebooks to scripts format wherever possible. * [x] Convert training/testing notebooks to scripts format wherever possible.
* [ ] Reproduce all the results on CIT and retrain the densenets with the help of Hari.
* [ ] retrain densenets using whitened QTs, wider qrange etc, XGB with QTs with missing data etc. * [ ] retrain densenets using whitened QTs, wider qrange etc, XGB with QTs with missing data etc.
* [ ] increase training set size, check convergence. * [ ] increase training set size, check convergence.
* [ ] Prepare injection set for super-sub pairs. Lensed and Unlensed. * [ ] Prepare injection set for super-sub pairs. Lensed and Unlensed.
...@@ -339,7 +334,7 @@ Meeting ID: 860 7262 9011 Password: 001303 ...@@ -339,7 +334,7 @@ Meeting ID: 860 7262 9011 Password: 001303
* [ ] Ask/check if S190413ac made it to the catalogue being an High far event(1e-4). * [ ] Ask/check if S190413ac made it to the catalogue being an High far event(1e-4).
* [x] Organise and send the event list to Justin etc. as a preliminary set of results. * [x] Organise and send the event list to Justin etc. as a preliminary set of results.
* [ ] Write notebook to visually inspect the candidate real event pairs in O3. * [x] Write notebook to visually inspect the candidate real event pairs in O3.
* [ ] Gather injection parameters for O3b background from Apratim. * [ ] Gather injection parameters for O3b background from Apratim.
## 14 September 2021 ## 14 September 2021
......