Result-Review-O3b-lensing.md

@@ -17,10 +17,12 @@ This repository deals with the result review of the machine learning based pipel
 
 | Script | Short description | Status | git hash | Comment | final sign-off |
 |--------|-------------------|--------|----------|---------|----------------|
+| [data_download.py](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/data_download_preparation/data_download.py) | Script for downloading events info, skymaps(.fits) from GraceDB and strain data from ligo servers using GWpy. Note: needs valid ligo key path(line 26) for accessing non-public data in Gracedb., eg: `/tmp/x509up*` | DC: OK | 2e3215024a42c08081d612c3713ffd54fbba5f7e | ------- | -------------- |
+| [data_prepare.py](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/data_download_preparation/data_prepare.py) | Preparing Qtransform images, dataframes and skymaps for O3 real events given the event list and raw data downloaded from LIGO servers. Note: raw data on CIT is in `/home/srashti.goyal/lensid-ml-o3/data_download_preparation/O3` | DC: OK | 2e3215024a42c08081d612c3713ffd54fbba5f7e | --- | -------------- |
 | [ml_predict_workflow.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/ml_predict_workflow.py), [config_O3_events.yaml](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/config_O3_events.yaml) | Script for computing ML predictions for a given event pairs in dataframe, their skymaps, Qtransforms and the trained ML models. Optionally computes the False Positive Probabilities given the background. It requires config file as input. Eg: `lensid_make_predictions -config /home/srashti.goyal/lensid-ml-O3/config_O3_events.yaml` Note: change `odir` in **config** file. | -------- | ---------- | DC: Should be considered in the results review | ---------------- |
 | [config_train_test_workflow.yaml](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/train_test_workflow/config_train_test_workflow.yaml) | config file for training and testing ML models. Note: change `base_out_dir` as desired. | OK-DC; question-jrc |  |  |  |
 | [get_candidates_compare_to_blu_tagged.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/O3_ML_gwtc3/get_candidates_compare_to_blu_tagged.ipynb) | Notebook for comparing ML and BLU results for the full O3 catalogue of BBHs. | -------- | ---------- | --------- | ---------------- |
-| [condor_lensid_make_predictions.py](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/condor_lensid_make_predictions.py), [config_O3_events.yaml](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/config_O3_events.yaml) | Condor Script for computing ML predictions and FPPs using, [ml_predict_workflow.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/ml_predict_workflow.py). Eg: `python condor_lensid_make_predictions` Note: change `exec_file_loc` in the script according to your installation and `odir` in config file. | OK-DC | ca47c5ce71fa7405b84c944235a8646abcd216d4 | --------- | ---------------- |
+| [condor_lensid_make_predictions.py](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/condor_lensid_make_predictions.py) | Condor Script for computing ML predictions and FPPs using, [ml_predict_workflow.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/ml_predict_workflow.py). Eg: `python condor_lensid_make_predictions` Note: change `exec_file_loc` in the script according to your installation and `odir` in config file. | OK-DC | ca47c5ce71fa7405b84c944235a8646abcd216d4 | --------- | ---------------- |
 
 
 ## Investigations
@@ -33,7 +35,7 @@ This repository deals with the result review of the machine learning based pipel
 | [background_injections_ML_blu.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/notebooks/O3a_events/background_injections_ML_blu.ipynb) | Notebook showing ML and BLU outputs for the background unlensed injections as simulated by Haris during O3a analysis. | ------ | -------- | ------- | -------------- |
 | [investigations_visualisations.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/review/investigations_visualisations.ipynb) | Notebook to visualise qtransforms and skymaps of the interesting candidate real event pairs. | -------- | Should this be reviewed? | --------- | ---------------- |
 | [comparison_with_golum.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/review/comparison_with_golum.ipynb) | Notebook to compare the Golum's results with lensid and posterior overlap FPPs for the candidate pairs in O3 | -------- | ------- | --------- | ---------------- |
-| [optimise_densenets.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/development/optimise_densenets.py) [notebook] | Optimise densenet learning rates, with and without whitening of strain | -------- | ------- | --------- | ---------------- |
+| [optimise_densenets.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/development/optimise_densenets.py)  | Optimise densenet learning rates, with and without whitening of strain | -------- | ------- | --------- | ---------------- |
 |[missing_strain_ML_qts.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/review/missing_strain_ML_qts.ipynb) | Implement XGBoost with QTs using imputed values instead of 1s(default) for the single/double detector real events. | -------- | ------- | --------- | ---------------- |
 
 ## Results