... | ... | @@ -105,6 +105,9 @@ The ML model predictions are compared to posterior overlap statistic results als |
|
|
| [ml_utils.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/ml_utils.py) | utility script containing all machine learning model functions for training, FAP computation, predictions etc. | in progress-jrc | -------- | Why is leakyrelu loaded? The logic of rfn(epoch) could be simplified.------- | -------------- |
|
|
|
|
|
|
## ML models: Training, Cross-validation, Optimisation, Testing, Comparison with BLU.
|
|
|
|
|
|
__Currently located in lensid/scripts. Shall I add these scripts into the package itself?__
|
|
|
|
|
|
| Scripts | Short description | Status | git hash | Comment | final sign-off |
|
|
|
|---------|-------------------|--------|----------|---------|----------------|
|
|
|
| [train_densenets_qts.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/scripts/train_densenets_qts.py) | Train densenet with qtransform for a given detector. Eg: `python train_densenets_qts.py -lensed_df ~/strong-lensing-ml/data/dataframes/train/lensed.csv -unlensed_df ~/strong-lensing-ml/data/dataframes/train/unlensed_half.csv -odir dense_out/cit/ -epochs 10 -data_dir ~/alice_data_lensid/qts/train/`. Note: tensorflowenv on pcdev12@CIT has GPU | --OK-jrc | -------- | jrc: is it only on pcdev12? | ---------------- |
|
... | ... | @@ -127,11 +130,15 @@ The ML model predictions are compared to posterior overlap statistic results als |
|
|
|------------------|-------------------|--------|----------|---------|----------------|
|
|
|
| [download_data.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/data_download_preparation/download_data.ipynb) | Notebook for downloading skymaps(.fits) from GraceDB and strain data from ligo servers using GWpy. | ------ | -------- | ------- | -------------- |
|
|
|
| [data_prepare.py](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/data_download_preparation/data_prepare.py) | Preparing Qtransform images, dataframes and skymaps for O3 real events given the event list and raw data downloaded from LIGO servers. Note: raw data on CIT is in `/home/srashti.goyal/lensid-ml-o3/data_download_preparation/O3` | ------ | -------- | ------- | -------------- |
|
|
|
| [ml_predict_workflow.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/scripts/ml_predict_workflow.py), [config_O3_events.yaml](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/scripts/config_O3_events.yaml) | Script for computing ML predictions for a given event pairs in dataframe, their skymaps, Qtransforms and the trained ML models. Optionally computes the False Positive Probabilities given the background. It requires config file as input. Eg: `python ml_predict_workflow.py -config config_O3_events.yaml` Note: change `odir` in config file. | -------- | ---------- | --------- | ---------------- |
|
|
|
|__In preparation__
|
|
|
[ml_predict_workflow.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/scripts/ml_predict_workflow.py), [config_O3_events.yaml](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/scripts/config_O3_events.yaml) | Script for computing ML predictions for a given event pairs in dataframe, their skymaps, Qtransforms and the trained ML models. Optionally computes the False Positive Probabilities given the background. It requires config file as input. Eg: `python ml_predict_workflow.py -config config_O3_events.yaml` Note: change `odir` in config file. | -------- | ---------- | --------- | ---------------- |
|
|
|
| [get_candidates_compare_to_blu.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/O3_ML_gwtc3/get_candidates_compare_to_blu.ipynb) | Notebook for comparing ML and BLU results for the full O3 catalogue of BBHs. | -------- | ---------- | --------- | ---------------- |
|
|
|
|[investigations_visualisations.ipynb](https://git.ligo.org/srashti.goyal/lensid-ml-o3/-/blob/master/review/investigations_visualisations.ipynb)|Notebook to visualise qtransforms and skymaps of the interesting candidate real event pairs.|--------|----------|---------|----------------|
|
|
|
|
|
|
## Top-level scripts: Training, Cross-validation, Optimisation, Testing, Comparison with BLU.
|
|
|
|
|
|
__In preparation__
|
|
|
|
|
|
| Scripts | Short description | Status | git hash | Comment | final sign-off |
|
|
|
|---------|-------------------|--------|----------|---------|----------------|
|
|
|
|[data_gen_train_test.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/scripts/data_gen_train_test.py)|Generate Qtransforms, Dataframes, Bayestar skymaps for training and testing given the injection parameters|--------|----------|---------|----------------|
|
... | ... | |