... | ... | @@ -102,13 +102,17 @@ The ML model predictions are compared to posterior overlap statistic results als |
|
|
| [lensid_get_features_sky_ml.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/feature_extraction/lensid_get_features_sky_ml.py) | Script for calculating features from the bayestar skymaps which go as input to "XGBoost with Skymaps" model. Eg: `lensid_get_features_sky_ml -infile check/lensed.csv -outfile check/lensed_sky.csv -data_dir check` | -OK-- | -------- | ------- | -------------- |
|
|
|
| [ml_utils.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/ml_utils.py) | utility script containing all machine learning model functions for training, FAP computation, predictions etc. | in progress------ | -------- | the fixed directory inn densebet_input_matrix should be user-defined. Why is leakyrelu loaded? The logic of rfn(epoch) could be simplified.------- | -------------- |
|
|
|
|
|
|
## ML models: Training, Cross-validation, Optimisation, Testing, Comparison with BLU, Investigations.
|
|
|
| Notebook | Short description | Status | git hash | Comment | final sign-off |
|
|
|
## ML models: Training, Cross-validation, Optimisation, Testing, Comparison with BLU.
|
|
|
| Scripts | Short description | Status | git hash | Comment | final sign-off |
|
|
|
|----------|-------------------|--------|----------|---------|----------------|
|
|
|
| [train_densenets_qts.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/scripts/train_densenets_qts.py) | Train densenet with qtransform for a given detector. Eg: `python train_densenets_qts.py -lensed_df ~/strong-lensing-ml/data/dataframes/train/lensed.csv -unlensed_df ~/strong-lensing-ml/data/dataframes/train/unlensed_half.csv -odir dense_out/cit/ -epochs 10 -data_dir ~/alice_data_lensid/qts/train/`. Note: tensorflowenv on pcdev12@CIT has GPU | ------ | -------- | ------- | -------------- |
|
|
|
| [train_crossvalidate_test_XGB_qts.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/scripts/train_crossvalidate_test_XGB_qts.py) | Trains, cross-validate and compare to BLU "XGBoost with QTs" model. Requires dataframe that already has the input features calculated from the Qtransform images and trained DenseNets. `python train_crossvalidate_test_XGB_qts.py ` | ------ | -------- | ------- | -------------- |
|
|
|
| [train_crossvalidate_test_XGB_sky.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/scripts/train_crossvalidate_test_XGB_sky.py) | Train, cross-validates and compare to BLU "XGBoost with Skymaps" model. Requires dataframe that already has the input features calculated from the Bayestar/PE skymaps. `python train_crossvalidate_test_XGB_sky.py ` | ------ | -------- | ------- | -------------- |
|
|
|
| [test_combined_ML_results.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/scripts/test_combined_ML_results.py) | Test and compare to BLU overall ML model. Requires dataframes that already has the ML predictions calculated from the qts and skymaps. `python test_combined_ML_results.py ` | ------ | -------- | ------- | -------------- |
|
|
|
|
|
|
## Investigations: Injection parameters, features statistics, PSDs etc.
|
|
|
| Notebook | Short description | Status | git hash | Comment | final sign-off |
|
|
|
|----------|-------------------|--------|----------|---------|----------------|
|
|
|
| [train_test_pars.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/review/train_test_pars.ipynb) | Notebook having plots of injection parameters for training, testing and O3a sets that are used. | ------ | -------- | ------- | -------------- |
|
|
|
| [ML_blu_FPPs_inj_pars_investigate.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/review/ML_blu_FPPs_inj_pars_investigate.ipynb) | Notebook comparing the ML and BLU FPPs for each pair in test set, and also investigationg correlations with the injection parameters. Also contain statistics of input sky features for lensed and unlensed test pairs. | ------ | -------- | ------- | -------------- |
|
|
|
| [PSD_plots.ipynb](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/review/PSD_plots.ipynb) | Notebook having plots of PSDs that are used. | ------ | -------- | ------- | -------------- |
|
... | ... | |