Skip to content

Changes

Page history
Update Code Review 2023 authored by Jean-Rene Cudell's avatar Jean-Rene Cudell
Show whitespace changes
Inline Side-by-side
Code-Review-2023.asciidoc
View page @ b83ebff2
...@@ -51,8 +51,8 @@ Can be found [here](https://git.ligo.org/srashti.goyal/lensid/-/wikis/Code-revie ...@@ -51,8 +51,8 @@ Can be found [here](https://git.ligo.org/srashti.goyal/lensid/-/wikis/Code-revie
| [lensid_create_lensed_inj_xmls.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_lensed_inj_xmls.py) | helper script that outputs LAL inj.xml file for lensed simulated events given the injection parameters for bayestar. | OK | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | a46b1d4a9755bae8438baaf053d2fb552a0808b9 | [diff](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/diff_package_03052023_reviewed_2022.diff#L14-44) | :heavy_check_mark: | | [lensid_create_lensed_inj_xmls.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_lensed_inj_xmls.py) | helper script that outputs LAL inj.xml file for lensed simulated events given the injection parameters for bayestar. | OK | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | a46b1d4a9755bae8438baaf053d2fb552a0808b9 | [diff](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/diff_package_03052023_reviewed_2022.diff#L14-44) | :heavy_check_mark: |
| [lensid_create_unlensed_inj_xmls.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_unlensed_inj_xmls.py) | helper script that outputs LAL inj.xml file for unlensed simulated events given the injection parameters for bayestar. minor changes in the parameter names. | OK | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | a46b1d4a9755bae8438baaf053d2fb552a0808b9 | [diff](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/diff_package_03052023_reviewed_2022.diff#L358-388) | :heavy_check_mark: | | [lensid_create_unlensed_inj_xmls.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_create_unlensed_inj_xmls.py) | helper script that outputs LAL inj.xml file for unlensed simulated events given the injection parameters for bayestar. minor changes in the parameter names. | OK | 32d0854b1a68cf21827e65ca1c36feb7ca53d0f5 | a46b1d4a9755bae8438baaf053d2fb552a0808b9 | [diff](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/diff_package_03052023_reviewed_2022.diff#L358-388) | :heavy_check_mark: |
| [lensid_create_bayestar_sky_lensed_injs.sh](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/scripts/lensid_create_bayestar_sky_lensed_injs.sh) | generates bayestar skymaps(.fits) for lensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `lensid_create_bayestar_sky_lensed_injs.sh -o check -s 10 -n 3 -i ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz -p ~/lensid/data/PSDs/analytical_psd.xml` Note: if this does not work try running this before `export PATH=$HOME/.local/bin:$PATH` | OK | 493ea099f42fc50d2cc081754d5395f57fafae76 | ------- | -------------- |:heavy_check_mark:| | [lensid_create_bayestar_sky_lensed_injs.sh](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/scripts/lensid_create_bayestar_sky_lensed_injs.sh) | generates bayestar skymaps(.fits) for lensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `lensid_create_bayestar_sky_lensed_injs.sh -o check -s 10 -n 3 -i ~/lensid/data/injection_pars/haris-et-al/lensed_inj_data.npz -p ~/lensid/data/PSDs/analytical_psd.xml` Note: if this does not work try running this before `export PATH=$HOME/.local/bin:$PATH` | OK | 493ea099f42fc50d2cc081754d5395f57fafae76 | ------- | -------------- |:heavy_check_mark:|
| [lensid_create_bayestar_sky_unlensed_injs.sh](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/scripts/lensid_create_bayestar_sky_unlensed_injs.sh) | generates bayestar skymaps(.fits) for unlensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `lensid_create_bayestar_sky_unlensed_injs.sh -o check -s 10 -n 3 -i ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz -p ~/lensid/data/PSDs/analytical_psd.xml` | OK | 493ea099f42fc50d2cc081754d5395f57fafae76 | ------- | -------------- | :heavy_check_mark: 83c208d82024a6abc49af67928a2d63653743ce3 | | [lensid_create_bayestar_sky_unlensed_injs.sh](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/scripts/lensid_create_bayestar_sky_unlensed_injs.sh) | generates bayestar skymaps(.fits) for unlensed simulated events, using analytical/O3a PSDs. Also converts them to cartesian format and save as .npz files. Eg: `lensid_create_bayestar_sky_unlensed_injs.sh -o check -s 10 -n 3 -i ~/lensid/data/injection_pars/haris-et-al/unlensed_inj_data.npz -p ~/lensid/data/PSDs/analytical_psd.xml` | OK | 493ea099f42fc50d2cc081754d5395f57fafae76 | ------- | -------------- | :heavy_check_mark: |
| [lensid_fits_to_cart.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/lensid_fits_to_cart.py) | helper script for converting HealPix skymap format(.fits) to cartesian. | OK | ac95f97e0c7e8d584b68ed364f353a5ed4bbb12d | need sanity check for hp.cartview during results review | unchanged | :heavy_check_mark: 83c208d82024a6abc49af67928a2d63653743ce3| | [lensid_fits_to_cart.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/utils/lensid_fits_to_cart.py) | helper script for converting HealPix skymap format(.fits) to cartesian. | OK | ac95f97e0c7e8d584b68ed364f353a5ed4bbb12d | need sanity check for hp.cartview during results review | unchanged | :heavy_check_mark: |
| [lensid_sky_injs_cart.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_sky_injs_cart.py) | helper script for managing IO of fits_to_cart.py script for injection study | OK | 493ea099f42fc50d2cc081754d5395f57fafae76 | OK | -unchaged | :heavy_check_mark: | | [lensid_sky_injs_cart.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/injections/lensid_sky_injs_cart.py) | helper script for managing IO of fits_to_cart.py script for injection study | OK | 493ea099f42fc50d2cc081754d5395f57fafae76 | OK | -unchaged | :heavy_check_mark: |
### Features extraction, Train/test/predict utilities ### Features extraction, Train/test/predict utilities
...@@ -67,8 +67,8 @@ Can be found [here](https://git.ligo.org/srashti.goyal/lensid/-/wikis/Code-revie ...@@ -67,8 +67,8 @@ Can be found [here](https://git.ligo.org/srashti.goyal/lensid/-/wikis/Code-revie
| Scripts | Short description | Status | git hash | Comment | final sign-off | | Scripts | Short description | Status | git hash | Comment | final sign-off |
|---------|-------------------|--------|----------|---------|----------------| |---------|-------------------|--------|----------|---------|----------------|
| [train_densenets_qts.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/train_test/train_densenets_qts.py) | Train densenet with qtransform for a given detector. Eg: `from lensid.train_test.train_densenets_qts import _main; _main('out/','/home/srashti.goyal/lensid_runs/uniform_dataset/npz/','/home/srashti.goyal/lensid_runs/uniform_dataset/dataframe/lensed.csv','/home/srashti.goyal/lensid_runs/uniform_dataset/dataframe/unlensed_half.csv',size_lensed=8000,size_unlensed=8000,batch_size=500,det='V1',epochs=20,lr=0.005,whitened=1,file_type='npz',colored=0,model_id=0)`. Note: requires `tensorflow-gpu` to load CUDA libraries. | OK |7631c2b7530be09372721c2d3d3f0e27e792a53c | changed significantly from the previous release for using the dataloader and npz files. | :heavy_check_mark: | | [train_densenets_qts.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/train_test/train_densenets_qts.py) | Train densenet with qtransform for a given detector. Eg: `from lensid.train_test.train_densenets_qts import _main; _main('out/','/home/srashti.goyal/lensid_runs/uniform_dataset/npz/','/home/srashti.goyal/lensid_runs/uniform_dataset/dataframe/lensed.csv','/home/srashti.goyal/lensid_runs/uniform_dataset/dataframe/unlensed_half.csv',size_lensed=8000,size_unlensed=8000,batch_size=500,det='V1',epochs=20,lr=0.005,whitened=1,file_type='npz',colored=0,model_id=0)`. Note: requires `tensorflow-gpu` to load CUDA libraries. | OK |7631c2b7530be09372721c2d3d3f0e27e792a53c | changed significantly from the previous release for using the dataloader and npz files. | :heavy_check_mark: |
[train_crossvalidate_test_XGB_sky.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/train_test/train_crossvalidate_test_XGB_sky.py) | Train, cross-validates and compare to BLU "XGBoost with Skymaps" model. Requires dataframe that already has the input features calculated from the Bayestar/PE skymaps. `python train_crossvalidate_test_XGB_sky.py -help` | OK-DC; OK-jrc | a60740bb5a0cccb2be8e8184f16c0c7c93f8150b | No change. |:heavy_check_mark: | |[train_crossvalidate_test_XGB_sky.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/train_test/train_crossvalidate_test_XGB_sky.py) | Train, cross-validates and compare to BLU "XGBoost with Skymaps" model. Requires dataframe that already has the input features calculated from the Bayestar/PE skymaps. `python train_crossvalidate_test_XGB_sky.py -help` | OK-DC; OK-jrc | a60740bb5a0cccb2be8e8184f16c0c7c93f8150b | No change. |:heavy_check_mark: |
[ml_predict_workflow.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/ml_predict_workflow.py) | Make predictions gives the Qtransforms, Skymaps and trained ML models. Needs CONFIG.yaml file for running. example: [config_O3_events_18022022](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/examples/config_O3_events_18022022.yaml) `lensid_make_predictions --config /home/srashti.goyal/lensid/package/examples/config_O3_events_18022022.yaml` | OK | 83c208d82024a6abc49af67928a2d63653743ce3 | Minor changes w.r.t result review O3. |:heavy_check_mark: | |[ml_predict_workflow.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/ml_predict_workflow.py) | Make predictions gives the Qtransforms, Skymaps and trained ML models. Needs CONFIG.yaml file for running. example: [config_O3_events_18022022](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/examples/config_O3_events_18022022.yaml) `lensid_make_predictions --config /home/srashti.goyal/lensid/package/examples/config_O3_events_18022022.yaml` | OK | 83c208d82024a6abc49af67928a2d63653743ce3 | Minor changes w.r.t result review O3. |:heavy_check_mark: |
......