Skip to content

Changes

Page history
Update Code Review 2023 authored by Srashti Goyal's avatar Srashti Goyal
Show whitespace changes
Inline Side-by-side
Code-Review-2023.md
View page @ 86fad9e7
...@@ -67,14 +67,14 @@ Can be found [here](https://git.ligo.org/srashti.goyal/lensid/-/wikis/Code-revie ...@@ -67,14 +67,14 @@ Can be found [here](https://git.ligo.org/srashti.goyal/lensid/-/wikis/Code-revie
|---------|-------------------|--------|----------|---------|----------------| |---------|-------------------|--------|----------|---------|----------------|
| [train_densenets_qts.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/train_test/train_densenets_qts.py) | Train densenet with qtransform for a given detector. Eg: `from lensid.train_test.train_densenets_qts import _main; _main('out/','/home/srashti.goyal/lensid_runs/uniform_dataset/npz/','/home/srashti.goyal/lensid_runs/uniform_dataset/dataframe/lensed.csv','/home/srashti.goyal/lensid_runs/uniform_dataset/dataframe/unlensed_half.csv',size_lensed=8000,size_unlensed=8000,batch_size=500,det='V1',epochs=20,lr=0.005,whitened=1,file_type='npz',colored=0,model_id=0)`. Note: requires `tensorflow-gpu` to load CUDA libraries. | |7631c2b7530be09372721c2d3d3f0e27e792a53c | changed significantly from the preivous release for using the dataloader and npz files. | ---------------- | | [train_densenets_qts.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/train_test/train_densenets_qts.py) | Train densenet with qtransform for a given detector. Eg: `from lensid.train_test.train_densenets_qts import _main; _main('out/','/home/srashti.goyal/lensid_runs/uniform_dataset/npz/','/home/srashti.goyal/lensid_runs/uniform_dataset/dataframe/lensed.csv','/home/srashti.goyal/lensid_runs/uniform_dataset/dataframe/unlensed_half.csv',size_lensed=8000,size_unlensed=8000,batch_size=500,det='V1',epochs=20,lr=0.005,whitened=1,file_type='npz',colored=0,model_id=0)`. Note: requires `tensorflow-gpu` to load CUDA libraries. | |7631c2b7530be09372721c2d3d3f0e27e792a53c | changed significantly from the preivous release for using the dataloader and npz files. | ---------------- |
[train_crossvalidate_test_XGB_sky.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/train_test/train_crossvalidate_test_XGB_sky.py) | Train, cross-validates and compare to BLU "XGBoost with Skymaps" model. Requires dataframe that already has the input features calculated from the Bayestar/PE skymaps. `python train_crossvalidate_test_XGB_sky.py -help` | OK-DC; OK-jrc | a60740bb5a0cccb2be8e8184f16c0c7c93f8150b | No change. |:heavy_check_mark: | [train_crossvalidate_test_XGB_sky.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/train_test/train_crossvalidate_test_XGB_sky.py) | Train, cross-validates and compare to BLU "XGBoost with Skymaps" model. Requires dataframe that already has the input features calculated from the Bayestar/PE skymaps. `python train_crossvalidate_test_XGB_sky.py -help` | OK-DC; OK-jrc | a60740bb5a0cccb2be8e8184f16c0c7c93f8150b | No change. |:heavy_check_mark: |
[ml_predict_workflow.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/lensid/ml_predict_workflow.py) | Make predictions gives the Qtransforms, Skymaps and trained ML models. Needs CONFIG.yaml file for running. example: [config_O3_events_18022022](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/examples/config_O3_events_18022022.yaml) `lensid_make_predictions --config /home/srashti.goyal/lensid/package/examples/config_O3_events_18022022.yaml` | | 83c208d82024a6abc49af67928a2d63653743ce3 | Minor changes w.r.t result review O3. | |
yet to add cross-validate + ensemble + test script for QTs and a script for combining QT and Sky outputs.
## Other scripts ## Other scripts
| Scripts | Short description | Status | git hash | Comment | final sign-off | | Scripts | Short description | Status | git hash | Comment | final sign-off |
|---------|-------------------|--------|----------|---------|----------------| |---------|-------------------|--------|----------|---------|----------------|
| [condor_data_gen_train_test_config.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/train_test_workflow/condor_data_gen_train_test_config.py) | Generate Qtransforms, Dataframes, Bayestar skymaps for training and testing given the injection parameters using condor dag jobs submission. Note: change `exec_file_loc` in the script according to your installation and `base_out_dir` as desired. Eg: [config_train_test_workflow.yaml](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/train_test_workflow/config_train_test_workflow.yaml) | | 1a3fc5d0ce285cec52576ccd7de645f79e772879 | very minor change, just added `request_disk` option [diff](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/review/diff_condor_data_gen.diff) | ---------------- | | [condor_data_gen_train_test_config.py](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/train_test_workflow/condor_data_gen_train_test_config.py) | Generate Qtransforms, Dataframes, Bayestar skymaps for training and testing given the injection parameters using condor dag jobs submission. Note: change `exec_file_loc` in the script according to your installation and `base_out_dir` as desired. Eg: [config_o4_datagen.yaml](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/package/examples/config_o4_datagen.yaml) `python condor_data_gen_train_test_config.py -config ../package/examples/config_o4_datagen.yaml` | | 1a3fc5d0ce285cec52576ccd7de645f79e772879 | very minor change, just added `request_disk` option [diff](https://git.ligo.org/srashti.goyal/lensid/-/blob/master/review/diff_condor_data_gen.diff) | ---------------- |
......