parallel bilby 2.0

This MR makes major changes to pBilby

• updates to bilby>=2.0
• updates to dynesty>=2
• adds unit-tests
• read in new ROQ formats

Name                                      Stmts   Miss  Cover
-------------------------------------------------------------
parallel_bilby/__init__.py                    2      0   100%
parallel_bilby/analysis/__init__.py           5      0   100%
parallel_bilby/analysis/analysis_run.py     107      7    93%
parallel_bilby/analysis/likelihood.py        50     15    70%
parallel_bilby/analysis/main.py             124     16    87%
parallel_bilby/analysis/plotting.py          40      3    92%
parallel_bilby/analysis/read_write.py        71      8    89%
parallel_bilby/analysis/sample_space.py      12      0   100%
parallel_bilby/generation.py                 92      9    90%
parallel_bilby/parser.py                     90      6    93%
parallel_bilby/schwimmbad_fast.py           238     20    92%
parallel_bilby/slurm/__init__.py              1      0   100%
parallel_bilby/slurm/slurm.py               117     29    75%
parallel_bilby/slurm/template_slurm.sh        3      1    67%
parallel_bilby/utils.py                      29      2    93%
-------------------------------------------------------------
TOTAL                                       981    116    88%

This MR will: Close #70 (closed) Close #69 (closed) Close #68 (closed) Close #65 (closed) Close #60 (closed) Close #36 (closed) Close #56 (closed)

marked this merge request as draft

added 1 commit

• 5e54b5fc - ADACS AVajpeyi_2021B

Compare with previous version

added 3 commits

• 5e54b5fc...9d848ae5 - 2 commits from branch master
• e51a5ce9 - Merge branch 'master' into 'restructure'

Compare with previous version

marked this merge request as ready

changed the description

requested review from @avi.vajpeyi

mentioned in issue #60 (closed)

Regarding #60 (closed), @conrad.chan made 3 suggestions for possible ways to proceed:

1. Go back to using the old version of dynesty, which is non-deterministic but compatible with bilby - this will break some of the tests, but you’ll still be able to take advantage of the refactoring that we did (which was possible because we had those tests). Future code development will be a little more difficult without a good end to end test though.

2. Update, patch, or fork bilby for the new dynesty interface - to my understanding, this involves passing through rstate in a few function calls. This is the best long term solution, and benefits bilby as well

3. Fork dynesty so that the interface is compatible with what bilby expects, and also what pbilby expects. This is the best option if getting a change approved in bilby is too difficult. The dynesty devs seem to make big changes on minor releases, and they don’t follow an release versioning seriously - so if the bilby devs have no intention of using a newer version of dynesty in the future anyway, you don’t have much to lose, and you gain better control over your dependencies

mentioned in issue #48 (closed)

added 119 commits

• 061803ee - moved setup steps into separate class (refactoring work-in-progress)
• efe002bc - Merge branch 'restructure' of gitlab.com:chanconrad/parallel-bilby into restructure
• f7668dd8 - use updated sampler object from newer version of dynesty, and use the...
• 00abc0ae - fix non-determinstic RNG
• 92f46584 - Merge branch 'voodo' into restructure
• b9c068bc - remove debugging line
• 891a857e - Merge branch 'voodo' into restructure
• cda3de39 - Fix RNG interface when restarting from checkpoint
• c6c78873 - move format_result to read_write.py
• 92c10b65 - run pre-commit on run.py
• 30883565 - Fix changed imports
• 9f5288ab - rename analysis.py to main.py
• e2caa858 - rename utils.py to likelihood.py
• f4a254fd - rename run.py to analysis_run.py
• 3619764a - remove ptemcee_analysis.py - deprecated
• 7538c0e0 - remove unused implementation of ptemcee
• 9fa71172 - new file: sample_space.py
• dff862aa - make rstate an attribute of the run object
• 2e3c4c54 - minor refactoring
• 822c9bdd - use run.* variables directly rather than unpacking
• a8173b5a - remove unnecessary ndim argument
• 107167c9 - wrap NestedSampler creation in a function
• 6fd11279 - refactor early halting conditions
• 9cbb7307 - change format_result interface to get variables from run object
• bb8eb0c5 - fix search/replace mistake
• a28563c1 - fix test error
• c91b53f5 - fix test error -- removed ptmcee parser args, since ptmcee was removed in...
• 6d096fcc - make interface to analysis explicit
• ece994c8 - set default outdir to be whatever is defined in data_dump
• 913c6aa7 - add back tearDown() for e2e test
• 03b2acc6 - lint
• 210732e1 - unit test for readwrite routines
• c87b696b - install deepdiff for pipeline tests
• e08b184a - Use a working version of dynesty for pipeline tests
• e8b6ef8a - fix typo (amend previous commit)
• 0255af04 - use https to install from git repo
• 235c1239 - fix apt-get update error for pipeline
• 6889063a - debug template dir for pipeline
• e2c6438d - debug
• da20e8be - add slurm template to package data
• 770366d0 - remove debugging lines
• 054b6338 - Merge branch 'slurm-template-fix' into restructure
• 9703759f - specify testing dependencies
• d35a013c - set the good version of dynesty as a requirement
• 3eb7ea6e - revert previous commit
• cd104e31 - increase timeout for GW150914 E2E test
• 3c363b08 - added test for fill_sample()
• 9ad27b7f - decrease significant digits for sample_space test
• 33da1369 - amend significant digits from previous commit to 7
• 9c738cfd - amend significant digits from previous commit to 6
• 1c99f325 - reduce precision for numpy comparisons
• 199a427e - test_readwrite: finally clean up test file
• 36a05657 - add teardown for fill sample test
• baf33d10 - added unit test of reorder_loglikelihoods()
• 3ea678b2 - fix outdir creation logic
• 0e6f1fc9 - analysis_run tests: prior_transform_function
• ea7bed2a - added test for log_likelihood_function()
• b1e215c0 - add test for log_prior_function
• 04193d98 - add test for get_initial_point_from_prior
• c722cd9e - add test for get_initial_points_from_prior
• 023f74ff - add test for get_nested_sampler
• 61daebc0 - reduce verbosity of mpi tests in pipeline
• 35c0b1f4 - mark test_get_nested_sampler as mpi test
• b173fcff - append coverage from MPI tests
• 6b51b849 - mark tests to skip when using mpi
• e54fe229 - write to separate .coverage files to prevent MPI race condition in test
• 91cc71d8 - test dill usage in schwimmbad
• 882088f4 - add test for mpi timer
• f6c8252f - Merge branch 'schwimmbad_tests' into restructure
• af80bde7 - rename test files
• 14e1bbcc - test for maximum number of iterations
• 58c3da12 - schwimmbad: only send kill signal if pool is open
• 67cbcf7e - fix time based halting condition not triggering
• 07a2160c - add test for run time halting
• 33430ade - tidy logic for early_stop
• 0fc895ec - pre-commit version updates
• f9da3660 - updated setup.cfg options for isort
• d046e46f - pre-commit compliance
• 4ab5ba2d - max_its test: only check pickle file on master
• 569c523a - raise runtimeerror when attempting to close closed pool
• 8defb6f3 - resume test (WIP)
• eb55d6b3 - save random number generator state to resume file
• 504257bc - test resume results are the same
• 3d973be4 - Remove overriding nqueue to -1
• b254107c - add approx comparison for resume test
• 68b561c5 - include random state in readwrite test
• 1510359c - Merge branch 'resume_testing' into restructure
• 48786ff1 - fix checkpointing logic
• 7a9dd78b - ROQ test
• 09e5b93a - Merge branch 'roq-test' into restructure
• 8e90cf21 - remove catching ImportError for mpi44py
• fd220d9a - Change which task saves timing interval
• 5295eea0 - test_schwimmbad: test that the pool is enabled
• 6fe337d2 - test_schwimmbad: test timing of serial sections
• 15616331 - test_schwimmbad: test manual pool creation, test null timer
• c7396565 - fix manual pool test
• 0753edd4 - moved test utils
• be430bcc - bugfix: import path for mpi_master in test_roq
• 1884c6ce - overhauled test suite + changed interface to generate_runner()
• e589e3ed - test_schwimmbad: loosen tolerance for serial time test
• f9d60b99 - test_roq: fix case dict syntax
• efce2b46 - us os.path for test_dir
• a7b331fd - added defaults for slurm CLI args, instead of required=True
• 4f750f91 - modified test_generation to also write and check for slurm script
• b3f085f0 - kill pytest if mpi task is stuck
• 2353ae8b - document sample_space.py
• 4b43ee30 - docstring for format_result()
• fb8670a3 - Merge remote-tracking branch 'adacs/restructure' into restructure
• 78970bf8 - document generation.py
• 342ce81d - Merge branch 'restructure' of gitlab.com:chanconrad/parallel-bilby into restructure
• 7665ecae - document analysis [skip ci]
• ad0c3a71 - remove unnecessary shebangs [skip ci]
• 59ce7e4d - docs for analysis_run
• d768064c - Merge remote-tracking branch 'adacs/restructure' into restructure
• 35b7003e - document test/utils.py [skip ci]
• 6269b00d - Add descriptions for every test
• 3e302e7b - fix pre-commits
• aaa67c83 - Merge branch 'restructure' of gitlab.com:chanconrad/parallel-bilby into restructure
• 847ea639 - Merge branch 'restructure' of git.ligo.org:lscsoft/parallel_bilby into restructure

Compare with previous version

added 1 commit

• 0343d1e2 - update python version

Compare with previous version

added 1 commit

• 19d30a87 - update image

Compare with previous version

added 1 commit

• 1707b6af - +update py

Compare with previous version

added 2 commits

• 49abcd0e - +update py
• 5a6b4b57 - +update dynesty print method api change

Compare with previous version

added 1 commit

• bceb2227 - +api updates

Compare with previous version

mentioned in merge request !80 (closed)

mentioned in issue #41 (closed)

mentioned in issue #59 (closed)

Bilby 1.4 has Dynesty>2!

added 1 commit

• a6be7b33 - add updated versions of dynesty and bilby

Compare with previous version