Fix 240 - Sampling seed not passed to samplers

This PR addresses #240 (closed) by adding "sampling_seed" to sampler_kwargs.

If the user defines a value for the seed (seed, random_seed, sampling_seed) in sampler_kwargs then that value takes priority over sampling_seed.

Verification

I've run dynesty, bilby_mcmc, pymultinest and nessai twice with a fixed generation and sampling seed to verify that the same results returned.

• dynesty should be unaffected by this change since it doesn't have a "seed" keyword argument
• bilby_mcmc will not have identical results because torch is not seeded
• pymultinest and nessai should return identical ln-evidences.

The results are all available here: https://ligo.gravity.cf.ac.uk/~michael.williams/bilby-dev/bilby-pipe/issue-240/

Summary of results:

grep "ln_evidence" ./outdir*/log_data_analysis/*.err
./outdir_bilby_mcmc_A/log_data_analysis/bilby_mcmc_data0_0_analysis_H1L1.err:ln_evidence:    nan +/-    nan
./outdir_bilby_mcmc/log_data_analysis/bilby_mcmc_data0_0_analysis_H1L1.err:ln_evidence:    nan +/-    nan
./outdir_dynesty_A/log_data_analysis/dynesty_data0_0_analysis_H1L1.err:ln_evidence: -16131.997 +/-  0.016
./outdir_dynesty/log_data_analysis/dynesty_data0_0_analysis_H1L1.err:ln_evidence: -16131.997 +/-  0.016
./outdir_multinest_A/log_data_analysis/multinest_data0_0_analysis_H1L1.err:ln_evidence: -16132.006 +/-  0.020
./outdir_multinest_B/log_data_analysis/multinest_data0_0_analysis_H1L1.err:ln_evidence: -16132.006 +/-  0.020
./outdir_nessai_A/log_data_analysis/nessai_data0_0_analysis_H1L1.err:ln_evidence: -16132.017 +/-  0.020
./outdir_nessai/log_data_analysis/nessai_data0_0_analysis_H1L1.err:ln_evidence: -16132.017 +/-  0.020

Tests

The CI will fail at the moment because this PR requires changes in bilby first.

Related issues/MRs

Requires this change in bilby: bilby!1122 (closed)

Closes #240 (closed)

To-do

• Consider adding a unit test for this
• Release of bilby for tests to pass

mentioned in merge request bilby!1122 (closed)

mentioned in merge request bilby!1134 (merged)

marked the checklist item Release of bilby for tests to pass as completed

There new release of bilby should be live now so I'm going to rerun the CI

marked this merge request as ready

added 8 commits

• 55fe39cd...50d3d028 - 7 commits from branch lscsoft:master
• 8a028473 - Merge branch 'master' of git.ligo.org:michael.williams/bilby_pipe into fix-240-sampling-seed

Compare with previous version

Tests are failing in part due to the issue Colm mentioned here: !462 (comment 588237) but also due to some of the changes made in this MR. I'll take a look at the tests to try to figure out if it's an issue with the changes or the tests just need updating

added 1 commit

• be95d907 - Update tests for sampling seed changes

Compare with previous version

marked the checklist item Consider adding a unit test for this as completed

added 19 commits

• be95d907...7600db53 - 18 commits from branch lscsoft:master
• bad2ca68 - Merge branch 'master' of git.ligo.org:lscsoft/bilby_pipe into fix-240-sampling-seed

Compare with previous version

@gregory.ashton the tests are fixed now so it would be great if we could get this into the next release if possible

changed milestone to %1.0.7

approved this merge request

merged

mentioned in commit dd02cacc