Fix 240 - Sampling seed not passed to samplers

This PR addresses #240 (closed) by adding "sampling_seed" to sampler_kwargs.

If the user defines a value for the seed (seed, random_seed, sampling_seed) in sampler_kwargs then that value takes priority over sampling_seed.

Verification

I've run dynesty, bilby_mcmc, pymultinest and nessai twice with a fixed generation and sampling seed to verify that the same results returned.

• dynesty should be unaffected by this change since it doesn't have a "seed" keyword argument
• bilby_mcmc will not have identical results because torch is not seeded
• pymultinest and nessai should return identical ln-evidences.

The results are all available here: https://ligo.gravity.cf.ac.uk/~michael.williams/bilby-dev/bilby-pipe/issue-240/

Summary of results:

grep "ln_evidence" ./outdir*/log_data_analysis/*.err
./outdir_bilby_mcmc_A/log_data_analysis/bilby_mcmc_data0_0_analysis_H1L1.err:ln_evidence:    nan +/-    nan
./outdir_bilby_mcmc/log_data_analysis/bilby_mcmc_data0_0_analysis_H1L1.err:ln_evidence:    nan +/-    nan
./outdir_dynesty_A/log_data_analysis/dynesty_data0_0_analysis_H1L1.err:ln_evidence: -16131.997 +/-  0.016
./outdir_dynesty/log_data_analysis/dynesty_data0_0_analysis_H1L1.err:ln_evidence: -16131.997 +/-  0.016
./outdir_multinest_A/log_data_analysis/multinest_data0_0_analysis_H1L1.err:ln_evidence: -16132.006 +/-  0.020
./outdir_multinest_B/log_data_analysis/multinest_data0_0_analysis_H1L1.err:ln_evidence: -16132.006 +/-  0.020
./outdir_nessai_A/log_data_analysis/nessai_data0_0_analysis_H1L1.err:ln_evidence: -16132.017 +/-  0.020
./outdir_nessai/log_data_analysis/nessai_data0_0_analysis_H1L1.err:ln_evidence: -16132.017 +/-  0.020

Tests

The CI will fail at the moment because this PR requires changes in bilby first.

Related issues/MRs

Requires this change in bilby: bilby!1122 (closed)

Closes #240 (closed)

To-do

• Consider adding a unit test for this
• Release of bilby for tests to pass

bilby_pipe/data_analysis.py

@@ -115,20 +115,23 @@ class DataAnalysisInput(Input):
 
     @property
     def sampling_seed(self):
-        return self._samplng_seed
+        return self._sampling_seed
 
     @sampling_seed.setter
     def sampling_seed(self, sampling_seed):
         if sampling_seed is None:
             sampling_seed = np.random.randint(1, 1e6)
-        self._samplng_seed = sampling_seed
+        self._sampling_seed = sampling_seed
         np.random.seed(sampling_seed)
         logger.info(f"Sampling seed set to {sampling_seed}")
 
-        if self.sampler == "cpnest":
-            self.sampler_kwargs["seed"] = self.sampler_kwargs.get(
-                "seed", self._samplng_seed
-            )
+        if not any(
+            [
+                k in self.sampler_kwargs
+                for k in bilby.core.sampler.Sampler.sampling_seed_equiv_kwargs
+            ]
+        ):
+            self.sampler_kwargs["sampling_seed"] = self._sampling_seed
 
     @property
     def interferometers(self):