Fix checkpointing exceptions
My runs are throwing out the following exception every time they try to checkpoint:
Traceback (most recent call last):
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20230425/lib/python3.9/site-packages/dynesty/dynesty.py", line 910, in __call__
return self.func(np.asarray(x).copy(), *self.args, **self.kwargs)
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20230425/lib/python3.9/site-packages/bilby/core/sampler/dynesty.py", line 53, in _log_likelihood_wrapper
return _sampling_convenience_dump.likelihood.log_likelihood_ratio()
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20230425/lib/python3.9/site-packages/bilby/gw/likelihood/base.py", line 412, in log_likelihood_ratio
log_l = self.compute_log_likelihood_from_snrs(total_snrs)
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20230425/lib/python3.9/site-packages/bilby/gw/likelihood/base.py", line 432, in compute_log_likelihood_from_snrs
log_l = self.time_marginalized_likelihood(
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20230425/lib/python3.9/site-packages/bilby/gw/likelihood/base.py", line 761, in time_marginalized_likelihood
log_l_tc_array = self.distance_marginalized_likelihood(
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20230425/lib/python3.9/site-packages/bilby/gw/likelihood/base.py", line 748, in distance_marginalized_likelihood
return self._interp_dist_margd_loglikelihood(
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20230425/lib/python3.9/site-packages/bilby/core/utils/calculus.py", line 231, in __call__
output[~bad], ier = bispeu(*self.tck, x[~bad], y[~bad])
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20230425/lib/python3.9/site-packages/bilby/core/sampler/dynesty.py", line 731, in write_current_state_and_exit
super(Dynesty, self).write_current_state_and_exit(signum=signum, frame=frame)
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20230425/lib/python3.9/site-packages/bilby/core/sampler/base_sampler.py", line 727, in write_current_state_and_exit
sys.exit(self.exit_code)
SystemExit: 77
This applies whether or not the runs are converging, so I'm not sure that it's causing any real problems (perhaps slowing things down?). Here is an example log file that shows this exception occuring: https://ldas-jobs.ligo.caltech.edu/~yannick.lecoeuche/glitch-ylecoeuche/fastscatter2/GW190521/outdir_B/log_data_analysis/label_data5_1240770945-0_analysis_H1L1V1.err