Skip to content

Dynesty sampling breaks if a checkpoint is being written before all prior samples are drawn

Noticed because bilby_pipe jobs were getting put on hold. If drawing from the prior is not finished by the time the bilby_pipe job reaches its periodic_restart_time and tries to checkpoint, the below error occurs because the sampler is not yet instantiated. This puts bilby_pipe jobs on hold.

Drawing from the prior can take a long time if the priors are non-sensical, e.g., if both prior.minimum = 0 and prior.maximum = 0. Therefore, a good solution to this issue would be for bilby to flag if priors are weird.

Writing checkpoint file bilby_pipe_test/result/S190425_initial_analysis_public_data_bilby_pipe_test_data0_1240215503-0171_analysis_L1V1_dynesty_resume.pickle
Traceback (most recent call last):
 File "/local/condor/execute/dir_126817/condor_exec.exe", line 11, in <module>
  load_entry_point('bilby-pipe==0.3.8', 'console_scripts', 'bilby_pipe_analysis')()
 File "/home/isobel.romero-shaw/anaconda3/lib/python3.6/site-packages/bilby_pipe-0.3.8-py3.6.egg/bilby_pipe/data_analysis.py", line 322, in main
  analysis.run_sampler()
 File "/home/isobel.romero-shaw/anaconda3/lib/python3.6/site-packages/bilby_pipe-0.3.8-py3.6.egg/bilby_pipe/data_analysis.py", line 299, in run_sampler
  **self.sampler_kwargs,
 File "/home/isobel.romero-shaw/anaconda3/lib/python3.6/site-packages/bilby-0.6.3-py3.6.egg/bilby/core/sampler/__init__.py", line 176, in run_sampler
  result = sampler.run_sampler()
 File "/home/isobel.romero-shaw/anaconda3/lib/python3.6/site-packages/bilby-0.6.3-py3.6.egg/bilby/core/sampler/dynesty.py", line 234, in run_sampler
  self.kwargs['nlive']))
 File "/home/isobel.romero-shaw/anaconda3/lib/python3.6/site-packages/bilby-0.6.3-py3.6.egg/bilby/core/sampler/base_sampler.py", line 449, in get_initial_points_from_prior
  if self.check_draw(theta, warning=False):
 File "/home/isobel.romero-shaw/anaconda3/lib/python3.6/site-packages/bilby-0.6.3-py3.6.egg/bilby/core/sampler/base_sampler.py", line 470, in check_draw
  if np.isinf(self.log_prior(theta)):
 File "/home/isobel.romero-shaw/anaconda3/lib/python3.6/site-packages/bilby-0.6.3-py3.6.egg/bilby/core/sampler/base_sampler.py", line 378, in log_prior
  return self.priors.ln_prob(params)
 File "/home/isobel.romero-shaw/anaconda3/lib/python3.6/site-packages/bilby-0.6.3-py3.6.egg/bilby/core/prior.py", line 418, in ln_prob
  for key in sample], axis=axis)
 File "/home/isobel.romero-shaw/anaconda3/lib/python3.6/site-packages/bilby-0.6.3-py3.6.egg/bilby/core/prior.py", line 418, in <listcomp>
  for key in sample], axis=axis)
 File "/home/isobel.romero-shaw/anaconda3/lib/python3.6/site-packages/bilby-0.6.3-py3.6.egg/bilby/core/prior.py", line 1663, in ln_prob
  return -0.5 * ((self.mu - val) ** 2 / self.sigma ** 2 + np.log(2 * np.pi * self.sigma ** 2))
 File "/home/isobel.romero-shaw/anaconda3/lib/python3.6/site-packages/bilby-0.6.3-py3.6.egg/bilby/core/sampler/dynesty.py", line 415, in write_current_state_and_exit
  self.write_current_state(plot=False)
 File "/home/isobel.romero-shaw/anaconda3/lib/python3.6/site-packages/bilby-0.6.3-py3.6.egg/bilby/core/sampler/dynesty.py", line 444, in write_current_state
  unit_cube_samples=self.sampler.saved_u,
AttributeError: 'Dynesty' object has no attribute 'sampler'
Edited by Isobel Romero-Shaw