Skip to content

make checkpoint backup before overwriting

David Keitel requested to merge david-keitel/parallel_bilby:backup-checkpoint into master

@rory-smith @gregory.ashton @avi.vajpeyi Simplified implementation to resolve #35 (closed):

  • backup file has same path as main checkpoint plus .bk ending
  • only done if --rotate-checkpoints option given

This isn't "rotation" in the classical sense (making e.g. .0 and .1 files and writing to them alternatingly), but sufficiently equivalent to increase redundancy and enable users to go back to a previous stage if the latest checkpoint is corrupt.

(I noticed there already is a temporary-file safety net against corruption at the time of writing to disk, but from our experience this is insufficient to guard against some types of file system failures, and in addition the persistent backup also allows to go back to a previous state of the sampler if there's a problem between checkpoints.)

The off-by-default is because it increases disk footprint substantially and so I imagine you wouldn't want to spring this new behaviour onto users automatically.

Merge request reports