Skip to content

pastro_uploader: Close model tempfile

Cort Posnansky requested to merge o4b-online-bugfix-mkstemp into o4b-online

In the pastro uploader, the add_rankstat_to_model method never closes the temporary file descriptor opened with tempfile.mkstemp. This means the file on the execute node is held open even after shutil.move copies it to the home directory and unlinks the local temp file. As a result, that disk space stays reserved, and the disk usage increases by the size of the pastro model (1.4GB) every time the model file is updated (every 4 hours).

This MR causes the job to close the file after it is created. self.model.to_h5 still takes in the temporary file name, opening and closing the file itself.

This bug was not caught until now because Condor (for now) does not track disk usage of files which have been unlinked.

Edited by Cort Posnansky

Merge request reports