pastro_uploader: Close model tempfile
In the pastro uploader, the add_rankstat_to_model
method never closes the temporary file descriptor opened with tempfile.mkstemp
. This means the file on the execute node is held open even after shutil.move
copies it to the home directory and unlinks the local temp file. As a result, that disk space stays reserved, and the disk usage increases by the size of the pastro model (1.4GB) every time the model file is updated (every 4 hours).
This MR causes the job to close the file after it is created. self.model.to_h5
still takes in the temporary file name, opening and closing the file itself.
This bug was not caught until now because Condor (for now) does not track disk usage of files which have been unlinked.
Edited by Cort Posnansky