Skip to content

ligolw_run_sqlite: fix issue where xml files were not being updated when using --tmp-space

Patrick Godwin requested to merge patrick.godwin/python-ligo-lw:run_sqlite_xml_tmpspace into master

After the port to dbtables.workingcopy, an issue was discovered when running ligolw_run_sqlite on XML files when using the --tmp-space feature where the original files were not being correctly updated, essentially making this a no-op.

If you look at the logs with --verbose, you can see that it doesn't look quite right:

1/1: using '/local/patrick.godwin/tmpd2hd5pdaxml.gz' as workspace
copying 'test.xml.gz' to '/local/patrick.godwin/tmpd2hd5pdaxml.gz' ... done.
reading 'test.xml.gz' ...
setting the sqlite temp_store_directory to /local/patrick.godwin ... done
Executing SQL ...
CREATE INDEX tmpindex1 ON coinc_event (coinc_event_id)
CREATE INDEX tmpindex2 ON coinc_event_map (coinc_event_id)
UPDATE coinc_event SET likelihood = 0;
... Done.
writing 'test.xml.gz' ...
moving '/local/patrick.godwin/tmpd2hd5pdaxml.gz' to 'test.xml.gz' ... done.

Here, after the file was copied to a temp space, the original file was read in and written back out. Because the SQL was executed on the temporary copy, nothing ends up being done.

After modifying insert_from_url and extract to take in the working filename rather than the original, we can get back the original behavior:

1/1: using '/local/patrick.godwin/tmp5l4djjzaxml.gz' as workspace
copying 'test.xml.gz' to '/local/patrick.godwin/tmp5l4djjzaxml.gz' ... done.
reading '/local/patrick.godwin/tmp5l4djjzaxml.gz' ...
setting the sqlite temp_store_directory to /local/patrick.godwin ... done
Executing SQL ...
CREATE INDEX tmpindex1 ON coinc_event (coinc_event_id)
CREATE INDEX tmpindex2 ON coinc_event_map (coinc_event_id)
UPDATE coinc_event SET likelihood = 0;
... Done.
writing '/local/patrick.godwin/tmp5l4djjzaxml.gz' ...
moving '/local/patrick.godwin/tmp5l4djjzaxml.gz' to 'test.xml.gz' ... done.

Merge request reports