lw.utils.load_url doesn't work for non-local files
I came across the traceback below when running gstlal_inspiral_marginalize_likelihoods_online from master at CIT, which downloads a rankingstat file from the running inspiral jobs through a wget request, set up on the inspiral side with bottle route. I'm using master of gstlal and python-ligo-lw.
Traceback (most recent call last):
File "/home/gstlalcbc/observing/3/online/code_test_itacac_master_181213/opt/bin/gstlal_inspiral_calc_rank_pdfs", line 102, in <module>
rankingstat = far.marginalize_pdf_urls(urls, "RankingStat", verbose = options.verbose)
File "/home/gstlalcbc/observing/3/online/code_test_itacac_master_181213/opt/lib64/python2.7/site-packages/gstlal/far.py", line 1055, in marginalize_pdf_urls
xmldoc = ligolw_utils.load_url(url, verbose = verbose, contenthandler = RankingStat.LIGOLWContentHandler)
File "/home/gstlalcbc/observing/3/online/code_test_itacac_master_181213/opt/lib64/python2.7/site-packages/ligo/lw/utils/__init__.py", line 408, in load_url
with fileobj() as fileobj:
AttributeError: addinfourl instance has no attribute '__exit__'
Doing some digging, it looks like the object returned by urllib.request.urlopen() on line 407 [1] doesn't have an __exit__
function, but it does appear to have a __close__
function.
[1] https://git.ligo.org/kipp.cannon/python-ligo-lw/blob/master/ligo/lw/utils/init.py#L407
I looked in the urllib.request documentation to see if they provide a function that has an __exit__
function, but I didn't see anything obvious. I hacked in a temporary work around for myself, which I've put below in case anyone else wants to use it, but it's probably not the correct way to fix this.
diff --git a/ligo/lw/utils/__init__.py b/ligo/lw/utils/__init__.py
index a482c1db25..4855ca39cf 100644
--- a/ligo/lw/utils/__init__.py
+++ b/ligo/lw/utils/__init__.py
@@ -403,10 +403,12 @@ def load_url(url, verbose = False, **kwargs):
scheme, host, path = urllib.parse.urlparse(url)[:3]
if scheme.lower() in ("", "file") and host.lower() in ("", "localhost"):
fileobj = lambda: open(path, "rb")
+ with fileobj() as fileobj:
+ xmldoc = load_fileobj(fileobj, **kwargs)
else:
- fileobj = lambda: urllib.request.urlopen(url)
- with fileobj() as fileobj:
+ fileobj = urllib.request.urlopen(url)
xmldoc = load_fileobj(fileobj, **kwargs)
+ fileobj.close()
return xmldoc