Fix automatic grabbing of Fermi sky maps
Currently when running external_skymaps.get_external_skymap
we get
[2023-05-09 13:45:08,430: INFO/MainProcess/MainThread] Task gwcelery.tasks.external_skymaps.get_external_skymap[e0956a4b-4bb8-4d62-92ed-8256a2586993] received
[2023-05-09 13:45:08,431: INFO/ForkPoolWorker-1/MainThread] Task gwcelery.tasks.external_skymaps.get_external_skymap[e0956a4b-4bb8-4d62-92ed-8256a2586993] retry: Retry in 23s: <HTTPError 404: 'Not Found'>
[2023-05-09 13:45:08,435: ERROR/MainProcess/MainThread] Task handler raised error: <MaybeEncodingError: Error sending result: '"(1, <ExceptionInfo: Retry(Retry(...), <HTTPError 404: 'Not Found'>, 23)>, None)"'. Reason: ''TypeError("cannot pickle \'_io.BufferedReader\' object")''.>
Traceback (most recent call last):
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/site-packages/billiard/pool.py", line 366, in workloop
put((READY, (job, i, result, inqW_fd)))
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/site-packages/billiard/queues.py", line 366, in put
self.send_payload(ForkingPickler.dumps(obj))
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/site-packages/billiard/reduction.py", line 56, in dumps
cls(buf, protocol).dump(obj)
billiard.pool.MaybeEncodingError: Error sending result: '"(1, <ExceptionInfo: Retry(Retry(...), <HTTPError 404: 'Not Found'>, 23)>, None)"'. Reason: ''TypeError("cannot pickle \'_io.BufferedReader\' object")''.
[2023-05-09 13:45:32,237: ERROR/ForkPoolWorker-1/MainThread] Task gwcelery.tasks.external_skymaps.get_external_skymap[e0956a4b-4bb8-4d62-92ed-8256a2586993] raised unexpected: OSError()
Traceback (most recent call last):
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/site-packages/celery/app/trace.py", line 451, in trace_task
R = retval = fun(*args, **kwargs)
File "/home/emfollow-test/.local/lib/python3.9/site-packages/sentry_sdk/integrations/celery.py", line 207, in _inner
reraise(*exc_info)
File "/home/emfollow-test/.local/lib/python3.9/site-packages/sentry_sdk/_compat.py", line 60, in reraise
raise value
File "/home/emfollow-test/.local/lib/python3.9/site-packages/sentry_sdk/integrations/celery.py", line 202, in _inner
return f(*args, **kwargs)
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/site-packages/celery/app/trace.py", line 734, in __protected_call__
return self.run(*args, **kwargs)
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/site-packages/celery/app/autoretry.py", line 54, in run
ret = task.retry(exc=exc, **retry_kwargs)
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/site-packages/celery/app/task.py", line 717, in retry
raise_with_context(exc)
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/site-packages/celery/app/autoretry.py", line 34, in run
return task._orig_run(*args, **kwargs)
File "/home/emfollow-test/.local/lib/python3.9/site-packages/gwcelery/tasks/external_skymaps.py", line 289, in get_external_skymap
return urllib.request.urlopen(skymap_link, context=context).read()
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/urllib/request.py", line 214, in urlopen
return opener.open(url, data, timeout)
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/urllib/request.py", line 523, in open
response = meth(req, response)
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/urllib/request.py", line 632, in http_response
response = self.parent.error(
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/urllib/request.py", line 555, in error
result = self._call_chain(*args)
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/urllib/request.py", line 494, in _call_chain
result = func(*args)
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/urllib/request.py", line 747, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/urllib/request.py", line 523, in open
response = meth(req, response)
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/urllib/request.py", line 632, in http_response
response = self.parent.error(
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/urllib/request.py", line 561, in error
return self._call_chain(*args)
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/urllib/request.py", line 494, in _call_chain
result = func(*args)
File "/cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-py39-20221118/lib/python3.9/urllib/request.py", line 641, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
OSError