Remove ValueError from retries to catch expection when trying to grab external sky map
This MR removes ValueError from being retried on when grabbing an external sky map, which will allow this error to be caught in an exception correctly as intended here.
This MR will partially fix #395 (closed) and addresses this error from Sentry.
Note that this partially reverses the change from 2da93771