Unpin the astropy dependency

In !1008 (merged) the astropy version was pinned to avoid issues with gwpy, we should take this pin out once the upstream issue is resolved.