Aligned spin conversion fails when secondary spin is zero.
We currently check if spins are aligned but not if they're zero. So when one spin is zero and the other is aligned it does the full precessing spin conversion which introduces numerical noise in the primary in-plane spin components.
An example of a failing prior is
chi_1 = bilby.gw.prior.AlignedSpin()
chi_2 = 0