SEOBNRv3_opt does not exit gently
I am running lalinference_mcmc on master branch (at ATLAS and ARCCA) with wf generator SEOBNRv3_opt (commit 581360fb). Occasionally there is an MCMC step which gives rise to nan in SEOBNRv3_opt and the generator forces the mcmc job to terminate.
Error messages in lalinference.out:
> Template function called is "LALInferenceTemplateXLALSimInspiralChooseWaveform"
> NAN in derivative! derivs0,1,2,3,4,5,6,7,8,9,10,11 = -nan -nan -nan -nan -nan -nan -nan -nan -nan -nan -nan -nan | divby0 = 0
> NAN in derivative! derivs0,1,2 = -nan -nan -nan | divby0 = 0
> NAN in derivative! derivs0,1,2,3,4,5,6,7,8,9,10,11 = -nan -nan -nan -nan -nan -nan nan nan nan nan nan -nan | divby0 = 1
> NAN in derivative! derivs0,1,2,3,4,5,6,7,8,9,10,11 = -nan -nan -nan -nan -nan -nan nan nan nan nan nan -nan | divby0 = 1
Error messages in lalinference.err:
> XLAL Error - XLALSpinPrecHcapRvecDerivative_exact: nan in the output dvalues
> XLAL Error - XLALInspiralPrecSpinFactorizedFlux_exact: nan polvalues: -nan 0.0000000000 -nan -nan
> XLAL Error - XLALInspiralPrecSpinFactorizedFlux_exact: nan polvalues: -nan 0.0000000000 -nan -nan
> XLAL Error - XLALEOBSpinPrecStopConditionBasedOnPR: nan reached at r2 = -nan
> XLAL Error - XLALSpinPrecHcapRvecDerivative_exact: nan in input values
> gsl: interp.c:150: ERROR: interpolation error
> Default GSL error handler invoked.
> [a0115:45305] *** Process received signal ***
> [a0115:45305] Signal: Aborted (6)
> [a0115:45305] Signal code: (-6)
> [a0115:45305] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xf890) [0x2acfeb860890]
> [a0115:45305] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x37) [0x2acfede1a067]
> [a0115:45305] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x148) [0x2acfede1b448]
> [a0115:45305] [ 3] /usr/lib/libgsl.so.0(+0x78f2d) [0x2acfecd03f2d]
> [a0115:45305] [ 4] /usr/lib/libgsl.so.0(gsl_interp_eval+0x33) [0x2acfecd2d333]
> [a0115:45305] [ 5] /home/kwan-yeung.ng/lscsoft/opt/lalsuite/lib/liblalsimulation.so.16(XLALSimIMRSpinEOBWaveformAll+0x638b) [0x2acfe758462b]
> [a0115:45305] [ 6] /home/kwan-yeung.ng/lscsoft/opt/lalsuite/lib/liblalsimulation.so.16(XLALSimIMRSpinEOBWaveform+0xc7) [0x2acfe759d267]
> [a0115:45305] [ 7] /home/kwan-yeung.ng/lscsoft/opt/lalsuite/lib/liblalsimulation.so.16(XLALSimInspiralChooseTDWaveform+0x11fd) [0x2acfe760a2ad]
> [a0115:45305] [ 8] /home/kwan-yeung.ng/lscsoft/opt/lalsuite/lib/liblalsimulation.so.16(XLALSimInspiralChooseTDWaveformFromCache+0x232) [0x2acfe7664ad2]
> [a0115:45305] [ 9] /home/kwan-yeung.ng/lscsoft/opt/lalsuite/lib/liblalinference.so.13(LALInferenceTemplateXLALSimInspiralChooseWaveform+0x628) [0x2acfe6783298]
> [a0115:45305] [10] /home/kwan-yeung.ng/lscsoft/opt/lalsuite/lib/liblalinference.so.13(+0x29897) [0x2acfe6759897]
> [a0115:45305] [11] /home/kwan-yeung.ng/lscsoft/opt/lalsuite/bin/lalinference_mcmc() [0x40585d]
> [a0115:45305] [12] /home/kwan-yeung.ng/lscsoft/opt/lalsuite/bin/lalinference_mcmc() [0x409073]
> [a0115:45305] [13] /usr/lib/x86_64-linux-gnu/libgomp.so.1(GOMP_parallel+0x3f) [0x2acfedbd87df]
> [a0115:45305] [14] /home/kwan-yeung.ng/lscsoft/opt/lalsuite/bin/lalinference_mcmc() [0x40a0f4]
> [a0115:45305] [15] /home/kwan-yeung.ng/lscsoft/opt/lalsuite/bin/lalinference_mcmc() [0x403c05]
> [a0115:45305] [16] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x2acfede06b45]
> [a0115:45305] [17] /home/kwan-yeung.ng/lscsoft/opt/lalsuite/bin/lalinference_mcmc() [0x403cec]
> [a0115:45305] *** End of error message ***
Edited by Adam Mercer