Fix Numpy 1.20.0 by initializing PyArray_Descr portably

Description

According to the Numpy documentation:

To ensure compatibility:

• Never declare a non-pointer instance of the struct
• Never perform pointer arithmatic
• Never use sizof(PyArray_Descr)

The SWIG bindings broke the first of these rules. Improper order of initialization led to a null pointer that was causing a segfault.

Fixes #414 (closed). (We shall see!)

API Changes and Justification

Backwards Compatible Changes

• This change introduces no API changes
• This change adds new API calls

Backwards Incompatible Changes

• This change modifies an existing API
• This change removes an existing API

If any of the Backwards Incompatible check boxes are ticked please provide a justification why this change is necessary and why it needs to be done in a backwards incompatible way.

Review Status

N/A

assigned to @karl-wette

added 3 commits

• 6b8cdd1c - Fix Numpy 1.20.0 by initializing PyArray_Descr portably
• 4faed458 - Revert "lal: pin runtime numpy to <1.20.0a0"
• 241a2530 - WIP: temporarily print error message

Compare with previous version

marked this merge request as draft from leo-singer/lalsuite@241a2530

added infra_general infra_swig lal + 1 deleted label

added 3 commits

• 38619779 - Fix Numpy 1.20.0 by initializing PyArray_Descr portably
• 4dba49d1 - Revert "lal: pin runtime numpy to <1.20.0a0"
• 3bd32d78 - WIP: temporarily print error message

Compare with previous version

added 2 commits

• 733b9886 - Fix Numpy 1.20.0 by initializing PyArray_Descr portably
• 1e55e1d1 - Revert "lal: pin runtime numpy to <1.20.0a0"

Compare with previous version

Oh, well. This didn't work :-(

@leo-singer I think this is still on the right track; the segfault is no longer happening at import lal time (which I understand was the previous error), but later on during the tests. So it may be something in your changes to the dtype that need to be fixed. I'll see if I can reproduce this locally per @duncanmmacleod 's instructions.

@leo-singer @adam-mercer @duncanmmacleod I think I may now have a fix. @leo-singer was on the right track, just needed a few tweaks - and a critical Py_INCREF() restored. I'll test this on my own fork first, if it works I'll see if I can update this MR.

added 34 commits

• 1e55e1d1...3d9aa6ce - 31 commits from branch lscsoft:master
• f2251ce1 - Fix Numpy 1.20.0 by initializing PyArray_Descr portably
• e60ee429 - Revert "lal: pin runtime numpy to <1.20.0a0"
• 57242de5 - Fixes to "Fix Numpy 1.20.0 by initializing PyArray_Descr portably"

Compare with previous version

marked this merge request as ready

assigned to @duncanmmacleod and @adam-mercer and unassigned @karl-wette

@duncanmmacleod @adam-mercer This should now be ready to merge, once the pipeline passes. (Already tested it passes on my own fork.)

Thanks! I've started a pipeline that builds some wheels as well.

approved this merge request

enabled an automatic merge when the pipeline for 57242de5 succeeds

@adam-mercer Looks like this won't merge as lalinference_testjob is broken: https://git.ligo.org/leo-singer/lalsuite/-/jobs/1184235

@adam-mercer Looks like this won't merge as lalinference_testjob is broken: https://git.ligo.org/leo-singer/lalsuite/-/jobs/1184235

That should be fixed by !1539 (merged).

merged

mentioned in issue bilby#555 (closed)

added pkg_general + 1 deleted label

added pkg_wheel + 1 deleted label and removed infra_general pkg_general labels

added pkg_conda label