Memory deallocation issue in itacac/snglinspiraltable
Analysis: Mario O3 replay
One of the inspiral jobs has failed with the following message in the err file. The rest of the analysis/jobs are still running.
Path to .err on CIT: /home/gstlalcbc/observing/4/dev/runs/trigs.mario_O3_replay/logs/gstlal_inspiral_00361-14565618-0.err
WARNING: Overriding HOME environment variable with SINGULARITYENV_HOME is not permitted
WARNING: While bind mounting '/home/gstlalcbc/observing/4/dev/runs/trigs.mario_O3_replay:/home/gstlalcbc/observing/4/dev/runs/trigs.mario_O3_replay': destination is already in the mount point list
WARNING: While bind mounting '/home:/home': destination is already in the mount point list
WARNING: While bind mounting '/hdfs:/hdfs': destination is already in the mount point list
WARNING: While bind mounting '/archive:/archive': destination is already in the mount point list
WARNING: While bind mounting '/cvmfs:/cvmfs': destination is already in the mount point list
WARNING: While bind mounting '/scratch/lalsimulation:/scratch/lalsimulation': destination is already in the mount point list
/usr/bin/gstlal_inspiral:465: UserWarning: It is not advised to choose PSD lengths below 32s, since a loss in sensitivity may occur
warnings.warn("It is not advised to choose PSD lengths below 32s, since a loss in sensitivity may occur")
/usr/lib64/python3.6/site-packages/gstlal/httpinterface.py:123: UserWarning: disabling service discovery, this web server won't be able to advertise the location of the services it provides.
warnings.warn("disabling service discovery, this web server won't be able to advertise the location of the services it provides.")
*** Error in `python3': free(): invalid pointer: 0x00002ab0ac015020 ***
======= Backtrace: =========
/usr/lib64/libc.so.6(+0x81329)[0x2aafec8e7329]
/usr/lib64/python3.6/site-packages/gstlal/_snglinspiraltable.so(+0x4630)[0x2ab01c12e630]
/usr/lib64/libpython3.6m.so.1.0(_PyCFunction_FastCallDict+0x2aa)[0x2aafebb00efa]
/usr/lib64/libpython3.6m.so.1.0(+0x15066f)[0x2aafebb6b66f]
/usr/lib64/libpython3.6m.so.1.0(_PyEval_EvalFrameDefault+0x3a7)[0x2aafebb5fdb7]
/usr/lib64/libpython3.6m.so.1.0(_PyFunction_FastCallDict+0x15a)[0x2aafebb6c92a]
/usr/lib64/libpython3.6m.so.1.0(_PyObject_FastCallDict+0x10e)[0x2aafebac31fe]
/usr/lib64/libpython3.6m.so.1.0(PyObject_CallFunctionObjArgs+0xe2)[0x2aafebac4692]
/usr/lib64/libpython3.6m.so.1.0(+0xb6ffc)[0x2aafebad1ffc]
/usr/lib64/libpython3.6m.so.1.0(_PyObject_GenericSetAttrWithDict+0xcf)[0x2aafebb0299f]
/usr/lib64/libpython3.6m.so.1.0(PyObject_SetAttr+0x5d)[0x2aafebb02b3d]
/usr/lib64/libpython3.6m.so.1.0(_PyEval_EvalFrameDefault+0x43aa)[0x2aafebb63dba]
/usr/lib64/libpython3.6m.so.1.0(+0x14fa33)[0x2aafebb6aa33]
/usr/lib64/libpython3.6m.so.1.0(+0x15050a)[0x2aafebb6b50a]
/usr/lib64/libpython3.6m.so.1.0(+0x150793)[0x2aafebb6b793]
/usr/lib64/libpython3.6m.so.1.0(_PyEval_EvalFrameDefault+0x3a7)[0x2aafebb5fdb7]
/usr/lib64/libpython3.6m.so.1.0(+0x14fa33)[0x2aafebb6aa33]
/usr/lib64/libpython3.6m.so.1.0(+0x15050a)[0x2aafebb6b50a]
/usr/lib64/libpython3.6m.so.1.0(+0x150793)[0x2aafebb6b793]
/usr/lib64/libpython3.6m.so.1.0(_PyEval_EvalFrameDefault+0x3a7)[0x2aafebb5fdb7]
/usr/lib64/libpython3.6m.so.1.0(_PyFunction_FastCallDict+0x15a)[0x2aafebb6c92a]
/usr/lib64/libpython3.6m.so.1.0(_PyObject_FastCallDict+0x10e)[0x2aafebac31fe]
/usr/lib64/libpython3.6m.so.1.0(_PyObject_Call_Prepend+0x61)[0x2aafebac3311]
/usr/lib64/libpython3.6m.so.1.0(PyObject_Call+0x43)[0x2aafebac2f83]
/usr/lib64/python3.6/site-packages/gi/_gi.cpython-36m-x86_64-linux-gnu.so(+0x1c67a)[0x2aaff1abe67a]
/usr/lib64/libgobject-2.0.so.0(g_closure_invoke+0x138)[0x2aaff2136a18]
/usr/lib64/libgobject-2.0.so.0(+0x220fd)[0x2aaff21490fd]
/usr/lib64/libgobject-2.0.so.0(g_signal_emit_valist+0xafc)[0x2aaff2150d7c]
/usr/lib64/libgobject-2.0.so.0(g_signal_emit+0x8f)[0x2aaff215137f]
/usr/lib64/libgstapp-1.0.so.0(+0x8b67)[0x2ab032ffab67]
/usr/lib64/libgstbase-1.0.so.0(+0x35ad1)[0x2ab000b64ad1]
/usr/lib64/libgstbase-1.0.so.0(+0x36db4)[0x2ab000b65db4]
/usr/lib64/libgstreamer-1.0.so.0(+0x772c5)[0x2aaff36eb2c5]
/usr/lib64/libgstreamer-1.0.so.0(gst_pad_push+0x10e)[0x2aaff36f35ae]
/usr/lib64/libgstbase-1.0.so.0(+0x18eb3)[0x2ab000b47eb3]
/usr/lib64/gstreamer-1.0/libgstgstlalinspiral.so(+0x5f01)[0x2ab01f859f01]
/usr/lib64/gstreamer-1.0/libgstgstlalinspiral.so(+0x83a8)[0x2ab01f85c3a8]
/usr/lib64/libgstbase-1.0.so.0(+0x198ed)[0x2ab000b488ed]
/usr/lib64/libgstreamer-1.0.so.0(+0xac5a9)[0x2aaff37205a9]
/usr/lib64/libglib-2.0.so.0(+0x73f4c)[0x2aaff23ebf4c]
/usr/lib64/libglib-2.0.so.0(+0x735b0)[0x2aaff23eb5b0]
/usr/lib64/libpthread.so.0(+0x7ea5)[0x2aafebf48ea5]
/usr/lib64/libc.so.6(clone+0x6d)[0x2aafec9649fd]
Edited by Patrick Godwin