On a model crash local_dc exited with an error of 'free(): invalid pointer'
See FRS:
When the models crashed local_dc crashed as well. Notice there had been no messages in about the last month.
Sep 17 14:05:03 h1susey local_dc[1572]: Model h1sustmsy rate = 16384 dcuid = 99 Sep 17 14:05:03 h1susey local_dc[1572]: Model h1susetmypi rate = 65536 dcuid = 127 Oct 15 16:26:50 h1susey local_dc[1572]: free(): invalid pointer Oct 15 16:26:50 h1susey systemd[1]: rts-local_dc.service: Main process exited, code=killed, status=6/ABRT Oct 15 16:26:50 h1susey systemd[1]: rts-local_dc.service: Failed with result 'signal'. Oct 15 16:26:55 h1susey systemd[1]: rts-local_dc.service: Service RestartSec=5s expired, scheduling restart. Oct 15 16:26:55 h1susey systemd[1]: rts-local_dc.service: Scheduled restart job, restart counter is at 13. Oct 15 16:26:55 h1susey systemd[1]: Stopped Advanced LIGO RTS local data concentrator. Oct 15 16:26:55 h1susey systemd[1]: Started Advanced LIGO RTS local data concentrator.
I am noting that cps_xmit also failed. Again note that we saw a large amount of time with no issues prior to this crash.
Oct 08 16:35:11 h1susey cps_xmit[19800]: Client running refcount 1 Oct 15 16:26:50 h1susey cps_xmit[19800]: Closing client Oct 15 16:26:50 h1susey cps_xmit[19800]: Closing client Oct 15 16:26:50 h1susey cps_xmit[19800]: Closing out OpenMX and exiting Oct 15 16:26:50 h1susey systemd[1]: rts-transport@cps_xmit.service: Main process exited, code=exited, status=1/FAILURE Oct 15 16:26:50 h1susey systemd[1]: rts-transport@cps_xmit.service: Failed with result 'exit-code'. Oct 15 16:26:55 h1susey systemd[1]: rts-transport@cps_xmit.service: Service RestartSec=5s expired, scheduling restart. Oct 15 16:26:55 h1susey systemd[1]: rts-transport@cps_xmit.service: Scheduled restart job, restart counter is at 1. Oct 15 16:26:55 h1susey systemd[1]: Stopped Advanced LIGO RTS DAQD transport. Oct 15 16:26:55 h1susey systemd[1]: Started Advanced LIGO RTS DAQD transport. Oct 15 16:26:55 h1susey cps_xmit[5413]: /usr/bin/cps_xmit compiled Aug 14 2020 : 16:21:37 Oct 15 16:26:55 h1susey cps_xmit[5413]: Writing DAQ data to local shared memory and sending out on the publisher Oct 15 16:26:55 h1susey cps_xmit[5413]: ifo mapped to 0x7f5d7c541000 Oct 15 16:26:56 h1susey cps_xmit[5413]: Making mbuff area local_dc with size 104857600 Oct 15 16:26:56 h1susey cps_xmit[5413]: local_dc mmapped address is 0x7f5d7c541000 Oct 15 16:26:56 h1susey cps_xmit[5413]: Client running refcount 1 Oct 15 17:11:22 h1susey cps_xmit[5413]: Closing client Oct 15 17:11:22 h1susey cps_xmit[5413]: Closing client Oct 15 17:11:22 h1susey cps_xmit[5413]: Closing out OpenMX and exiting