igwn-alert seems to have a memory leak
To demonstrate the problem we used the dqr account which is subscribed to these topics:
Topics for group gracedb-playground associated with the current credential:
mdc_superevent
superevent
In one window run a listener:
$ igwn-alert -g gracedb-playground listen
In another window monitor the memory usage:
out="${HOME}/alert-mem.txt"
echo "Size, RSS, VSZ" >>${out}
while [ 1 ]
do
ps -eo pid,ppid,sz:12,rssize,vsize | grep "${pids}"|awk '{print $3, $4, $5;}' >>${out}
sleep 300
done
This script monitors memory usage and shows a steady growth:
The code responsible is https://git.ligo.org/computing/igwn-alert/client/-/blob/main/igwn_alert/client.py#L226
for payload, metadata in s.read(
metadata=True,
batch_size=self.batch_size,
batch_timeout=self.batch_timeout):
This calls hop.io.read which looks like: https://github.com/scimma/hop-client/blob/012f41381c47a30df03d79ad521564be4c58924f/hop/io.py#L342
for message in self._consumer.stream(autocommit=autocommit, **kwargs):
if self.ignoretest and self.is_test(message):
continue
yield self._unpack(message, metadata=metadata)