Skip to content

GStreamer 1.0: Allocate snglsnr memory once only

Timothy Davies requested to merge tdavies__reduce_postcoh_memcpy into gstreamer_python_upgrade

Related to #95

Similar to !139 (merged). This change should mean that we only allocate memory for SNR in postcoh once, rather than once per buffer.

Tests

Latest

I've had a background run going for 15 hours, along with one on !139 (merged) and one on gstreamer_python_upgrade.

The runs didn't launch at exactly the same time, and I've only been comparing now and then (recording runtime vs livetime in marginalized stats), but the improvement seems to be about 1.4x. It's enough that I'm confident this is a good improvement, but don't take that as a precise measure of how much faster a run on CIT will be for the full duration of the run, for example.

I've also run gw170817 with Warp Reduce enabled. It's deterministic, but slightly different to the last MR. However, it's Also identical to !139 (merged) with warp reduce. So I Think a dependency in conda may have changed slightly. I'll do a new run of gstreamer_python_upgrade to confirm.

gw170817

(Copy from !139 (merged), I've only done 1 run for !140 (merged)) I've also compiled with warp reduce and several runs. All got the same results, but they're different to my Old run of MR137. However, I've rerun MR137 (with !138 (merged) merged in) and it now gets the same results as !139 (merged) and !140 (merged). I'm guessing it's due to a dependency change in conda.

Old

As in !139 (merged), but I've not tried gw170817 with warp reduce yet. Performance improvement Seems to be ~2x from some tests before merging !138 (merged).

I've started a new run, but one node quickly ran into a segfault. I'm doing a new run in case it's an unrelated/transient error. (I wasn't running with gdb, and the backtrace isn't particularly informative:

Program terminated with signal 6, Aborted.
#0  0x0000151068669387 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.17-326.el7_9.x86_64
(gdb) backtrace
#0  0x0000151068669387 in raise () from /lib64/libc.so.6
#1  0x000015106866aa78 in abort () from /lib64/libc.so.6
#2  0x00001510686abf67 in __libc_message () from /lib64/libc.so.6
#3  0x00001510686b4329 in _int_free () from /lib64/libc.so.6
#4  0x0000150fee3f3356 in ?? () from /fred/oz016/tdavies/projects/gstreamer_python_upgrade/dependencies/lib/gstreamer-1.0/libgstframecpp.so
#5  0x0000150fee3f3627 in ?? () from /fred/oz016/tdavies/projects/gstreamer_python_upgrade/dependencies/lib/gstreamer-1.0/libgstframecpp.so
#6  0x0000150fee3f36d9 in ?? () from /fred/oz016/tdavies/projects/gstreamer_python_upgrade/dependencies/lib/gstreamer-1.0/libgstframecpp.so
#7  0x0000150fee3f3be5 in ?? () from /fred/oz016/tdavies/projects/gstreamer_python_upgrade/dependencies/lib/gstreamer-1.0/libgstframecpp.so
#8  0x0000150fee3f5691 in ?? () from /fred/oz016/tdavies/projects/gstreamer_python_upgrade/dependencies/lib/gstreamer-1.0/libgstframecpp.so
#9  0x0000151060f564cc in gst_pad_chain_data_unchecked ()
   from /fred/oz016/envs/mambaforge/envs/igwn-py38-testing-spiir/lib/python3.8/site-packages/gi/../../../././libgstreamer-1.0.so.0
#10 0x0000151060f587ba in gst_pad_push_data ()
   from /fred/oz016/envs/mambaforge/envs/igwn-py38-testing-spiir/lib/python3.8/site-packages/gi/../../../././libgstreamer-1.0.so.0
#11 0x0000151060f60012 in gst_pad_push ()
   from /fred/oz016/envs/mambaforge/envs/igwn-py38-testing-spiir/lib/python3.8/site-packages/gi/../../../././libgstreamer-1.0.so.0
#12 0x00001510537fabce in gst_base_src_loop ()
   from /fred/oz016/envs/mambaforge/envs/igwn-py38-testing-spiir/lib/python3.8/site-packages/gi/../../.././././libgstbase-1.0.so.0
#13 0x0000151060f90bd1 in gst_task_func ()
   from /fred/oz016/envs/mambaforge/envs/igwn-py38-testing-spiir/lib/python3.8/site-packages/gi/../../../././libgstreamer-1.0.so.0
#14 0x0000151061d22027 in g_thread_pool_thread_proxy ()
   from /fred/oz016/envs/mambaforge/envs/igwn-py38-testing-spiir/lib/python3.8/site-packages/gi/../../../libglib-2.0.so.0
#15 0x0000151061d21730 in g_thread_proxy ()
   from /fred/oz016/envs/mambaforge/envs/igwn-py38-testing-spiir/lib/python3.8/site-packages/gi/../../../libglib-2.0.so.0
#16 0x0000151069319ea5 in start_thread () from /lib64/libpthread.so.0
#17 0x0000151068731b0d in clone () from /lib64/libc.so.6
(gdb)
Edited by Timothy Davies

Merge request reports