Skip to content

Fixes to kafka2lsmp from tests at UWM

Patrick Brockill requested to merge (removed):kafka2lsmp_fixes_round1 into main

A number of fixes to get kafka2lsmp running properly at UWM:

  • A section in framekafkaconsumer.py's indentation was incorrect, leading to the fast-forwarding mechanism not working (leading to quite a bit of unnecessary chatting between the consumer and the broker(s) if the streams get behind or data is lost).

  • For some reason on RHEL 7/8 systems, "import importlib", then "importlib.util" does not work, but "import importlib.util" works fine. The former method seems to work elsewhere. No idea what's up here.

  • A small omission in the lsmp connect() function in kafka2lsmp.py was needed (otherwise, the client fails to connect to the shared memory).

  • Added importing "gds.pygdsio" back in, as these libraries have now been compiled and widely distributed on LIGO clusters.

Merge request reports