Kafka producer bootstep

Addresses #385 (closed).

This is a WIP. In particular, I'm not sure if the send() method should live inside the Producer, and what other arguments it should have.

The Kafka bootstrap configuration is also currently a placeholder.

To-dos:

• Add new packages as dependencies (currently suffering from the htcondor dependency issue on Macs)
• Choose between python-confluent-kafka, adc-streaming, and hop-client (or some combination) [updated Feb 7, 2022]
• Add correct host to configuration

changed milestone to %Review Readiness Presentation

added Kafka Alerts label

added 1 commit

• dba3b901 - Initial commit for Kafka producer bootstep

Compare with previous version

changed the description

mentioned in merge request !852 (merged)

resolved all threads

added 1 commit

• 459dc937 - Initial commit for Kafka producer bootstep

Compare with previous version

changed the description

added 1 commit

• cb0b0269 - Switch from python-kafka to confluent-python-kafka

Compare with previous version

added 4 commits

• 393f0708 - 1 commit from branch emfollow:main
• 9d29270e - Initial commit for Kafka producer bootstep
• b7b89811 - Switch from python-kafka to confluent-python-kafka
• 033df7a8 - Use hop client for Kafka implementation

Compare with previous version

BTW @geoffrey.mo I was taking a look at your checklist about the htcondor issue. I just wanted to mention that the issue was coming from ligo.em-bright having htcondor as a dependency. I have made a new release (0.1.6) that takes this out. Could you check if that resolves the issue

added 4 commits

• 038600ad - 1 commit from branch emfollow:main
• 697f9e37 - Initial commit for Kafka producer bootstep
• c7c0aa0f - Switch from python-kafka to confluent-python-kafka
• 5ee79d32 - Use hop client for Kafka implementation

Compare with previous version

added 1 commit

• 36b5cb6f - Update kafka bootstep to be more modular

Compare with previous version

resolved all threads

resolved all threads