Simplify stream workflow
Compare changes
Files
11@@ -152,36 +152,12 @@ As such, you will also have to manage a config file for your synthetic data in a
This PR attempts to simplify the streaming workflow, removing functionality that is unused, reducing complexity and addressing various LSP warnings. In particular:
KafkaReporter
. This implementation had its host of problems and we switched to using the various DiskReporters for O3. Instead of improving its implementation, we call the DiskReporter-based implementations 'good enough' for use.utils.py
. This was only needed due to pickling (why?) but since we don't need to support the fork workflow, this is not needed anymore. Instead, fold in the relevant functionality to the single place that uses it, SNAXKafkaDataLoader
.logger
to StreamProcessor
, as it's pointless to do so.SNAXKafkaDataLoader
.StreamProcessor
tests.Closes #67 (closed).