Skip to content

Reduce finalsink latency by processing buffers for current time once we have all of them

Andrew Gozzard requested to merge agozzard__finalsink_opportunistic_clustering into spiir-O4-EW-development

Resolves #44 (closed)

FinalSink currently waits until it receives a buffer from time t+1 before processing buffers from time t. By keeping track of how many buffers we have received for time t we can instead process them as soon as we have them all. This involves some bookkeeping and tweaks to be compatible with the existing code.

Still handles the possibility of a missing or extremely delayed buffer even though we aren't sure how possible that is in reality (at least for a live run).

Currently a draft to get suggestsions on remaining TODOs (particularly readable variable names) and feedback on logic and the way it has been hacked into finalsink.

TODO:

  • Comment(s) explaining the bookkeeping
  • Comment(s) explaining how/why it's safe to skip waiting for the next buffer?
  • Better variable name for have_all_buffers_for_current_timestamp
  • Better variable name for latest_processable_data_time
Edited by Andrew Gozzard

Merge request reports