Skip to content

reed's assigned tasks for streaming development

This merge request should accomplish the following:

  • implemented a parser for Kafka connections within configparser.evalkwargs, setting up a conditional looking for the special option name: kafka__source
  • fix a bug in how TimeMap assigned default "delta_time" values. Now, all values in the returned vector should be set to the default value when appropriate, rather than the difference between the vector's gps time and the default value.
  • make CalibrationMap fundamentally incremental, so values can be added and removed from it with quickly updated interpolation objects stored internally. This prevents us from having to recompute the stored interpolation objects for all observations when we make small additions or subtractions from the data set. CalibrationMap now also contains auto_optimize logic, so the user just has to continue to add/remove observations and the high-level class method will automatically decide whether the underlying KDE's need to be re-optimized to find a better bandwidth. The changes to CalibrationMap are also reflected within CalibrationMapReporter.

I've tested these changes somewhat. The kafka parser in configparser is so straightforward that I didn't bother testing it; this will likely bite me later on. TimeMap seems to not raise any errors, although I don't have a specific test set up to prove it does the correct thing. I've tested CalibrationMap's changes quite a bit more. Below, I show the results for both the coverage of the pdf and cdf for both example normal and beta distributions (see below). The coverage remains excellent. I also confirmed that I can add and remove samples and obtain the same interpolation object values as before, meaning there is not systematic error. This required some careful handling of high-precision math with logpdf interpolation objects.

normal pdf normal cdf beta pdf beta cdf

Edited by Reed Essick

Merge request reports