Skip to content

Overhaul logging in programs

This builds off of !48 (closed) with a few changes:

  • Only configure the logger within the applications themselves. This allows downstream programs to configure the logger in the way they wish while still being able to access all of the logging messages provided by this library.
  • Define --verbose/--quiet CLI arguments to control the log level in addition to using $LOG_LEVEL that set the log level to DEBUG and WARNING, respectively.
  • Clean up logging messages throughout, using the native log formatting so that string formatting is deferred until the logs are printed. This avoids unnecessary formatting if the log level won't print the messages to stderr.
  • Remove any toggles to args.verbose for some logging messages, instead putting these messages under logging.debug so that they can all be controlled through a single log level.
  • Configure non-root loggers not managed by this library as part of log configuration. Since we configure the root logger and rely on propagation (https://docs.python.org/3/library/logging.html#logging.Logger.propagate) to configure all loggers, some extraneous messages from kafka-python and watchdog show up at the DEBUG level. We set these to only show up at the WARNING or ERROR level to avoid extraneous logs.

CI failures are unrelated, but we should address these at some point:

  • python3.6 test failure - due to gpstime leapsecond file
  • conda-forge build - due to kafka-python not supporting python 3.12
Edited by Patrick Godwin

Merge request reports