Skip to content

Some aesthetic/performance changes for reports

Patrick Godwin requested to merge report_stuff into master

This MR adds a few things:

  • Fix an issue with importing html if neither the python2 or python3 libraries were available, due to how I structured the try/except loop.
  • Add an offset option to CadenceManager in utils.py. This allows some finer control over the reporting strides, allowing us to align to the same gps ranges as the DQ summary pages.
  • I cut the row height in the OVL feature importance plots by a factor of 2, by moving all the veto information to a single line. Pics below. It still feels a bit long but eh, what can you do. This will get around an issue sometimes where it crashes in the middle of saving a plot to disk from the plot being too long.
  • I removed the transform in the logL timeseries, opting instead for a symlog scale with a linear region around [-10, 10]. Pics below. This one is a bit more superfluous but I kinda like seeing the raw logL values better. What do you think?
  • Added an option for reports, single_calib_plots which defaults to True but turns off the individual calibration coverage/distribution plots for the single classifier section. I was looking at the streaming report logs, and it looks like a single report sometimes takes 4+ hours creating all these plots just due to the number of calibration maps produced (100+). I think we should run with these off in the online runs to reduce the time to create reports.
  • Generate feature importance figures one at a time rather than all at once, which cuts down on the memory usage. In production, I see the memory usage being 50+ GB sometimes while when I was running it with this change, I didn't see it go above 10 GB.
Before After

Example report: https://ldas-jobs.ligo-la.caltech.edu/~patrick.godwin/idq/test_report_formatting/START-12618/1261864818_1261951218/report-L1iDQonline.html

Merge request reports

Loading