Some aesthetic/performance changes for reports
This MR adds a few things:
- Fix an issue with importing
html
if neither the python2 or python3 libraries were available, due to how I structured the try/except loop. - Add an
offset
option toCadenceManager
inutils.py
. This allows some finer control over the reporting strides, allowing us to align to the same gps ranges as the DQ summary pages. - I cut the row height in the OVL feature importance plots by a factor of 2, by moving all the veto information to a single line. Pics below. It still feels a bit long but eh, what can you do. This will get around an issue sometimes where it crashes in the middle of saving a plot to disk from the plot being too long.
- I removed the transform in the logL timeseries, opting instead for a symlog scale with a linear region around [-10, 10]. Pics below. This one is a bit more superfluous but I kinda like seeing the raw logL values better. What do you think?
- Added an option for reports,
single_calib_plots
which defaults to True but turns off the individual calibration coverage/distribution plots for the single classifier section. I was looking at the streaming report logs, and it looks like a single report sometimes takes 4+ hours creating all these plots just due to the number of calibration maps produced (100+). I think we should run with these off in the online runs to reduce the time to create reports. - Generate feature importance figures one at a time rather than all at once, which cuts down on the memory usage. In production, I see the memory usage being 50+ GB sometimes while when I was running it with this change, I didn't see it go above 10 GB.
Before | After |
---|---|
Example report: https://ldas-jobs.ligo-la.caltech.edu/~patrick.godwin/idq/test_report_formatting/START-12618/1261864818_1261951218/report-L1iDQonline.html