Increase robustness of iDQ plots/reports
Here are a few tasks to increase the robustness of reports generated from DQR + under the iDQ singularity container.
- Look into getting the plots running without error with usetex = False. This can tend to be problematic with cache issues under DQR and also cause issues when running within the iDQ singularity container.
- Set agg.path.chunksize for iDQ to an appropriate size. Otherwise, the iDQ timeseries jobs will sometimes fail over.
- Decrease the number of plots that get generated by default for the reports. There can be many plots generated over a 1 day period causing DQR jobs to run much longer than should be expected.