Organization for report generation (and related) development

Louis and I agreed to use a top-level issue to organize our work on report generation. This should avoid generating a giant flood of issues without prioritization. I have moved over every issue I could find from (a) the calibration F2F notes and (b) the issue tracker on Louis's fork of pydarm, and organized them as best I could, starting with high priorities and low-hanging fruit.

How we'll use this document:

When we start working on an item from this list, we will:

• create a specific issue for it with any relevant details
• assign the appropriate person to that issue
• link the issue to the appropriate item on this document
• bold the checklist item for visiblity

When the issue is closed, we will mark the item on this list as completed too.

More detail for many items can be found in the calibration F2F meeting notes. Please point out any items that have been missed, suggest wording edits, or cross-link to other issues.

1. Small and clearly defined issues and/or bug fixes

Bug fixes

• Fix percentage conversion issue -> #187

Data to display

• Show H_C values in both units (ct/m and mA/pm) -> claimed by Louis
• Show the reference model values at the top (specifically ‘free’ parameters from reference model) -> claimed by Ansel

Plot visual presentation

• magnitude plot: full log on the left, but narrowed range for residual -> #187
• phase plot: full 180 range on the left plot, but narrow/adaptive axes on the right -> #187
• Font size for axis tick labels -> #187
• Corner plot visual spacing, get rid of whitespace -> #187
• Different marker shapes for different data series -> #187

Organization and labeling

• Switch to new xml format specified at F2F meeting -> claimed by Louis
• LN -> UIM, PUM, TST

Other

• Put existing plots into the correct order in the PDF (see plan in cal F2F notes) -> claimed by Louis
• Fix variable names in sensing to align with actuation section
• If user puts in a fit region that goes outside the data, raise a warning for the user -> claimed by Louis

2. More in-depth or less well defined changes

• TeX the parameters table (in this section because: non-monospace table alignment)
• Better way to handle y-axis scaling parameter?
• Put in a burn in trace in the white space on the corner plots (top right) #193
• Issue warning if loop and pcal measurements are not close in time #179
• Related to above, consider labeling measurement time stamps and/or duration between them on the plots to make unusual time gaps visible
• Do we also need the vertical dashed lines for the fit region on the history plots?

3. Additional plots (please see cal F2F notes for context - not all info here)

• Another plot that Jeff thought of: broadband measurement comparing PCAL2DeltaLExternal, the correction that pydarm predicts, and their residual
• Yet another plot Jeff thought of: new final answer
• Crossover plots (Joe's suggestion) - take one ini file -> give it 1 count of darm count, then see what happens to UIM, PUM, TST, Total A, 1/C
• "stack" rather than "overlay" measurements (Example of stacking code as referenced by this alog)

4. Measurement management items

These items cannot be fully addressed without further discussion/planning for how to manage and organize measurement and files.

• Interface with git to get ini files by date/time range
• Load MCMC results from json (not as simple as literally loading a json - requires a way to find the correct json file.)
• Add ability to exclude measurement set (tied to validation related items below)
• Add option for a custom string to tag output files (or custom directory in new scheme?) (helpful for exploratory/investigative plots)
• Make a set of measurements go into an existing timestamped directory, e.g. if measurement is paused and re-started
• Add a way to log things that happened during measurement (earthquakes, wind, arm power) (maybe keep a notes doc in the measurement directory?)

5. "Validation" and decision-making items

• Create a way for user to indicate whether data should be archived (did it run successfully?)
• Create a way for user to indicate whether data looks ok (distinct from above)
• Create a way for user indicate whether the model should be changed
• If data ok and model does not need to be changed, indicate somehow that data should be used for GPR
• Report should indicate if free parameters have changed from reference (need to specify threshold for this alert)

6. Interaction/incorporation with other tools

• Integrate GPR code. GPR should run after each measurement/MCMC. First check that the current measurement falls within expectation from previous GPR. Next run GPR using current measurement as part of the set
• Uncertainty: if the model needs to be updated, we need to update soft links to current GPR, current reference MCMC params, etc.
• Check front-end configuration variables (free parameters)