Record online registration state
Current --online
operation:
- Look in diskcache for files with GPS time >
minimum-gps
(default: 0) - Get complete list of DIDs in datasets with matching frame time based on dataset name
- Compare lists to see what needs to be registered.
When we have a lot of large datasets, this adds a delay and involves looking at large lists.
Better if we also only looked for DIDs in the relevant time range.
To do so: list all datasets in time range since last succesfully registered file. To identify last successfully registered file, write GPS time of file to disk.
This means the online registration clients become stateful and k8s deployment will require a PVC so they can resume.
Alternatively, we could allow clients to still parse the full catalog on resume.