Skip to content

Data discovery plugin

Patrick Godwin requested to merge patrick.godwin/locklost:data_discovery_plugin into master

I've added a discover plugin that checks for available data before moving on to the other plugins, implementing a request mentioned in #2 (closed).

What it does is first look in /dev/shm for the span of data requested. If a job launches too early and data is not available yet, it'll sleep until the right time (could be up to 15 seconds since we request for 15 seconds of data into the future for refine) and check again.

If /dev/shm doesn't work, it'll query for LDR data a few times until a maximum timeout (so it doesn't stall indefinitely), which I've set to 5 minutes. If it finds the data, great, otherwise it'll raise an error and prevent the other plugins from continuing. This should only occur if there is a systemic problem with the LDR.

In addition, in setup.py I needed to declare the plugins submodule as a package, otherwise if you don't run locklost within the repo directory, I've run into the particular issue (at least on python 2, not sure about python 3)

Traceback (most recent call last):
  File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "build/bdist.linux-x86_64/egg/locklost/__main__.py", line 10, in <module>
  File "build/bdist.linux-x86_64/egg/locklost/analyze.py", line 8, in <module>
ImportError: No module named plugins

Merge request reports