Suggestion: add higher order parameter class to generate compiled parameter files
This relates to the discussions about handling precomputed parameters in a generic way (#40 (closed), #49 (closed)).
I suggest we add a class that acts like a higher order parameter file and can be optionally used instead of specifying a parameter file directly. In this class users can define the location of one or many input parameter files to load in order (with successive files overwriting any duplicate parameters in earlier files), and any precomp routines you wish to define along with the name of the parameter(s) they should define. This class could then generate a single, "machine readable" representation of the parameter file for use in pygwinc
. This file could be intentionally obfuscated to discourage users from modifying it directly, e.g. by pickling it or dumping to YAML without any whitespace. The class could check if such a file exists, and if not, generate it. It could also have fancier make
-like checks for file modification times, triggering a recompilation if one of the input parameter files (or the higher order parameter class itself) changes.
We could also add the ability to define sanity checks in this class, e.g. to check precomp parameters are within certain bounds.
Here is an example of how this class could look:
class HigherOrderStruct:
def __init__(self, *input_files):
self._struct = Struct()
self.input_files = input_files
@property
def struct(self):
"""Get struct including precomputed parameters."""
if self.modified():
self._compile()
self._do_checks()
return pickle.load(self.output_path)
@property
def output_path(self):
# generate hash from paths and modification times of input files and this file, in order,
# and use this as the filename. can be stored in user's cache or temp directory?
pass
def modified(self):
# check if the file named self.output_path exists
def _compile(self):
for input_file in self.input_files:
self._struct.update(Struct.from_file(input_file))
for params, precomp_method in self._PRECOMP_METHODS: # populated by decorator
values = precomp_method(self._struct)
for param, value in zip(params, values):
self._struct[param] = value
pickle.dump(self.output_path, self._struct)
def _do_checks(self):
# perform checks defined by e.g. self.set_param_tolerance.
# trigger exception on failure
pass
@defines('Materials.MirrorVolume', 'Materials.MirrorMass')
def mirror_stuff(self, ifo):
volume = pi*ifo.Materials.MassRadius**2 * Materials.MassThickness
mass = volume * ifo.Materials.Substrate.MassDensity
return volume, mass
def set_param_tolerance(self, param, value, rtol=1e-6, atol=1e-9):
# Add some internal check to run once the parameter file is compiled.
# In this case, it could use math.isclose.
self._param_tolerances.append((param, value, rtol, atol))
This approach would:
- Provide a clear API for users to define their own precomp routines without having to directly modify the
Struct
object created from their parameter file before passing it topygwinc
. - Let users define a hierarchy of parameter files. This could be useful for instance to allow users to load some set of shared parameters like material properties, provided by gwinc, before their own custom parameter file, avoiding repetition.
- Avoid having to call precomp routines during subsequent runs of pygwinc (helps when running optimisations).
- Catch bugs created by changes to precomp routines by allowing users to define tolerances on expected values.
This stuff is exactly how I am using pygwinc
currently.
Thoughts? Happy to make a PR if you think this is useful...