Skip to content

XLALFileResolvePathLong(): try path relative to location of liblalsupport.so

Karl Wette requested to merge ANU-CGA/lalsuite:relative-lal-data-path into master

Description

XLALFileResolvePathLong() is the function in lalsupport which resolves locations of data files using e.g. LAL_DATA_PATH. Currently it falls back to a directory decided as compile time, which is usually $(pkgdatadir). But since Python wheels re-arrange where data files are located, this fallback isn't of help, requiring LAL_DATA_PATH to be set.

This MR add an additional fallback for XLALFileResolvePathLong(): find the location of the liblalsupport.so file itself at run time, and add a relative path decided at compile time. Since the directory layout of e.g. wheels should be known at compile time, this fallback should be robust to wheels being installed in any location.

Two new options are added to lal/configure:

  • --with-relative-data-path enables the relative path fallback, and sets the relative path to the data files from the location of liblalsupport.so
  • --with-fallback-data-path enables the compile-time fallback directory; this is enabled by default, but can be disabled in cases where it doesn't make sense, e.g. for wheels where it would point to an ephemeral build directory

The code to locate liblalsupport.so from within liblalsupport.so itself is provided by Where Am I?, a two-file drop-in external library. It's MIT licensed so should be fined with GPLv2. It supports Linux, MacOSX, and a bunch of other platform, so should be pretty portable. I guess there's a long-term maintenance risk if this breaks in future and needs to be fixed, but hopefully not; it seems to use pretty low-level system APIs which should be stable long term.

The wheel builds in .gitlab-ci.yml are now configured with

  • Linux: --with-relative-data-path="../lalapps/data": liblalsupport.so is installed in lalsuite.libs/ (relative to wheel root), data files are installed in lalapps/data/
  • MacOSX: --with-relative-data-path="../../lalapps/data": liblalsupport.so is installed in lal/.dylibs/ (relative to wheel root), data files are installed in lalapps/data/
  • --without-fallback-data-path, to disable looking for the ephemeral compile-time build directory

The lal_data_path_fixup code in wheel/setup.py.in has been removed as it is no longer needed. I've left the class build_py(_build_py) override as a no-op, in case other build overrides are needed in future.

wheel-test now checks that data files can be found from both Python scripts and C execuables, using the relative path fallback. It calls Python & C code from LALSimulation, Python code from LALPulsar, and C code from LALApps/pulsar

API Changes and Justification

Backwards Compatible Changes

  • This change introduces no API changes
  • This change adds new API calls

Backwards Incompatible Changes

  • This change modifies an existing API
  • This change removes an existing API

Review Status

cc @adam-mercer @duncanmmacleod For the upcoming release, if possible

cc @leo-singer for comments

cc @david-keitel for interest

Edited by Karl Wette

Merge request reports