gstlalcalibration plugin requires FFTW but doesn't link against it
Summary
The gstlalcalibration Gst plugin uses code from FFTW, but makes no attempt to link against it. This can be seen by running readelf -d on the installed library:
$ dnf provides /usr/lib64/gstreamer-1.0/libgstgstlalcalibration.so
Last metadata expiration check: 0:20:23 ago on Thu 01 Dec 2022 02:54:34 AM PST.
gstlal-calibration-1.4.0-1.el8.x86_64 : GSTLAL Calibration
Repo : @System
Matched from:
Filename : /usr/lib64/gstreamer-1.0/libgstgstlalcalibration.so
gstlal-calibration-1.4.0-1.el8.x86_64 : GSTLAL Calibration
Repo : igwn-testing
Matched from:
Filename : /usr/lib64/gstreamer-1.0/libgstgstlalcalibration.so
$ readelf -d /usr/lib64/gstreamer-1.0/libgstgstlalcalibration.so
Dynamic section at offset 0x96270 contains 46 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libpython3.6m.so.1.0]
0x0000000000000001 (NEEDED) Shared library: [libgstlalcalibration.so.0]
0x0000000000000001 (NEEDED) Shared library: [liblalmetaio.so.10]
0x0000000000000001 (NEEDED) Shared library: [liblalsupport.so.14]
0x0000000000000001 (NEEDED) Shared library: [libgstlal.so.1]
0x0000000000000001 (NEEDED) Shared library: [libgstlaltags.so.1]
0x0000000000000001 (NEEDED) Shared library: [libgstlaltypes.so.1]
0x0000000000000001 (NEEDED) Shared library: [libgsl.so.23]
0x0000000000000001 (NEEDED) Shared library: [libgslcblas.so.0]
0x0000000000000001 (NEEDED) Shared library: [libgstaudio-1.0.so.0]
0x0000000000000001 (NEEDED) Shared library: [liblal.so.20]
0x0000000000000001 (NEEDED) Shared library: [libgstbase-1.0.so.0]
0x0000000000000001 (NEEDED) Shared library: [libgstcontroller-1.0.so.0]
0x0000000000000001 (NEEDED) Shared library: [libgstreamer-1.0.so.0]
0x0000000000000001 (NEEDED) Shared library: [libgobject-2.0.so.0]
0x0000000000000001 (NEEDED) Shared library: [libglib-2.0.so.0]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000e (SONAME) Library soname: [libgstgstlalcalibration.so]
...Linux behaviour
This is not (currently) a failure on Linux because the GNU linker automatically links libgstgstlalcalibration against all of the libraries that is dependents link against. This can be seen by running ldd on the same library:
$ ldd /usr/lib64/gstreamer-1.0/libgstgstlalcalibration.so
linux-vdso.so.1 (0x00007fff4754f000)
libpython3.6m.so.1.0 => /lib64/libpython3.6m.so.1.0 (0x00007fab201aa000)
libgstlalcalibration.so.0 => /lib64/libgstlalcalibration.so.0 (0x00007fab1ff89000)
liblalmetaio.so.10 => /lib64/liblalmetaio.so.10 (0x00007fab1fd6d000)
liblalsupport.so.14 => /lib64/liblalsupport.so.14 (0x00007fab1f9e8000)
libgstlal.so.1 => /lib64/libgstlal.so.1 (0x00007fab1f7cf000)
libgstlaltags.so.1 => /lib64/libgstlaltags.so.1 (0x00007fab1f5cd000)
libgstlaltypes.so.1 => /lib64/libgstlaltypes.so.1 (0x00007fab1f3c5000)
libgsl.so.23 => /lib64/libgsl.so.23 (0x00007fab1ef34000)
libgslcblas.so.0 => /lib64/libgslcblas.so.0 (0x00007fab1ecf5000)
libgstaudio-1.0.so.0 => /lib64/libgstaudio-1.0.so.0 (0x00007fab1ea7a000)
liblal.so.20 => /lib64/liblal.so.20 (0x00007fab1e71e000)
libgstbase-1.0.so.0 => /lib64/libgstbase-1.0.so.0 (0x00007fab1e4a2000)
libgstcontroller-1.0.so.0 => /lib64/libgstcontroller-1.0.so.0 (0x00007fab1e290000)
libgstreamer-1.0.so.0 => /lib64/libgstreamer-1.0.so.0 (0x00007fab1df4c000)
libgobject-2.0.so.0 => /lib64/libgobject-2.0.so.0 (0x00007fab1dcf9000)
libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x00007fab1d9e0000)
libm.so.6 => /lib64/libm.so.6 (0x00007fab1d65e000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fab1d446000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fab1d226000)
libc.so.6 => /lib64/libc.so.6 (0x00007fab1ce60000)
libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007fab1c977000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fab1c773000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007fab1c56f000)
libmetaio.so.1 => /lib64/libmetaio.so.1 (0x00007fab1c365000)
libz.so.1 => /usr/lib64/libz.so.1 (0x00007fab1c14d000)
libsz.so.2 => /usr/lib64/libsz.so.2 (0x00007fab1bf4a000)
libhdf5.so.103 => /usr/lib64/libhdf5.so.103 (0x00007fab1b9bf000)
libhdf5_hl.so.100 => /usr/lib64/libhdf5_hl.so.100 (0x00007fab1b79b000)
libfftw3.so.3 => /usr/lib64/libfftw3.so.3 (0x00007fab1b201000)
libfftw3f.so.3 => /usr/lib64/libfftw3f.so.3 (0x00007fab1ac20000)
liblalsimulation.so.31 => /lib64/liblalsimulation.so.31 (0x00007fab1a0da000)
liblalburst.so.6 => /lib64/liblalburst.so.6 (0x00007fab19ecb000)
liblalinspiral.so.17 => /lib64/liblalinspiral.so.17 (0x00007fab19bd0000)
libgstallocators-1.0.so.0 => /lib64/libgstallocators-1.0.so.0 (0x00007fab199cb000)
libgsttag-1.0.so.0 => /lib64/libgsttag-1.0.so.0 (0x00007fab1978e000)
liborc-0.4.so.0 => /lib64/liborc-0.4.so.0 (0x00007fab19510000)
libgmodule-2.0.so.0 => /lib64/libgmodule-2.0.so.0 (0x00007fab1930c000)
librt.so.1 => /lib64/librt.so.1 (0x00007fab19104000)
libgnutls.so.30 => /lib64/libgnutls.so.30 (0x00007fab18d13000)
libpcre.so.1 => /lib64/libpcre.so.1 (0x00007fab18aa2000)
libffi.so.6 => /lib64/libffi.so.6 (0x00007fab18899000)
/lib64/ld-linux-x86-64.so.2 (0x00007fab20986000)
libaec.so.0 => /usr/lib64/libaec.so.0 (0x00007fab18691000)
liblalframe.so.13 => /lib64/liblalframe.so.13 (0x00007fab1842c000)
libp11-kit.so.0 => /lib64/libp11-kit.so.0 (0x00007fab18102000)
libidn2.so.0 => /lib64/libidn2.so.0 (0x00007fab17ee4000)
libunistring.so.2 => /lib64/libunistring.so.2 (0x00007fab17b63000)
libtasn1.so.6 => /lib64/libtasn1.so.6 (0x00007fab17950000)
libnettle.so.6 => /lib64/libnettle.so.6 (0x00007fab17716000)
libhogweed.so.4 => /lib64/libhogweed.so.4 (0x00007fab174e6000)
libgmp.so.10 => /lib64/libgmp.so.10 (0x00007fab1724e000)
libframel.so.8 => /lib64/libframel.so.8 (0x00007fab16fcf000)
libframecppc.so.3 => /lib64/libframecppc.so.3 (0x00007fab16d83000)
libframecpp.so.15 => /lib64/../lib64/libframecpp.so.15 (0x00007fab16b79000)
libframecpp8.so.11 => /lib64/../lib64/libframecpp8.so.11 (0x00007fab16803000)
libframecpp7.so.6 => /lib64/../lib64/libframecpp7.so.6 (0x00007fab16558000)
libframecpp6.so.10 => /lib64/../lib64/libframecpp6.so.10 (0x00007fab16237000)
libframecpp4.so.10 => /lib64/../lib64/libframecpp4.so.10 (0x00007fab15f4a000)
libframecpp3.so.8 => /lib64/../lib64/libframecpp3.so.8 (0x00007fab15cb2000)
libframecppcmn.so.12 => /lib64/../lib64/libframecppcmn.so.12 (0x00007fab15a2c000)
libldastoolsal.so.7 => /lib64/../lib64/libldastoolsal.so.7 (0x00007fab157ab000)
libstdc++.so.6 => /lib64/../lib64/libstdc++.so.6 (0x00007fab15416000)I have no idea whether this behaviour (of the GNU linker) is standard, or reliable, but depending on implicit links seems like asking for trouble.
macOS behaviour
On macOS this is a big issue because the Clang linker doesn't automatically propagate links from dependents. Consider the following code:
Build script
#!/bin/bash
. $(conda info --base)/etc/profile.d/conda.sh
. $(conda info --base)/etc/profile.d/mamba.sh
set -ex
# create conda environment for this build
ENV_NAME="gstlal-calibration-build"
if $(conda activate ${ENV_NAME} 2>/dev/null); then # env exists
mamba env remove --name ${ENV_NAME}
fi
mamba env create \
--name ${ENV_NAME} \
--file environment.yml \
;
mamba activate "${ENV_NAME}"
# configure
./00init.sh
./configure \
--prefix=${CONDA_PREFIX} \
;
# build
make -j
# test
make check -j
# install
make install -jConda environment.yml
channels:
- conda-forge
dependencies:
# build:
- autoconf
- automake
- c-compiler
- libtool
- make
- pkg-config
# host:
- fftw
- gsl
- gstreamer
- gst-plugins-base
- gstlal >=1.10.0
- libglib
- liblal >=7.2.4
- liblalmetaio >=3.0.2
- numpy
- python
name: gstlal-calibrationBuild error
libtool: link: x86_64-apple-darwin13.4.0-clang -o .libs/libgstgstlalcalibration.so -bundle .libs/libgstgstlalcalibration_la-gstlalcalibration.o .libs/libgstgstlalcalibration_la-gstlal_add_constant.o .libs/libgstgstlalcalibration_la-gstlal_wings.o .libs/libgstgstlalcalibration_la-gstlal_complexfirbank.o .libs/libgstgstlalcalibration_la-gstlal_smoothcalibfactors.o .libs/libgstgstlalcalibration_la-gstlal_smoothkappas.o .libs/libgstgstlalcalibration_la-gstlal_constantupsample.o .libs/libgstgstlalcalibration_la-gstlal_resample.o .libs/libgstgstlalcalibration_la-gstlal_logicalundersample.o .libs/libgstgstlalcalibration_la-gstlal_demodulate.o .libs/libgstgstlalcalibration_la-gstlal_insertgap.o .libs/libgstgstlalcalibration_la-gstlal_fccupdate.o .libs/libgstgstlalcalibration_la-gstlal_transferfunction.o .libs/libgstgstlalcalibration_la-gstlal_trackfrequency.o .libs/libgstgstlalcalibration_la-gstlal_adaptivefirfilt.o .libs/libgstgstlalcalibration_la-gstlal_dqtukey.o .libs/libgstgstlalcalibration_la-gstlal_property.o .libs/libgstgstlalcalibration_la-gstlal_typecast.o .libs/libgstgstlalcalibration_la-gstlal_matrixsolver.o .libs/libgstgstlalcalibration_la-gstlal_sensingtdcfs.o .libs/libgstgstlalcalibration_la-gstlal_makediscont.o .libs/libgstgstlalcalibration_la-gstlal_randreplace.o .libs/libgstgstlalcalibration_la-gstlal_stdev.o .libs/libgstgstlalcalibration_la-gstlal_minmax.o .libs/libgstgstlalcalibration_la-gstlal_asd.o .libs/libgstgstlalcalibration_la-gstlal_calibcorr.o .libs/libgstgstlalcalibration_la-gstlal_sweep.o -L/Users/duncanmacleod/opt/mambaforge/envs/gstlal-calibration-build/lib -lpython3.10 ../../lib/.libs/libgstlalcalibration.dylib -llalmetaio -llalsupport -lgstlal -lgstlaltags -lgstlaltypes -lgsl -lgslcblas -lcblas -lgstaudio-1.0 -llal -lgstbase-1.0 -lgstcontroller-1.0 -lgstreamer-1.0 -lgobject-2.0 -lglib-2.0 -lintl -lm -pthread -pthread -march=core2 -mtune=haswell -mssse3 -fstack-protector-strong -O2 -Wl,-rpath -Wl,/Users/duncanmacleod/opt/mambaforge/envs/gstlal-calibration-build/lib -Wl,-rpath -Wl,/Users/duncanmacleod/opt/mambaforge/envs/gstlal-calibration-build/lib -Wl,-rpath -Wl,/Users/duncanmacleod/opt/mambaforge/envs/gstlal-calibration-build/lib -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-rpath -Wl,/Users/duncanmacleod/opt/mambaforge/envs/gstlal-calibration-build/lib -pthread -Wl,-exported_symbols_list,.libs/libgstgstlalcalibration-symbols.expsym
ld: warning: -pie being ignored. It is only used when linking a main executable
Undefined symbols for architecture x86_64:
"_fftw_destroy_plan", referenced from:
_free_workspace in libgstgstlalcalibration_la-gstlal_complexfirbank.o
_MakeFilter in libgstgstlalcalibration_la-gstlal_fccupdate.o
_set_caps in libgstgstlalcalibration_la-gstlal_transferfunction.o
_finalize in libgstgstlalcalibration_la-gstlal_transferfunction.o
"_fftw_execute", referenced from:
_filter in libgstgstlalcalibration_la-gstlal_complexfirbank.o
_create_fdd_workspace in libgstgstlalcalibration_la-gstlal_complexfirbank.o
_MakeFilter in libgstgstlalcalibration_la-gstlal_fccupdate.o
_update_fir_filters_double in libgstgstlalcalibration_la-gstlal_transferfunction.o
_find_transfer_functions_double in libgstgstlalcalibration_la-gstlal_transferfunction.o
"_fftw_free", referenced from:
_free_workspace in libgstgstlalcalibration_la-gstlal_complexfirbank.o
_MakeFilter in libgstgstlalcalibration_la-gstlal_fccupdate.o
_set_caps in libgstgstlalcalibration_la-gstlal_transferfunction.o
_finalize in libgstgstlalcalibration_la-gstlal_transferfunction.o
"_fftw_malloc", referenced from:
_create_fdd_workspace in libgstgstlalcalibration_la-gstlal_complexfirbank.o
_MakeFilter in libgstgstlalcalibration_la-gstlal_fccupdate.o
_set_caps in libgstgstlalcalibration_la-gstlal_transferfunction.o
"_fftw_plan_dft_1d", referenced from:
_MakeFilter in libgstgstlalcalibration_la-gstlal_fccupdate.o
"_fftw_plan_dft_c2r_1d", referenced from:
_create_fdd_workspace in libgstgstlalcalibration_la-gstlal_complexfirbank.o
_set_caps in libgstgstlalcalibration_la-gstlal_transferfunction.o
"_fftw_plan_dft_r2c_1d", referenced from:
_create_fdd_workspace in libgstgstlalcalibration_la-gstlal_complexfirbank.o
_set_caps in libgstgstlalcalibration_la-gstlal_transferfunction.o
"_fftwf_destroy_plan", referenced from:
_free_workspace in libgstgstlalcalibration_la-gstlal_complexfirbank.o
_set_caps in libgstgstlalcalibration_la-gstlal_transferfunction.o
_finalize in libgstgstlalcalibration_la-gstlal_transferfunction.o
"_fftwf_execute", referenced from:
_filter in libgstgstlalcalibration_la-gstlal_complexfirbank.o
_create_fds_workspace in libgstgstlalcalibration_la-gstlal_complexfirbank.o
_update_fir_filters_float in libgstgstlalcalibration_la-gstlal_transferfunction.o
_find_transfer_functions_float in libgstgstlalcalibration_la-gstlal_transferfunction.o
"_fftwf_free", referenced from:
_free_workspace in libgstgstlalcalibration_la-gstlal_complexfirbank.o
_set_caps in libgstgstlalcalibration_la-gstlal_transferfunction.o
_finalize in libgstgstlalcalibration_la-gstlal_transferfunction.o
"_fftwf_malloc", referenced from:
_create_fds_workspace in libgstgstlalcalibration_la-gstlal_complexfirbank.o
_set_caps in libgstgstlalcalibration_la-gstlal_transferfunction.o
"_fftwf_plan_dft_c2r_1d", referenced from:
_create_fds_workspace in libgstgstlalcalibration_la-gstlal_complexfirbank.o
_set_caps in libgstgstlalcalibration_la-gstlal_transferfunction.o
"_fftwf_plan_dft_r2c_1d", referenced from:
_create_fds_workspace in libgstgstlalcalibration_la-gstlal_complexfirbank.o
_set_caps in libgstgstlalcalibration_la-gstlal_transferfunction.o
ld: symbol(s) not found for architecture x86_64
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [Makefile:528: libgstgstlalcalibration.la] Error 1
make[2]: Leaving directory '/Users/duncanmacleod/git/gstlal-calibration/gst/lal'
make[1]: *** [Makefile:407: all-recursive] Error 1
make[1]: Leaving directory '/Users/duncanmacleod/git/gstlal-calibration/gst'
make: *** [Makefile:448: all-recursive] Error 1This isn't a new issue, I've just been (silently) patching around it in the conda-forge builds of this project for years - apologies for not reporting it earlier.
Edited by Duncan Macleod