glue issueshttps://git.ligo.org/lscsoft/glue/-/issues2024-03-04T11:50:36Zhttps://git.ligo.org/lscsoft/glue/-/issues/43Glue doesn't declare it's minimum required version of setuptools for Debian2024-03-04T11:50:36ZDuncan Macleodduncan.macleod@ligo.orgGlue doesn't declare it's minimum required version of setuptools for DebianNow that all of the version metadata is in `pyproject.toml`, on Debian (at least) we need `python3-setuptools (>=61.0.0), but this isn't declared, see `https://git.ligo.org/computing/sccb/-/issues/1454#note_958035.Now that all of the version metadata is in `pyproject.toml`, on Debian (at least) we need `python3-setuptools (>=61.0.0), but this isn't declared, see `https://git.ligo.org/computing/sccb/-/issues/1454#note_958035.lscsoft-glue 4.0.1Duncan Macleodduncan.macleod@ligo.orgDuncan Macleodduncan.macleod@ligo.orghttps://git.ligo.org/lscsoft/glue/-/issues/42Drop support for EL7/RHEL7/SL72024-03-01T14:10:11ZRobert BruntzDrop support for EL7/RHEL7/SL7Scientific Linux 7 (SL7, and the RHEL7 underlying it) is now severely deprecated at LIGO(see ["LIGO Lab O4b Python2 and Scientific Linux 7 Security Policy for Computing Clusters"](https://dcc.ligo.org/LIGO-M2400052)). We should drop sup...Scientific Linux 7 (SL7, and the RHEL7 underlying it) is now severely deprecated at LIGO(see ["LIGO Lab O4b Python2 and Scientific Linux 7 Security Policy for Computing Clusters"](https://dcc.ligo.org/LIGO-M2400052)). We should drop support for SL7/EL7/RHEL7 at some point. (It's also the EL7 parts of CI that consistently take noticeably longer than anything else to finish.)https://git.ligo.org/lscsoft/glue/-/issues/41Build and upload wheels to PyPI2024-03-01T14:01:14ZLeo P. SingerBuild and upload wheels to PyPIhttps://git.ligo.org/lscsoft/glue/-/issues/40Rename master branch to main2024-02-29T18:20:38ZLeo P. SingerRename master branch to mainhttps://git.ligo.org/lscsoft/glue/-/issues/39Remove deprecated module iterutils2024-03-03T18:14:16ZRobert BruntzRemove deprecated module iterutils`iterutils.py` was [deprecated in Jan. 2022](https://git.ligo.org/lscsoft/glue/-/blob/master/glue/iterutils.py?blame=1#L48) (moved to
`lal.iterutils` inside the python(3)-lal package). It should be removed from `glue` at some point.`iterutils.py` was [deprecated in Jan. 2022](https://git.ligo.org/lscsoft/glue/-/blob/master/glue/iterutils.py?blame=1#L48) (moved to
`lal.iterutils` inside the python(3)-lal package). It should be removed from `glue` at some point.lscsoft-glue 4.0.0https://git.ligo.org/lscsoft/glue/-/issues/38Fix Homepage in debian/control2024-03-01T14:00:46ZRobert BruntzFix Homepage in debian/controlThe [file `debian/control`](https://git.ligo.org/lscsoft/glue/-/blob/master/debian/control) has the line `Homepage: https://www.lsc-group.phys.uwm.edu/daswg/projects/glue.html`, which redirects to [a missing LIGO GitLab page](https://com...The [file `debian/control`](https://git.ligo.org/lscsoft/glue/-/blob/master/debian/control) has the line `Homepage: https://www.lsc-group.phys.uwm.edu/daswg/projects/glue.html`, which redirects to [a missing LIGO GitLab page](https://computing.docs.ligo.org/daswgweb/projects/glue.html).lscsoft-glue 4.0.0https://git.ligo.org/lscsoft/glue/-/issues/37Bug in glue.pipeline - variable `ostart` used before it's defined2024-03-01T12:22:40ZRobert BruntzBug in glue.pipeline - variable `ostart` used before it's definedIn [`glue.pipeline`](https://git.ligo.org/lscsoft/glue/-/blob/master/glue/pipeline.py), class `ScienceData` is defined around line 1950, and within it, method `coalesce` is defined (around line 2300). That method has the line `x = Scien...In [`glue.pipeline`](https://git.ligo.org/lscsoft/glue/-/blob/master/glue/pipeline.py), class `ScienceData` is defined around line 1950, and within it, method `coalesce` is defined (around line 2300). That method has the line `x = ScienceSegment(tuple([id,ostart,ostop,ostop-ostart]))`, but variable `ostart` is not passed in or defined in that method up to that point.
I have not investigated what effect this might have on the functionality of the method.https://git.ligo.org/lscsoft/glue/-/issues/36Remove glue.gpstime2024-02-29T21:24:14ZRobert BruntzRemove glue.gpstimeThe `glue.gpstime` module [was deprecated](https://git.ligo.org/lscsoft/glue/-/merge_requests/109) in Jan. 2022 and should be removed, probably such that it is not included in `glue 4.0.0`.The `glue.gpstime` module [was deprecated](https://git.ligo.org/lscsoft/glue/-/merge_requests/109) in Jan. 2022 and should be removed, probably such that it is not included in `glue 4.0.0`.lscsoft-glue 4.0.0https://git.ligo.org/lscsoft/glue/-/issues/35lscsoft-glue 3.0.2 and 4.0.0 planning2024-02-29T08:27:47ZDuncan Macleodduncan.macleod@ligo.orglscsoft-glue 3.0.2 and 4.0.0 planning@robert.bruntz, please consider the following attack order to finalise %"lscsoft-glue 3.0.2" and %"lscsoft-glue 4.0.0":
Glue merge request attack order:
- [x] https://git.ligo.org/lscsoft/glue/-/merge_requests/125+
- [x] https://git.li...@robert.bruntz, please consider the following attack order to finalise %"lscsoft-glue 3.0.2" and %"lscsoft-glue 4.0.0":
Glue merge request attack order:
- [x] https://git.ligo.org/lscsoft/glue/-/merge_requests/125+
- [x] https://git.ligo.org/lscsoft/glue/-/merge_requests/107+
- [x] https://git.ligo.org/lscsoft/glue/-/merge_requests/87+
- [x] https://git.ligo.org/lscsoft/glue/-/merge_requests/114+
- [x] https://git.ligo.org/lscsoft/glue/-/merge_requests/115+
- [x] https://git.ligo.org/lscsoft/glue/-/merge_requests/118+
- [x] https://git.ligo.org/lscsoft/glue/-/merge_requests/121+
- [x] https://git.ligo.org/lscsoft/glue/-/merge_requests/124+
- [x] https://git.ligo.org/lscsoft/glue/-/merge_requests/126+
- [x] https://git.ligo.org/lscsoft/glue/-/merge_requests/130+
- [x] https://git.ligo.org/lscsoft/glue/-/merge_requests/131+
Then we can finalise lscsoft-glue-3.0.2 as the last glue.ligolw-supporting release.
Then
- [x] https://git.ligo.org/lscsoft/glue/-/merge_requests/117+
- [x] https://git.ligo.org/lscsoft/glue/-/merge_requests/127+
- [x] https://git.ligo.org/lscsoft/glue/-/merge_requests/128+
- [x] https://git.ligo.org/lscsoft/glue/-/merge_requests/129+
Many of the above will require rebasing from me as upstream MRs are merged, so please yell.lscsoft-glue 3.0.2Duncan Macleodduncan.macleod@ligo.orgRobert BruntzDuncan Macleodduncan.macleod@ligo.orghttps://git.ligo.org/lscsoft/glue/-/issues/34Cannot build glue on Python 3.122024-02-29T08:09:22ZDuncan Macleodduncan.macleod@ligo.orgCannot build glue on Python 3.12This project does not build on Python 3.12, fundamentally because the `PyUnicode_GetSize` function has finally been [removed](https://docs.python.org/3/whatsnew/3.12.html#id9).
I don't really want anyone to fix this, I would rather push...This project does not build on Python 3.12, fundamentally because the `PyUnicode_GetSize` function has finally been [removed](https://docs.python.org/3/whatsnew/3.12.html#id9).
I don't really want anyone to fix this, I would rather push on with #33+.
The full build traceback is as follows:
```console
$ python3 -m build --wheel
* Creating venv isolated environment...
* Installing packages in isolated environment... (setuptools>=30.3.0, wheel)
* Getting build dependencies for wheel...
/private/var/folders/jl/fzzqnlqd3752qmr12xjb1qfr0000gt/T/build-env-z3_uowq3/lib/python3.12/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
!!
********************************************************************************
The license_file parameter is deprecated, use license_files instead.
By 2023-Oct-30, you need to update your project and remove deprecated calls
or your builds will no longer be supported.
See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
********************************************************************************
!!
parsed = self.parsers.get(option_name, lambda x: x)(value)
running egg_info
creating lscsoft_glue.egg-info
writing lscsoft_glue.egg-info/PKG-INFO
writing dependency_links to lscsoft_glue.egg-info/dependency_links.txt
writing requirements to lscsoft_glue.egg-info/requires.txt
writing top-level names to lscsoft_glue.egg-info/top_level.txt
writing manifest file 'lscsoft_glue.egg-info/SOURCES.txt'
reading manifest file 'lscsoft_glue.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'man/man1/*'
adding license file 'LICENSE'
writing manifest file 'lscsoft_glue.egg-info/SOURCES.txt'
* Installing packages in isolated environment... (wheel)
* Building wheel...
/private/var/folders/jl/fzzqnlqd3752qmr12xjb1qfr0000gt/T/build-env-z3_uowq3/lib/python3.12/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
!!
********************************************************************************
The license_file parameter is deprecated, use license_files instead.
By 2023-Oct-30, you need to update your project and remove deprecated calls
or your builds will no longer be supported.
See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
********************************************************************************
!!
parsed = self.parsers.get(option_name, lambda x: x)(value)
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.9-x86_64-cpython-312
creating build/lib.macosx-10.9-x86_64-cpython-312/glue
copying glue/text_progress_bar.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue
copying glue/pidfile.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue
copying glue/LDBDWClient.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue
copying glue/iterutils.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue
copying glue/gpstime.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue
copying glue/__init__.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue
copying glue/markup.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue
copying glue/utils.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue
copying glue/pipeline.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue
copying glue/lal.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue
copying glue/git_version.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue
copying glue/ldbd.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue
creating build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw
copying glue/ligolw/ligolw.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw
copying glue/ligolw/ilwd.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw
copying glue/ligolw/lsctables.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw
copying glue/ligolw/__init__.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw
copying glue/ligolw/param.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw
copying glue/ligolw/types.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw
copying glue/ligolw/dbtables.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw
copying glue/ligolw/table.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw
copying glue/ligolw/array.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw
creating build/lib.macosx-10.9-x86_64-cpython-312/glue/segmentdb
copying glue/segmentdb/query_engine.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/segmentdb
copying glue/segmentdb/segmentdb_utils.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/segmentdb
copying glue/segmentdb/__init__.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/segmentdb
creating build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw/utils
copying glue/ligolw/utils/coalesce_db.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw/utils
copying glue/ligolw/utils/ligolw_add.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw/utils
copying glue/ligolw/utils/search_summary.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw/utils
copying glue/ligolw/utils/segments.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw/utils
copying glue/ligolw/utils/__init__.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw/utils
copying glue/ligolw/utils/ligolw_sqlite.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw/utils
copying glue/ligolw/utils/coincs.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw/utils
copying glue/ligolw/utils/print_tables.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw/utils
copying glue/ligolw/utils/process.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw/utils
copying glue/ligolw/utils/time_slide.py -> build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw/utils
running build_ext
building 'glue.ligolw.tokenizer' extension
creating build/temp.macosx-10.9-x86_64-cpython-312
creating build/temp.macosx-10.9-x86_64-cpython-312/glue
creating build/temp.macosx-10.9-x86_64-cpython-312/glue/ligolw
clang -fno-strict-overflow -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /Users/duncanmacleod/opt/mambaforge/envs/py312/include -fPIC -O2 -isystem /Users/duncanmacleod/opt/mambaforge/envs/py312/include -Isrc -Iglue/ligolw -I/private/var/folders/jl/fzzqnlqd3752qmr12xjb1qfr0000gt/T/build-env-z3_uowq3/include -I/Users/duncanmacleod/opt/mambaforge/envs/py312/include/python3.12 -c glue/ligolw/tokenizer.RowBuilder.c -o build/temp.macosx-10.9-x86_64-cpython-312/glue/ligolw/tokenizer.RowBuilder.o
glue/ligolw/tokenizer.RowBuilder.c:240:2: warning: suggest braces around initialization of subobject [-Wmissing-braces]
PyObject_HEAD_INIT(NULL)
^~~~~~~~~~~~~~~~~~~~~~~~
/Users/duncanmacleod/opt/mambaforge/envs/py312/include/python3.12/object.h:141:11: note: expanded from macro 'PyObject_HEAD_INIT'
{ 1 }, \
^
glue/ligolw/tokenizer.RowBuilder.c:240:2: warning: incompatible pointer to integer conversion initializing 'Py_ssize_t' (aka 'long') with an expression of type 'void *' [-Wint-conversion]
PyObject_HEAD_INIT(NULL)
^~~~~~~~~~~~~~~~~~~~~~~~
/Users/duncanmacleod/opt/mambaforge/envs/py312/include/python3.12/object.h:142:9: note: expanded from macro 'PyObject_HEAD_INIT'
(type) \
^~~~~~
2 warnings generated.
clang -fno-strict-overflow -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /Users/duncanmacleod/opt/mambaforge/envs/py312/include -fPIC -O2 -isystem /Users/duncanmacleod/opt/mambaforge/envs/py312/include -Isrc -Iglue/ligolw -I/private/var/folders/jl/fzzqnlqd3752qmr12xjb1qfr0000gt/T/build-env-z3_uowq3/include -I/Users/duncanmacleod/opt/mambaforge/envs/py312/include/python3.12 -c glue/ligolw/tokenizer.RowDumper.c -o build/temp.macosx-10.9-x86_64-cpython-312/glue/ligolw/tokenizer.RowDumper.o
glue/ligolw/tokenizer.RowDumper.c:283:2: warning: suggest braces around initialization of subobject [-Wmissing-braces]
PyObject_HEAD_INIT(NULL)
^~~~~~~~~~~~~~~~~~~~~~~~
/Users/duncanmacleod/opt/mambaforge/envs/py312/include/python3.12/object.h:141:11: note: expanded from macro 'PyObject_HEAD_INIT'
{ 1 }, \
^
glue/ligolw/tokenizer.RowDumper.c:283:2: warning: incompatible pointer to integer conversion initializing 'Py_ssize_t' (aka 'long') with an expression of type 'void *' [-Wint-conversion]
PyObject_HEAD_INIT(NULL)
^~~~~~~~~~~~~~~~~~~~~~~~
/Users/duncanmacleod/opt/mambaforge/envs/py312/include/python3.12/object.h:142:9: note: expanded from macro 'PyObject_HEAD_INIT'
(type) \
^~~~~~
2 warnings generated.
clang -fno-strict-overflow -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /Users/duncanmacleod/opt/mambaforge/envs/py312/include -fPIC -O2 -isystem /Users/duncanmacleod/opt/mambaforge/envs/py312/include -Isrc -Iglue/ligolw -I/private/var/folders/jl/fzzqnlqd3752qmr12xjb1qfr0000gt/T/build-env-z3_uowq3/include -I/Users/duncanmacleod/opt/mambaforge/envs/py312/include/python3.12 -c glue/ligolw/tokenizer.Tokenizer.c -o build/temp.macosx-10.9-x86_64-cpython-312/glue/ligolw/tokenizer.Tokenizer.o
glue/ligolw/tokenizer.Tokenizer.c:739:2: warning: suggest braces around initialization of subobject [-Wmissing-braces]
PyObject_HEAD_INIT(NULL)
^~~~~~~~~~~~~~~~~~~~~~~~
/Users/duncanmacleod/opt/mambaforge/envs/py312/include/python3.12/object.h:141:11: note: expanded from macro 'PyObject_HEAD_INIT'
{ 1 }, \
^
glue/ligolw/tokenizer.Tokenizer.c:739:2: warning: incompatible pointer to integer conversion initializing 'Py_ssize_t' (aka 'long') with an expression of type 'void *' [-Wint-conversion]
PyObject_HEAD_INIT(NULL)
^~~~~~~~~~~~~~~~~~~~~~~~
/Users/duncanmacleod/opt/mambaforge/envs/py312/include/python3.12/object.h:142:9: note: expanded from macro 'PyObject_HEAD_INIT'
(type) \
^~~~~~
2 warnings generated.
clang -fno-strict-overflow -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /Users/duncanmacleod/opt/mambaforge/envs/py312/include -fPIC -O2 -isystem /Users/duncanmacleod/opt/mambaforge/envs/py312/include -Isrc -Iglue/ligolw -I/private/var/folders/jl/fzzqnlqd3752qmr12xjb1qfr0000gt/T/build-env-z3_uowq3/include -I/Users/duncanmacleod/opt/mambaforge/envs/py312/include/python3.12 -c glue/ligolw/tokenizer.c -o build/temp.macosx-10.9-x86_64-cpython-312/glue/ligolw/tokenizer.o
clang -bundle -undefined dynamic_lookup -Wl,-rpath,/Users/duncanmacleod/opt/mambaforge/envs/py312/lib -L/Users/duncanmacleod/opt/mambaforge/envs/py312/lib -Wl,-rpath,/Users/duncanmacleod/opt/mambaforge/envs/py312/lib -L/Users/duncanmacleod/opt/mambaforge/envs/py312/lib build/temp.macosx-10.9-x86_64-cpython-312/glue/ligolw/tokenizer.RowBuilder.o build/temp.macosx-10.9-x86_64-cpython-312/glue/ligolw/tokenizer.RowDumper.o build/temp.macosx-10.9-x86_64-cpython-312/glue/ligolw/tokenizer.Tokenizer.o build/temp.macosx-10.9-x86_64-cpython-312/glue/ligolw/tokenizer.o -o build/lib.macosx-10.9-x86_64-cpython-312/glue/ligolw/tokenizer.cpython-312-darwin.so
building 'glue.ligolw._ilwd' extension
clang -fno-strict-overflow -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /Users/duncanmacleod/opt/mambaforge/envs/py312/include -fPIC -O2 -isystem /Users/duncanmacleod/opt/mambaforge/envs/py312/include -Isrc -Iglue/ligolw -I/private/var/folders/jl/fzzqnlqd3752qmr12xjb1qfr0000gt/T/build-env-z3_uowq3/include -I/Users/duncanmacleod/opt/mambaforge/envs/py312/include/python3.12 -c glue/ligolw/ilwd.c -o build/temp.macosx-10.9-x86_64-cpython-312/glue/ligolw/ilwd.o
glue/ligolw/ilwd.c:299:24: error: implicit declaration of function 'PyUnicode_GetSize' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
Py_ssize_t tbl_len = PyUnicode_GetSize(tbl);
^
glue/ligolw/ilwd.c:299:24: note: did you mean 'PyUnicode_Resize'?
/Users/duncanmacleod/opt/mambaforge/envs/py312/include/python3.12/unicodeobject.h:208:17: note: 'PyUnicode_Resize' declared here
PyAPI_FUNC(int) PyUnicode_Resize(
^
glue/ligolw/ilwd.c:302:24: warning: incompatible pointer types passing 'PyUnicodeObject *' to parameter of type 'PyObject *' (aka 'struct _object *') [-Wincompatible-pointer-types]
PyUnicode_AsWideChar((PyUnicodeObject *) tbl, buff, tbl_len);
^~~~~~~~~~~~~~~~~~~~~~~
/Users/duncanmacleod/opt/mambaforge/envs/py312/include/python3.12/unicodeobject.h:286:15: note: passing argument to parameter 'unicode' here
PyObject *unicode, /* Unicode object */
^
glue/ligolw/ilwd.c:304:24: warning: incompatible pointer types passing 'PyUnicodeObject *' to parameter of type 'PyObject *' (aka 'struct _object *') [-Wincompatible-pointer-types]
PyUnicode_AsWideChar((PyUnicodeObject *) col, buff + tbl_len + 1, col_len);
^~~~~~~~~~~~~~~~~~~~~~~
/Users/duncanmacleod/opt/mambaforge/envs/py312/include/python3.12/unicodeobject.h:286:15: note: passing argument to parameter 'unicode' here
PyObject *unicode, /* Unicode object */
^
glue/ligolw/ilwd.c:391:2: warning: suggest braces around initialization of subobject [-Wmissing-braces]
PyObject_HEAD_INIT(NULL)
^~~~~~~~~~~~~~~~~~~~~~~~
/Users/duncanmacleod/opt/mambaforge/envs/py312/include/python3.12/object.h:141:11: note: expanded from macro 'PyObject_HEAD_INIT'
{ 1 }, \
^
glue/ligolw/ilwd.c:391:2: warning: incompatible pointer to integer conversion initializing 'Py_ssize_t' (aka 'long') with an expression of type 'void *' [-Wint-conversion]
PyObject_HEAD_INIT(NULL)
^~~~~~~~~~~~~~~~~~~~~~~~
/Users/duncanmacleod/opt/mambaforge/envs/py312/include/python3.12/object.h:142:9: note: expanded from macro 'PyObject_HEAD_INIT'
(type) \
^~~~~~
4 warnings and 1 error generated.
error: command '/usr/bin/clang' failed with exit code 1
ERROR Backend subprocess exited when trying to invoke build_wheel
```https://git.ligo.org/lscsoft/glue/-/issues/33Remove glue.ligolw2024-03-01T12:34:29ZDuncan Macleodduncan.macleod@ligo.orgRemove glue.ligolwThe `glue.ligolw` subpackage has been deprecated for a _long_ time, and should be removed.
The last internal uses of the subpackage will be resolved by #32.The `glue.ligolw` subpackage has been deprecated for a _long_ time, and should be removed.
The last internal uses of the subpackage will be resolved by #32.lscsoft-glue 4.0.0https://git.ligo.org/lscsoft/glue/-/issues/32Migrate glue.ldbd and glue.segmentdb to DQSegDB client package2024-02-29T08:09:23ZDuncan Macleodduncan.macleod@ligo.orgMigrate glue.ldbd and glue.segmentdb to DQSegDB client packageThe `glue.ldbd` and `glue.segmentdb` modules/packages are the last library users of `glue.ligolw` and are only used downstream by the DQSegDB client package.
Those modules should be migrated in full to the DQSegDB client package until s...The `glue.ldbd` and `glue.segmentdb` modules/packages are the last library users of `glue.ligolw` and are only used downstream by the DQSegDB client package.
Those modules should be migrated in full to the DQSegDB client package until such time as the become obsolete over there as well.lscsoft-glue 4.0.0https://git.ligo.org/lscsoft/glue/-/issues/31Remove LDBDWClient module and any usage2024-02-23T16:48:57ZDuncan Macleodduncan.macleod@ligo.orgRemove LDBDWClient module and any usageThe `glue.LDBDWClient` module is not required any more, and should be removed.
The only function that uses it is `glue.segmentdb.segmentdb_utils.setup_database` which is no longer used by anything else in this project, or in the DQSegDB...The `glue.LDBDWClient` module is not required any more, and should be removed.
The only function that uses it is `glue.segmentdb.segmentdb_utils.setup_database` which is no longer used by anything else in this project, or in the DQSegDB client or server packages.lscsoft-glue 4.0.0https://git.ligo.org/lscsoft/glue/-/issues/30Use a local DTD file, if one is available2023-09-22T21:17:01ZRobert BruntzUse a local DTD file, if one is availableI think there was an attempt at some point to recognize and use a local DTD file if one existed, rather than always using the one on the CIT server ( http://ldas-sw.ligo.caltech.edu/doc/ligolwAPI/html/ligolw_dtd.txt ). This would probab...I think there was an attempt at some point to recognize and use a local DTD file if one existed, rather than always using the one on the CIT server ( http://ldas-sw.ligo.caltech.edu/doc/ligolwAPI/html/ligolw_dtd.txt ). This would probably speed up publishing and would improve resilience, since currently [publishing stops](https://git.ligo.org/computing/helpdesk/-/issues/4653) when the `ldas-sw.ligo.caltech.edu` server goes offline, Apache on that server is turned off, etc.
There is an old commit that did something like this here ( https://git.ligo.org/lscsoft/glue/-/commit/e8d2eb25ea575b203b89a840063a38c0c3ec69a0 ), but obviously that bypass is not being used (and might not even apply - I haven't traced through the code to see where this should happen in the current code). It would probably be best to first see if that code *should* be using the local DTD file, if so, then check if that file exists, if so, then figure out why it isn't working.https://git.ligo.org/lscsoft/glue/-/issues/29Please finish transition from python3-glue to python3-lscsoft-glue2023-11-22T09:25:36ZSteffen GrunewaldPlease finish transition from python3-glue to python3-lscsoft-glueThere's an eternal conflict of this glue's python3 package with one provided by Debian proper, causing strange build errors in the middle of LALsuite.
Since `lscsoft-glue` build-depends on `lal` and e.g. `lalburst` depends on `python3-g...There's an eternal conflict of this glue's python3 package with one provided by Debian proper, causing strange build errors in the middle of LALsuite.
Since `lscsoft-glue` build-depends on `lal` and e.g. `lalburst` depends on `python3-glue` this causes unnecessary interruptions: Debian's `python3-glue` package is at version 1.0-1 or so but nevertheless gets included by `Depends:` not specifying version range.
`python3-glue` (this one) has for long been a transitional package only. Please get rid of it, and force LALsuite developers to use the right dependency.https://git.ligo.org/lscsoft/glue/-/issues/28Build (test) failure with Ubuntu Jammy - version 3.0.12024-02-29T08:09:23ZSteffen GrunewaldBuild (test) failure with Ubuntu Jammy - version 3.0.1Attempting to build (and run tests) on Ubuntu 22.04 Jammy, which uses Python 3.10, results in tests failing:
```
dh_auto_test -O--buildsystem=pybuild ...Attempting to build (and run tests) on Ubuntu 22.04 Jammy, which uses Python 3.10, results in tests failing:
```
dh_auto_test -O--buildsystem=pybuild
dh_auto_test: warning: Compatibility levels before 10 are deprecated (level 9 in use)
I: pybuild base:239: cd /build/lscsoft-glue-3.0.1/.pybuild/cpython3_3.10/build; python3.10 -m unittest discover -v
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
I: pybuild pybuild:326: env PYTHONPATH=/build/lscsoft-glue-3.0.1/.pybuild/cpython3_3.10/build PYTHONDONTWRITEBYTECODE=1 make -C /build/lscsoft-glue-3.0.1/test PYTHON=python3.10 -o test_ldbd ;
make[1]: Entering directory '/build/lscsoft-glue-3.0.1/test'
=== start glue_ligolw_ilwd_verify ===>
python3.10 glue_ligolw_ilwd_verify.py && { echo "Pass" ; true ; } || { echo "Fail" ; false ; }
**********************************************************************
File "/build/lscsoft-glue-3.0.1/.pybuild/cpython3_3.10/build/glue/ligolw/_ilwd.cpython-310-x86_64-linux-gnu.so", line ?, in glue.ligolw._ilwd.ilwdchar
Failed example:
print(x - y)
Expected:
table_a:column_b:0
Got:
0
**********************************************************************
1 items had failures:
1 of 9 in glue.ligolw._ilwd.ilwdchar
***Test Failed*** 1 failures.
Fail
make[1]: *** [Makefile:27: glue_ligolw_ilwd_verify] Error 1
make[1]: Leaving directory '/build/lscsoft-glue-3.0.1/test'
E: pybuild pybuild:369: test: plugin distutils failed with: exit code=2: env PYTHONPATH=/build/lscsoft-glue-3.0.1/.pybuild/cpython3_3.10/build PYTHONDONTWRITEBYTECODE=1 make -C /build/lscsoft-glue-3.0.1/test PYTHON=python3.10 -o test_ldbd ;
dh_auto_test: error: pybuild --test -i python{version} -p 3.10 returned exit code 13
make: *** [debian/rules:29: build] Error 25
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
```
Is this serious enough, or can it be ignored? (Perhaps there's a patch to make it work with Python 3.10?)lscsoft-glue 4.0.0https://git.ligo.org/lscsoft/glue/-/issues/27Refactor glue/segmentdb/segmentdb_utils.py to not use glue.ligolw2023-10-12T12:31:50ZDuncan Macleodduncan.macleod@ligo.orgRefactor glue/segmentdb/segmentdb_utils.py to not use glue.ligolwAFAICT `glue.segmentdb.segmentdb_utils` is the only location inside glue itself that (still) uses `glue.ligolw`. If this module is still required (I think it is required by DQSegDB), then it needs to be refactored to use `python-ligo-lw`...AFAICT `glue.segmentdb.segmentdb_utils` is the only location inside glue itself that (still) uses `glue.ligolw`. If this module is still required (I think it is required by DQSegDB), then it needs to be refactored to use `python-ligo-lw` before #23+ can be realised.lscsoft-glue 3.1.0https://git.ligo.org/lscsoft/glue/-/issues/26glue.ligolw bug report from openSUSE gcc maintainer2023-11-24T21:05:33ZKipp Cannonglue.ligolw bug report from openSUSE gcc maintainercompiling glue.ligolw with gcc's new -D_FORTIFY_SOURCE=3 results in the test suite crashing with a buffer overflow error. in ilwd.c replacing the line swprintf(&buff[tbl_len + 1 + col_len], tbl_len + col_len + 23, L":%ld", ilwd->i); wit...compiling glue.ligolw with gcc's new -D_FORTIFY_SOURCE=3 results in the test suite crashing with a buffer overflow error. in ilwd.c replacing the line swprintf(&buff[tbl_len + 1 + col_len], tbl_len + col_len + 23, L":%ld", ilwd->i); with swprintf(&buff[tbl_len + 1 + col_len], 22, L":%ld", ilwd->i); fixes it. sorry, I have no ability to confirm this change, and I don't have a more conveniently formatted patch.https://git.ligo.org/lscsoft/glue/-/issues/25glue.segmentdb.segmentdb_utils.get_all_files_in_range() silently fails to rem...2022-04-13T16:01:21ZRobert Bruntzglue.segmentdb.segmentdb_utils.get_all_files_in_range() silently fails to remove 2nd bad filename (and 4th, 6th, etc.)This is an outgrowth of `dqsegdb` [issue 111](https://github.com/ligovirgo/dqsegdb/issues/111), in which it was noted that `ligolw_publish_threaded_dqxml_dqsegdb` would not publish DQXML files from a directory containing 2 or more files ...This is an outgrowth of `dqsegdb` [issue 111](https://github.com/ligovirgo/dqsegdb/issues/111), in which it was noted that `ligolw_publish_threaded_dqxml_dqsegdb` would not publish DQXML files from a directory containing 2 or more files with bad filenames. It would silently ignore the unpublished files in the dir until/unless the bad filenames count was reduced to 1 or 0. Quoted from that page:
[quote]
The problem is in `glue.segmentdb.segmentdb_utils.get_all_files_in_range()` (which corresponds to `/usr/lib64/python3.6/site-packages/glue/segmentdb/segmentdb_utils.py`). The function gets a list of filenames, sorts them, then iterates over the list, and if it finds a bad filename, it removes it from the list. The problem is that the pointer/index in the list is pointing to the name that gets removed; inferring that all items in the list are enumerated, the next one in the list gets moved up to the one that just got removed, and then the iterator moves on to the next item in the list, numerically - which means that the one that was moved up to replace the one that was removed is never processed. If there is only one bad filename in the dir, this is not a problem. If there are 2 bad filenames in the dir, they will be sorted into the first 2 slots in the list, then the first one will be removed, the 2nd one will move into the 1st slot, and the iterator will move on to the 2nd slot, leaving the bad filename in the 1st slot, which causes trouble somewhere else down the line (probably around line 199 in `ligolw_publish_threaded_dqxml_dqsegdb`: `pending_files += lal.Cache.from_urls(segmentdb_utils.get_all_files_in_range(options.input_directory,s[0],s[1]),coltype=int).sieve(segment=s)`). This is a [known issue](https://stackoverflow.com/questions/8312829/how-to-remove-item-from-a-python-list-in-a-loop) (not necessarily a bug) in Python, and there are workarounds for it, with one of the simplest being to [traverse the list in reverse order](https://stackoverflow.com/questions/529424/traverse-a-list-in-reverse-order-in-python), so any removed items only affect items that have already been checked, e.g., `for filename in file_list[::-1]:`.
[/quote]Robert BruntzRobert Bruntzhttps://git.ligo.org/lscsoft/glue/-/issues/24Running ligolw_segment_insert_dqsegdb produces error 'glue.ldbd.LIGOLwParseEr...2022-01-12T09:01:16ZRobert BruntzRunning ligolw_segment_insert_dqsegdb produces error 'glue.ldbd.LIGOLwParseError: invalid stream length for given columns'Using the version of `glue` in [MR 84](https://git.ligo.org/lscsoft/glue/-/merge_requests/84) ("Fix glue.ldbd XML parsing under Python 3"), running `ligolw_segment_insert_dqsegdb` produces an error that includes text like `glue.ldbd.LIGO...Using the version of `glue` in [MR 84](https://git.ligo.org/lscsoft/glue/-/merge_requests/84) ("Fix glue.ldbd XML parsing under Python 3"), running `ligolw_segment_insert_dqsegdb` produces an error that includes text like `glue.ldbd.LIGOLwParseError: invalid stream length for given columns`.
Example: running `/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/bin/ligolw_segment_insert_dqsegdb --insert --segment-url=http://segments-backup.ldas.cit --ifos=G1 --name=TEST_FLAG_GPS_TIME_1325614162 --version=1 -S /home/detchar/dqsegdb/known_segments_test_file.txt -G /home/detchar/dqsegdb/active_segments_test_file.txt --explain="This is a test flag published by test_segments_machine.sh on detchar-cit in a run started at GPS time 1325614162." --comment="This is a test flag published by test_segments_machine.sh on detchar-cit in a run started at GPS time 1325614162."` in a Python 3 venv on the CIT cluster, using the version of glue in MR 84, produces the following error:
```
Traceback (most recent call last):
File "/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/bin/ligolw_segment_insert_dqsegdb", line 454, in <module>
publishAndExit(filepath,debug)
File "/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/bin/ligolw_segment_insert_dqsegdb", line 192, in publishAndExit
result=callInsertMultipleDQXMLThreaded(filepath,debug)
File "/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/bin/ligolw_segment_insert_dqsegdb", line 212, in callInsertMultipleDQXMLThreaded
result=apicalls.InsertMultipleDQXMLFileThreaded(infiles,logger,options.segment_url,hackDec11=False,debug=debug,threads=1)
File "/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/dqsegdb/apicalls.py", line 796, in InsertMultipleDQXMLFileThreaded
segment_md = setupSegment_md(filename,xmlparser,lwtparser,debug)
File "/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/dqsegdb/apicalls.py", line 734, in setupSegment_md
segment_md.parse(xmltext)
File "/root/venv_p3_glue_from_repo/lib64/python3.6/site-packages/glue/ldbd.py", line 390, in parse
self.table = self.lwtparser.parsetuple(ligolwtup)
File "/root/venv_p3_glue_from_repo/lib64/python3.6/site-packages/glue/ldbd.py", line 311, in parsetuple
raise LIGOLwParseError('invalid stream length for given columns')
glue.ldbd.LIGOLwParseError: invalid stream length for given columns
```
Investigating the above, the error comes from lines 307-311 of `ldbd.py`:
```
slen = len(stream)
ntyp = len(table[tab]['column'])
mlen, lft = divmod(slen,ntyp)
if lft != 0:
raise LIGOLwParseError('invalid stream length for given columns')
```
where `stream` = `['--segment-url', 'process:process_id:0', '/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/bin/ligolw_segment_insert_dqsegdb', 'lstring', 'http://segments-backup.ldas.cit', '--insert', 'process:process_id:0', '/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/bin/ligolw_segment_insert_dqsegdb', '', '', '--append', 'process:process_id:0', '/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/bin/ligolw_segment_insert_dqsegdb', 'int_8s', '0', '--ignore-future', 'process:process_id:0', '/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/bin/ligolw_segment_insert_dqsegdb', 'int_8s', '0', '--ignore-append-check', 'process:process_id:0', '/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/bin/ligolw_segment_insert_dqsegdb', 'int_8s', '0', '--debug', 'process:process_id:0', '/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/bin/ligolw_segment_insert_dqsegdb', 'int_8s', '0', '--ifos', 'process:process_id:0', '/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/bin/ligolw_segment_insert_dqsegdb', 'lstring', 'G1', '--name', 'process:process_id:0', '/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/bin/ligolw_segment_insert_dqsegdb', 'lstring', 'TEST_FLAG_GPS_TIME_1325614162', '--version', 'process:process_id:0', '/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/bin/ligolw_segment_insert_dqsegdb', 'lstring', '1', '--explain', 'process:process_id:0', '/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/bin/ligolw_segment_insert_dqsegdb', 'lstring', 'This is a test flag published by test_segments_machine.sh on detchar-cit in a run started at GPS time 1325614162.', '--comment', 'process:process_id:0', '/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/bin/ligolw_segment_insert_dqsegdb', 'lstring', 'This is a test flag published by test_segments_machine.sh on detchar-cit in a run started at GPS time 1325614162.', '--summary-file', 'process:process_id:0', '/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/bin/ligolw_segment_insert_dqsegdb', 'lstring', '/home/detchar/dqsegdb/known_segments_test_file.txt', '--segment-file', 'process:process_id:0', '/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/bin/ligolw_segment_insert_dqsegdb', 'lstring', '/home/detchar/dqsegdb/active_segments_test_file.txt', '--active-indicates-ifo-goodness', 'process:process_id:0', '/home/robert.bruntz/git_work/dqsegdb_2021.11.30/dqsegdb/bin/ligolw_segment_insert_dqsegdb', '', '', '']` (length = 71)
and `table[tab]['column']` = `{'param': 'lstring', 'process_id': 'ilwd:char', 'program': 'lstring', 'type': 'lstring', 'value': 'lstring'}` (length = 5).
Since 71%5 = 1 != 0, this raises an error. It is not yet clear how this mismatch occurs.
Note: the version of `glue` produced by the merging of MR 84 can be installed (e.g., in a virtual environment) by cloning the `glue` repo (`git clone https://git.ligo.org/lscsoft/glue.git`), then from that repo running `git fetch upstream +refs/merge-requests/84/head: && git checkout -qf FETCH_HEAD;`, then running `python3 -m pip install --upgrade pip`, then `python3 -m pip install .`.