python-ligo-lw merge requestshttps://git.ligo.org/kipp/python-ligo-lw/-/merge_requests2020-07-27T15:05:56Zhttps://git.ligo.org/kipp/python-ligo-lw/-/merge_requests/13Add optional Zstandard compression2020-07-27T15:05:56ZLeo P. SingerAdd optional Zstandard compressionZstandard provides much faster compression and decompression than
gzip at any compression ratio. The "zstandard" Python package is
an optional dependency to enable this new feature.
I tried running `ligolw_sqlite` on a recent gstlal coi...Zstandard provides much faster compression and decompression than
gzip at any compression ratio. The "zstandard" Python package is
an optional dependency to enable this new feature.
I tried running `ligolw_sqlite` on a recent gstlal coinc database
to convert it to XML. The uncompressed XML file took 32 seconds to
write and was 300M in size. The gzip-compressed XML file took 52
seconds and was 98M. The Zstandard-compressed XML file took 35
seconds and was 87M in size.
In short, Zstandard compression will give comparable file sizes
to gzip but with compression/decompression time that is negligible
compared to LIGO-LW processing itself.https://git.ligo.org/kipp/python-ligo-lw/-/merge_requests/16Add f_lower column to sngl_inspiral table2020-07-08T19:17:40ZLeo P. SingerAdd f_lower column to sngl_inspiral tablePipelines need to report their low frequency cutoff in order to
avoid luminosity distance biases.
See https://sympa.ligo.org/wws/arc/cbc/2020-04/msg00116.html.Pipelines need to report their low frequency cutoff in order to
avoid luminosity distance biases.
See https://sympa.ligo.org/wws/arc/cbc/2020-04/msg00116.html.https://git.ligo.org/kipp/python-ligo-lw/-/merge_requests/22WIP: enable zst compression using new compress= kwarg2021-12-09T06:54:19ZLeo P. SingerWIP: enable zst compression using new compress= kwargFIXME: The one-liner expression that many scripts have to pass to
`write_filename` will become unwieldy as the number of available
compression methods grows from 1 to 2 or even more. Instead of this:
```python
write_filename(
filename,...FIXME: The one-liner expression that many scripts have to pass to
`write_filename` will become unwieldy as the number of available
compression methods grows from 1 to 2 or even more. Instead of this:
```python
write_filename(
filename,
compress = compress = 'gz' if (options.output or "stdout").endswith(".gz") else False
)
```
this patch proposes a new `compress_for_filename` helper function:
```python
write_filename(
filename,
compress = compress_for_filename(filename)
)
```
However, even this breaks the DRY principle.https://git.ligo.org/kipp/python-ligo-lw/-/merge_requests/36Port to the Python limited API2024-02-21T18:31:12ZLeo P. SingerPort to the Python limited APIPort C code to the
[Python limited API](https://docs.python.org/3/c-api/stable.html).
This allows us to build a single binary for each architecture and
operating system that supports all Python versions.Port C code to the
[Python limited API](https://docs.python.org/3/c-api/stable.html).
This allows us to build a single binary for each architecture and
operating system that supports all Python versions.