Skip to content

Add optional Zstandard compression

Zstandard provides much faster compression and decompression than gzip at any compression ratio. The "zstandard" Python package is an optional dependency to enable this new feature.

I tried running ligolw_sqlite on a recent gstlal coinc database to convert it to XML. The uncompressed XML file took 32 seconds to write and was 300M in size. The gzip-compressed XML file took 52 seconds and was 98M. The Zstandard-compressed XML file took 35 seconds and was 87M in size.

In short, Zstandard compression will give comparable file sizes to gzip but with compression/decompression time that is negligible compared to LIGO-LW processing itself.

Merge request reports

Loading