Skip to content
  • Leo P. Singer's avatar
    Add optional Zstandard compression · 00e189e6
    Leo P. Singer authored
    Zstandard provides much faster compression and decompression than
    gzip at any compression ratio. The "zstandard" Python package is
    an optional dependency to enable this new feature.
    
    I tried running `ligolw_sqlite` on a recent gstlal coinc database
    to convert it to XML. The uncompressed XML file took 32 seconds to
    write and was 300M in size. The gzip-compressed XML file took 52
    seconds and was 98M. The Zstandard-compressed XML file took 35
    seconds and was 87M in size.
    
    In short, Zstandard compression will give comparable file sizes
    to gzip but with compression/decompression time that is negligible
    compared to LIGO-LW processing itself.
    00e189e6