LALInference posterior samples files are unreadable because they have duplicate values for nLocalTemps and randomSeed
The LALInference.posterior_samples.hdf5 files that are generated online are unreadable because the nLocalTemps and randomSeed variables are represented both as a column and as a table metadata attribute, and an error is produced when reading them in and ligo.skymap.io.hdf5.read_samples
attempts to normalize the table metadata attribute to a column. Here's the traceback:
ValueError: Duplicate column names
File "gwcelery/tasks/skymaps.py", line 144, in skymap_from_samples
['-j', '-o', tmpdir, samplefile.name])
File "/home/emfollow-playground/.local/lib/python3.7/site-packages/ligo/skymap/tool/ligo_skymap_from_samples.py", line 94, in main
data = io.read_samples(args.samples.name)
File "/home/emfollow-playground/.local/lib/python3.7/site-packages/ligo/skymap/io/hdf5.py", line 202, in read_samples
meta={'vary': FIXED}))
File "astropy/table/table.py", line 1707, in add_column
self.add_columns([col], [index], name, copy=copy, rename_duplicate=rename_duplicate)
File "astropy/table/table.py", line 1846, in add_columns
self._init_from_cols(newcols)
File "astropy/table/table.py", line 897, in _init_from_cols
self._make_table_from_cols(self, newcols)
File "astropy/table/table.py", line 961, in _make_table_from_cols
raise ValueError('Duplicate column names')
I see that this behavior has been in place for a long time in lalinference.cbcBayesMCMC2pos.reassign_metadata
, so I'm not sure why we are only seeing this error now.
Please either modify the reassign_metadata
function to remove the table-level attributes after they are copied into column values, or specify how to deal with the duplicate values so that I can update my code accordingly.
See: https://sentry.io/organizations/ligo-caltech/issues/1397362278