Adding "combine combine" feature
The pygwb_combine
script only works one way, combining output files from pygwb_pipe
into a single set of 3 files (point estimates, delta sigma/gating info, and coherences). With this MR, I'm adding the possibility to combine pre-combined files with pygwb_combine
with a second call of pygwb_combine
, such that, e.g., one can first combine all files referring to a single day of data into a combined set, and then combine those into weekly sets, etc...
There is some loss of backwards compatibility as I have had to change some naming conventions in the combined .npz
files. Specific name changes can be viewed in the diff.
A pygwb_pipe
job outputs two files: a "spectra" file, which includes information about the delta-sigma values of the segments, and a psd_csd
file, which includes data for the calculation of the coherence (hereafter referred to as "coherence" file).
When using pygwb_combine
for the first time, i.e., when combining the output of a set of pygwb_pipe
runs directly, the command looks like
pygwb_combine --data_path [path_to_spectra_files_from_pipeline] --alpha [chosen_alpha] --fref [chosen_fref] --param_file [pipeline_param_file_used_in_analysis] --combine_coherence True --coherence_path [path_to_coherence_files_from_pipeline] --out_path [path_to_store_combined_files]
this outputs three files : a spectra file, a delta_sigma file, and a coherence file. Step 2 of the combination then requires a command that includes the three files:
pygwb_combine --data_path [path_to_spectra_files_from_combine1] --alpha [chosen_alpha] --fref [chosen_fref] --param_file [pipeline_param_file_used_in_analysis] --combine_coherence True --coherence_path [path_to_coherence_files_from_combine1] --delta_sigma_path [path_to_dsc_files_from_combine1] --out_path [path_to_store_combined_files]