bayeswave-output.md 3.39 KB
Newer Older
1
# Understanding Output Data
James Clark's avatar
James Clark committed
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42
BayesWave stores its output in `outputDir`, provided by the user. If using through condor on the LDG, the progress of the run can be monitored through `logs/BayesWave_*out` where you can find the current iteration. 

Once the run is complete, the typical content of `outputDir` includes:

1. A `chains/` directory where the full chains are stored.

Depending on the settings of your run, you should have chains for each run mode (clean, glitch, signal, noise). Possible chains include

* `*_model.dat.0`: general information about the run, such as the number of iterations, the likelihood, the number of wavelets used etc. The `0` refers to the coldest chain
* `*_lorentz_IFO.dat.0`: parameters for the lorentzians used to fit the PSD in each IFO, where the `0` refers to the coldest chain. This is only present when running with `BayesLine`.
* `*_spline_IFO.dat.0`: parameters for the splines used to fit the PSD in each IFO, where the `0` refers to the coldest chain. This is only present when running with `BayesLine`.
* `signal_params_h0.dat.0`: parameters of the signal wavelets that fit power coherently across all detectors. The `h0` refers to the cross signal polarization, and the `0` refers to the coldest chain.
* `glitch_params_IFO.dat.0`: parameters of the glitch wavelets that fit power incoherently per detector IFO. The `0` refers to the coldest chain.


2. A `checkpoint/` directory that holds the last checkpointed state.

You are most likely not interested in the contents

3. A `post/` directory with the output of `BayesWavePost`.

Each model has its own subdirectory here such as `signal/` and `glitch/` with more or less similar content that is usually helpfully named. For example, in the `signal/` subdirectory some of the most useful files are:

* `fourier_domain_signal_median_residual_IFO.dat`: frequency bin, real, and imaginary part of the median residual (data after the median signal has been subtracted)
* `signal_anderson_darling_IFO.dat`: Anderson darling statistic and corresponding p value that test how well the PSD has whitened the data. This is only present when running with `BayesLine`. Columns include the sampling rate up to which this is computed, the AD, and the p value.
* Various median estimates characterizing the signal mode. These files in general are structured as

`frequency/time, median, lower50, upper50, lower90, upper90`

where (for example) upper 50 is the upper boundary of the 50th percentile.
* Various files containing moments characterizing the signal reconstructions such as SNR, bandwidth, duration, overlap etc. Each files contains a header describing the moments included and then lists the moments per iterations. The are computed from both the whitened and the colored signal (option given in the file name).


4. A `waveforms/` directory that is populated if you have run with the `gnuplot` flag with 200 draws from the posterior .

It is probably not advisable to run with this flag on the LDG as it produces a lot of output.

5. Various files printing PSDs and ASDs per IFO, such as `L1_burninpsd.dat` and `L1_fairdraw_psd.dat`. These are needed by `BayesWavePost`.

6. `evidence.dat` contains the evidence (and standard deviation) per model, while files such as `model_evidence.dat` contain various pieces that go into the evidence calculations (such as temperatures and likelihoods per chain).

43
7. A `plots/` directory that holds the output plots from `megaplot`.