... | ... | @@ -15,7 +15,7 @@ Thus, all tests have been performed using `bilby==0.6.5`, `cpnest==0.9.7`, `dyne |
|
|
### Convergence to analytical evidence
|
|
|
As we increase the number of live points, stochastic and systematic sampling errors should decrease and the mean value of the evidence should converge to the analytically known evidence. We choose to perform 100 runs with 32, 64, 128, ..., 4096 livepoints with the analytical likelihood both in the unimodal and bimodal case. All other settings are the defaults as they are specified in `bilby`. `bilby` mostly tries to emulate the defaults the samplers set themselves. The prior is taken to be uniform from -20 to 20 in each dimension.
|
|
|
|
|
|
The results can be seen below. The displayed errorbars are taken to be standard deviation of the 100 measured log evidences. If one is interested in the uncertainty on the *mean* of the log evidence after 100 runs, these errorbars need to be divided by `\sqrt 100 = 10`. While `dynesty`, `nestle`, and `polychord` converge to the analytical value, 'cpnest', 'dynamic_dynesty', and 'pymultinest' are significantly biased. We see this result as generally encouraging for `dynesty`. We also note that it is generally more difficult for samplers to recover the bimodal case, which is why it takes more live points to converge to the same level.
|
|
|
The results can be seen below. The displayed errorbars are taken to be standard deviation of the 100 measured log evidences. If one is interested in the uncertainty on the *mean* of the log evidence after 100 runs, these errorbars need to be divided by `\sqrt 100 = 10`. While `dynesty`, `nestle`, and `polychord` converge to the analytical value, `cpnest`, `dynamic_dynesty`, and `pymultinest` are systematically biased even for high numbers of live points. We see this result as generally encouraging for `dynesty`. We also note that it is generally more difficult for samplers to recover the bimodal case, which is why it takes more live points to converge to the same level.
|
|
|
|
|
|
![all_samplers_review_unimodal_summary](uploads/3904f7c48207f4abab73505ad4ef6fbc/all_samplers_review_unimodal_summary.png)
|
|
|
![all_samplers_review_bimodal_summary](uploads/dfb698950ab60945b6c869a8544d8096/all_samplers_review_bimodal_summary.png)
|
... | ... | |