How-to generate a PP plot for a sampler

Recipe to generate results

Here is a recipe generate a PP plot for dynesty. You'll need three things. A configuration file config.ini with content:

config.ini
label = bbh_pp_study
detectors = [H1, L1]
duration = 4
outdir=.
trigger-time = 0
prior-file = chirp_mass_ratio.prior
create-output = True
injection = True
n-injection = 500
accounting = ligo.dev.o3.cbc.pe.lalinference
deltaT = 0.1

# Sampler specific configuration: to be varied
sampler = dynesty
sampler-kwargs = {'nlive': 1024, 'walks': 60}

A prior-file chirp_mass_ratio.prior with content:

chirp_mass = Uniform(name='chirp_mass', minimum=25, maximum=100, unit='$M_{\\odot}$')
mass_ratio =  Uniform(name='mass_ratio', minimum=0.125, maximum=1)
a_1 =  0
a_2 =  0
tilt_1 =  0
tilt_2 =  0
phi_12 =  0
phi_jl =  0
dec =  Cosine(name='dec')
ra =  Uniform(name='ra', minimum=0, maximum=2 * np.pi)
iota =  Sine(name='iota')
psi =  Uniform(name='psi', minimum=0, maximum=np.pi)
phase =  Uniform(name='phase', minimum=0, maximum=2 * np.pi)
luminosity_distance = PowerLaw(alpha=2, minimum=100, maximum=1000, name='luminosity_distance')
geocent_time = 0

and a Makefile (a file literally named Makefile) with content:

SHELL = /bin/bash

simg = bilby_pipe_0.0.3.simg

.PHONY : clean submit

submit : clean $(simg)
	./$(simg) config.ini
	condor_submit_dag submit/dag*

clean :
	rm -rf log_* result submit

$(simg) :
	singularity pull --name $(simg) shub://lscsoft/bilby_pipe:0.0.3

Once these three files are in a directory, then running

$ make

will download the singularity container ¹, remove any old files (WARNING: take care not to delete anything you are trying to add too), generate the jobs and submit them.

Generating the pp plot itself

Finally to generate a PP plot, add this script (say named generate_pp_plots.py) with content:

import glob
import sys
import bilby

results_directory = sys.argv[1]

results_files = glob.glob(results_directory + '/*result.h5')
results = [bilby.core.result.read_in_result(f) for f in results_files]
bilby.core.result.make_pp_plot(results, filename='pp.png')

and run as

$ python generate_pp_plots.py results/

here the results is the path to where the output *result.h5 files are stored by default for the run. If you change this, you'll also need to change this path.

Tweaking the sampler

In the example above, we used dynesty with 1024 live points and walks=60. You can pass anything into the sampler_kwargs dictionary that you would usually pass to run_sampler in bilby. Note the quote marks though around the keys of the dictionary.

Notes

• To change the number of results generated, change the n-injection parameter in the configuration.
• Using the container is preferred as it guarantees the version of bilby and bilby_pipe are consistent (you should indicate the versions when reporting results).
• You can of course also use your own installation of bilby and bilby_pipe, replace ./bilby_pipe_0.0.3 in the Makescript with just bilby_pipe.