ROQ Analysis job failing at post-processing
Config ini for GW150914 with ROQ (data files located here):
trigger-time = 1126259462.3910 # Trigger time from G184098
likelihood-type=ROQGravitationalWaveTransient
roq-folder=/fred/oz006/ROQ_data/IMRPhenomPv2/4s
label = GW150914i_with_roq
outdir = outdir
detectors = [H1, L1]
duration = 4
channel-dict = {H1:DCS-CALIB_STRAIN_C02, L1:DCS-CALIB_STRAIN_C02}
psd-dict = {H1:../raw_data/h1_psd.dat, L1:../raw_data/l1_psd.dat}
data_dict = {H1=../raw_data/dataH1.gwf, L1=../raw_data/dataL1.gwf}
maximum-frequency=1024
minimum-frequency=20
sampling-frequency=2048
reference-frequency = 20
waveform-approximant = IMRPhenomPV2
deltaT = 0.2
time-marginalization=False
distance-marginalization=True
phase-marginalization=True
prior-file = 4s
nodes=12
ntasks-per-node=10
time=40:00:00
nlive=2000
nact=20
n-parallel = 1
mem-per-cpu=2GB
extra-lines = source ~/.bash_profile; conda activate parallel_bilby
slurm-extra-lines = partition=gstar dependency=singleton
Error:
15:21 bilby INFO : Start checkpoint writing
15:24 bilby INFO : Written checkpoint file /fred/oz980/avajpeyi/projects/pbilby_gw150914_tests/run_d/outdir/result/GW150914i_with_roq_0_checkpoint_resume.pickle
15:24 bilby INFO : write_current_state: 0:02:43.344877
15:24 bilby INFO : Writing 9493 current samples to /fred/oz980/avajpeyi/projects/pbilby_gw150914_tests/run_d/outdir/result/GW150914i_with_roq_0_samples.dat
15:24 bilby INFO : plot_current_state: 0:00:35.557845
15:25 bilby INFO : Using LAL version Branch: None;Tag: None;Id: 8bae27f7ace28704492a813974efe987919a8413;;Builder: Unknown User <>;Repository status: UNCLEAN: Modified working tree
15:26 bilby INFO : Using 51641 samples
15:26 bilby INFO : Generating posterior from marginalized parameters for nsamples=51641, POOL=119
===========================worker with rank 1 crashed===========================
===========================worker with rank 2 crashed===========================
Traceback (most recent call last):
File "/fred/oz980/avajpeyi/installations/parallel_bilby/parallel_bilby/schwimmbad_fast.py", line 96, in __init__
self.wait()
File "/fred/oz980/avajpeyi/installations/parallel_bilby/parallel_bilby/schwimmbad_fast.py", line 198, in wait
result = func(arg)
File "/fred/oz980/avajpeyi/installations/parallel_bilby/parallel_bilby/utils.py", line 38, in fill_sample
conversion.compute_snrs(sample, likelihood)
File "/fred/oz117/avajpeyi/.conda/envs/parallel_bilby/lib/python3.8/site-packages/bilby/gw/conversion.py", line 1120, in compute_snrs
signal = ifo.get_detector_response(signal_polarizations, sample)
File "/fred/oz117/avajpeyi/.conda/envs/parallel_bilby/lib/python3.8/site-packages/bilby/gw/detector/interferometer.py", line 299, in get_detector_response
det_response = self.antenna_response(
File "/fred/oz117/avajpeyi/.conda/envs/parallel_bilby/lib/python3.8/site-packages/bilby/gw/detector/interferometer.py", line 280, in antenna_response
polarization_tensor = gwutils.get_polarization_tensor(ra, dec, time, psi, mode)
File "/fred/oz117/avajpeyi/.conda/envs/parallel_bilby/lib/python3.8/site-packages/bilby/gw/utils.py", line 147, in get_polarization_tensor
raise ValueError("{} not a polarization mode!".format(mode))