PEsummary incompatible with updated htcondor
The submit file generated for pesummary is still using getenv = True, which is no longer working, ie:
executable = /home/tomasz.baka/.conda/envs/mdr/bin/summarypages
arguments = --webdir /home/tomasz.baka/public_html/mdr-gwtc3/GW191109_010717/mdr_graviton_mass/pesummary --labels mdr_graviton_mass --gw --approximant IMRPhenomXPHM --f_low 20 --f_ref 20 --cosmology Planck15_lal --redshift_method exact --nsamples_for_skymap 2000 --evolve_spins_fowards True --multi_process 4 --config /home/tomasz.baka/mdr_gwtc3/checkouts/GW191109_010717/C01_offline/mdr_graviton_mass.ini --samples /home/tomasz.baka/mdr_gwtc3/working/GW191109_010717/mdr_graviton_mass/result/mdr_graviton_mass_data0_1257296855-216458_analysis_H1L1_merge_result.hdf5 --calibration /home/cal/public_html/uncertainty/O3C01/H1/2020-06-12_O3_LHO_GPSTime_1257297300_C01_RelativeResponseUncertainty_FinalResults.txt /home/cal/public_html/uncertainty/O3C01/L1/2020-06-12_O3_LLO_GPSTime_1257297300_C01_RelativeResponseUncertainty_FinalResults.txt /home/cbc/pe/O3/calibrationenvelopes/Virgo/V_O3a_calibrationUncertaintyEnvelope_magnitude5percent_phase35milliradians10microseconds.txt --psd H1:/home/tomasz.baka/mdr_gwtc3/working/GW191109_010717/Prod0/trigtime_1257296855.216458082_0.0_0.0_0/post/clean/glitch_median_PSD_forLI_H1.dat L1:/home/tomasz.baka/mdr_gwtc3/working/GW191109_010717/Prod0/trigtime_1257296855.216458082_0.0_0.0_0/post/clean/glitch_median_PSD_forLI_L1.dat
output = /home/tomasz.baka/mdr_gwtc3/working/GW191109_010717/mdr_graviton_mass/pesummary.out
error = /home/tomasz.baka/mdr_gwtc3/working/GW191109_010717/mdr_graviton_mass/pesummary.err
log = /home/tomasz.baka/mdr_gwtc3/working/GW191109_010717/mdr_graviton_mass/pesummary.log
request_cpus = 4
getenv = true
batch_name = PESummary/GW191109_010717/mdr_graviton_mass
request_memory = 8192MB
request_disk = 8192MB
MY.flock_local = True
MY.DESIRED_Sites = "nogrid"
accounting_group_user = tomasz.baka
accounting_group = ligo.prod.o4.cbc.testgr.tiger
This is how the submit in newest version of bilby_pipe
looks like.
universe = vanilla
executable = /home/tomasz.baka/.conda/envs/mdr/bin/summarypages
request_memory = 16 GB
request_disk = 2.0GB
request_cpus = 1
initialdir = /home/tomasz.baka/pipeline_test/MDC/systematics/SXS_BBH_1386_iota_0p79
notification = Never
requirements =
log = 0.0/log_results_page/SXS_BBH_1386_iota_0p79_0p0_pesummary.log
output = 0.0/log_results_page/SXS_BBH_1386_iota_0p79_0p0_pesummary.out
error = 0.0/log_results_page/SXS_BBH_1386_iota_0p79_0p0_pesummary.err
accounting_group = ligo.prod.o4.cbc.testgr.tiger
accounting_group_user = tomasz.baka
priority = 0
environment = "HDF5_USE_FILE_LOCKING=FAlSE OMP_NUM_THREADS=1 OMP_PROC_BIND=false"
arguments = $(ARGS)
queue
Note that this new version still does not work - there are some additional environmental variables that need to be imported. One is CONDA_EXE
, needed for this piece of code in pesummary to work (the fallback case doesn't work and causes an error). You should be able to get this with getenv = CONDA_EXE
. (I think you should also import USER
to get proper webpage path in the logs)
But there is still at least one needed variable connected with latex. I get this error log. I have not been able to find the correct one yet. The pesummary works fine when not submitted through condor, so its not caused by any code error.