Skip to content

Add new CIT HTCondor management scripts.

Luke Davis requested to merge ldavis__update_prod_scripts into spiir-O4-EW-development

Add some new CIT HTCondor management scripts.
new-run.sh for submitting a job.

A reproducible environment can be setup by running:

# once https://git.ligo.org/patrick.clearwater/spiir-scripts/-/merge_requests/10 is merged:
git clone https://git.ligo.org/patrick.clearwater/spiir-scripts.git
cd spiir-scripts

# once https://git.ligo.org/lscsoft/spiir/-/merge_requests/159 is merged:
singularity build --sandbox spiir-base-py2 docker://containers.ligo.org/lscsoft/spiir:base-spiir-O4-EW-development
cd build
# `--singularity yes --singularity-base <spiir sandbox path>` is automatically set on CIT but needs to be specified on ozstar.
bash build-spiir.sh environment --singularity yes --singularity-base $(pwd)/../spiir-base-py2
bash build-spiir.sh build-spiir --name test-spiir --branch spiir-O4-EW-development --singularity yes --singularity-base $(pwd)/../spiir-base-py2
cd ..

# The following is if you would like to deploy using HTCondor on CIT (and run on MDC Playground Test group)
# I will already be running pastro and monitoring. These will not need to be deployed more than once.

# Once this MR is merged:
cp -r build/spiir-O4-EW-development/source/gstlal-spiir/share/configs/O4/monitoring .
cd monitoring
condor_submit_dag monitor.dag
cd ..

# Listening for gracedb uploads requires setting up SCiMMA credentials from: https://github.com/scimma/hop-client/wiki/Tutorial:-using-hop-client-with-the-SCiMMA-Hopskotch-server
cp -r build/spiir-O4-EW-development/source/gstlal-spiir/share/configs/O4/pastro .
cd pastro
condor_submit_dag p-astro-consumer.dag
cd ..

cp -r build/spiir-O4-EW-development/source/gstlal-spiir/share/configs/O4/prod .
cd prod
# USER can be automatically detected unless logged in as `spiir` in which case you must specify what you would like to set the accounting_group_user to.  
# SPIIR_NAME corresponds to the build-spiir --name directory name you built spiir into earlier.  
# TAG is an optional extra string you would like to add to the name of the run directory that will be generated.
USER=luke.davis SPIIR_NAME=test-spiir TAG=test-run bash new-run.sh
Edited by Luke Davis

Merge request reports