We aim to have the knowledge transfer from Divya, operating the SSM search for O4a offline and O4b. - Internal documentation till we transfer it to gwsci: [google doc](https://docs.google.com/document/d/1UfO6mr6R6j3BDLB1Vb8jV3KJTEZ2bercOZkKPYwrJgM/edit?usp=sharing) - Shio's temporary notes are [here](https://git.ligo.org/gstlal/projects/-/issues/1046). # Template banks ### Short author paper: - git: https://git.ligo.org/james.kennington/ssm-bank-paper - paper dcc: [dcc](https://dcc.ligo.org/P2400418-x0) - presentation slides: [dcc](https://dcc.ligo.org/G2402112-x0) ### Online bank - [DCC](https://dcc.ligo.org/T2200397): contains the PSD, the online ssm bank, and the online checkerboarded banks. - [template bank plots](https://ldas-jobs.gwave.ics.psu.edu/~shio.sakon/observing/4/dev/ssm/ssm_online/template_bank_plots/): plots of the full bank in m1-m2, mchirp-chieff, mtotal-chieff, m1-m2-chieff, m1-m2-mchirp, and the checkerboarded banks in log(m1)-log(m2) - bank sims - Note: [submit file](https://git.ligo.org/james.kennington/ssm-bank-paper/-/issues/6#note_1116641) - Full results (by Shio) - Full bank [plots](https://ldas-jobs.gwave.ics.psu.edu/~shio.sakon/observing/4/dev/ssm/ssm_online/banksim/full_bank_default_inj/), includes bns_low, bns_high - Full bank, nsbh inj total mass bug fixed: [plots](https://ldas-jobs.gwave.ics.psu.edu/~shio.sakon/observing/4/dev/ssm/ssm_online/banksim/full_bank_nsbh_rerun/), includes nsbh_low, nsbh_high - Checkerboarded bank (left half) [plots](https://ldas-jobs.gwave.ics.psu.edu/~shio.sakon/observing/4/dev/ssm/ssm_online/banksim/checkerboarded_bank_default_inj/left/) - Checkerboarded bank (right half) [plots](https://ldas-jobs.gwave.ics.psu.edu/~shio.sakon/observing/4/dev/ssm/ssm_online/banksim/checkerboarded_bank_default_inj/right/) - Paper quality plots (by Shio and Victoria) - [full-bank](https://ldas-jobs.gwave.ics.psu.edu/~shio.sakon/observing/4/dev/ssm/ssm_online/banksim/paper-quality-plots/), includes bbh, nsbh_low, nsbh_high, bns_low, bns_high - [checkerboard (left half)](https://ldas-jobs.ligo.caltech.edu/~wanting.niu/gstlal/observing/2/osg/ssm/plots/bank_sims_plots_sbank_231112/checkerboard_left/) - [checkerboard (right half)](https://ldas-jobs.ligo.caltech.edu/~wanting.niu/gstlal/observing/2/osg/ssm/plots/bank_sims_plots_sbank_231112/checkerboard_right/) ### Offline bank (manifold) - We need to generate the SSM offline bank with manifold because of mass models and pastros. - The final version of the bank is on [dcc](https://dcc.ligo.org/LIGO-T2400099-v3). The number of template is 3452006. - The template bank is in ICDS: `ligo/home/ligo.org/chad.hanna/manifold_ssm/fingers-crossed-33-aspect-ratios-1to8-small-pad-128-o4psd-proper-chi-035-no-fine/H1L1V1-O4_SSMOFFLINE_BANK-0-2000000000.h5` - template bank plots: [plots](https://ldas-jobs.gwave.ics.psu.edu/~shio.sakon/observing/4/dev/ssm/ssm_offline/template-bank/) - bank sim - SSM offline bank: [plots](https://ldas-jobs.gwave.ics.psu.edu/~shio.sakon/observing/4/dev/ssm/ssm_offline/banksim/manifold_ssm_v3/), contains bns_high, bns_low, nsbh_low, nsbh_high, low_low, high_low, high_high - SSM offline bank with the AllSky bank: [plots](https://ldas-jobs.gwave.ics.psu.edu/~shio.sakon/observing/4/dev/ssm/ssm_offline/banksim/allsky_with_ssm_offline/), includes high_high List of Banks generated: | Key | Summary | NumTemplates | 10th Pct | Link | |-----|---------|--------------|----------|------| | 00A | Ellipse mm03 | 5,511,540 | 0.9662 | [link](https://ldas-jobs.gwave.ics.psu.edu/~james.kennington/ssm-banks/mm03/) | | 00B | Ellipse mm04 | 1,189,289 | 0.9338 | [link](https://ldas-jobs.gwave.ics.psu.edu/~james.kennington/ssm-banks/mm04/) | | 00C | Dur64, fHigh1024 | 972,047 | 0.9444 | [link](https://ldas-jobs.gwave.ics.psu.edu/~james.kennington/ssm-banks/dur64/) | | 001 | Dur64 | 3,124,664 | 0.9647 | [link](https://ldas-jobs.gwave.ics.psu.edu/~james.kennington/ssm-banks/cfg001/) | | 002 | Dur32 | 1,406,696 | 0.9607 | [link](https://ldas-jobs.gwave.ics.psu.edu/~james.kennington/ssm-banks/cfg002/) | | 003 | Similar to 002, narrow ellipses | 1,406,662 | 0.9604 | [link](https://ldas-jobs.gwave.ics.psu.edu/~james.kennington/ssm-banks/cfg003/) | | 004 | Similar to 003, more padding | 1,598,317 | 0.9617 | [link](https://ldas-jobs.gwave.ics.psu.edu/~james.kennington/ssm-banks/cfg004/) | | 005 | Similar to 004, more padding for NS | 1,614,280 | 0.9627 | [link](https://ldas-jobs.gwave.ics.psu.edu/~james.kennington/ssm-banks/cfg005/) | | 006 | Similar to 005, more padding for NS | 1,738,659 | 0.9651 | [link](https://ldas-jobs.gwave.ics.psu.edu/~james.kennington/ssm-banks/cfg006/) | | FINAL | FINAL PRODUCTION BANK | 3452006 | 90% match = 0.98 | [link](https://dcc.ligo.org/LIGO-T2400099-v3/) | ### O3 SSM bank See [this issue](https://git.ligo.org/publications/O3/o3a-cbc-subsolar/-/wikis/Review-items/Searches/Template-Bank) for details. ### O4 dtdphi file <table> <tr> <th></th> <th>run-dir (CIT)</th> <th>build</th> <th>PSD</th> <th>config</th> </tr> <tr> <td>O4a (512Hz)</td> <td> `/home/cort.posnansky/observing/4/dev/dtdphi/ssm/fhigh_512/inspiral_dtdphi_pdf.h5` </td> <td> `/home/shomik.adhicary/Projects/2022_LR_Calculation/edited_norm_jacobian/build/gstlal-dev_230416` </td> <td> project \_O4 PSD (see Cort's directory) </td> <td> `/home/cort.posnansky/observing/4/dev/dtdphi/ssm/Makefile` </td> </tr> <tr> <td>O4a (1024Hz)</td> <td> `/home/wanting.niu/ssm/observing/o4a_offline/review/dtdphi/snr_diff/generate_dtdphi_at_1024/inspiral_dtdphi_pdf.h5` </td> <td> `/home/shomik.adhicary/Projects/2022_LR_Calculation/edited_norm_jacobian/build/gstlal-dev_230416` </td> <td> project \_O4 PSD (see Cort's directory) </td> <td> ``` --H-snr 5 --L-snr 7 --V-snr 4 --K-snr 4 --m1 1 --m2 1 --s1 0 --s2 0 --fhigh 1024 --flow 45 --singularity-image /home/shomik.adhicary/Projects/2022_LR_Calculation/edited_norm_jacobian/build/gstlal-dev_230416 --profile ldas ``` </td> </tr> </table> # Branches/Containers ### Online `o4b-online_ssm`: created Feb 15 off of `o4b-online`. - cherry-picked [250e30](https://git.ligo.org/lscsoft/gstlal/-/commit/250e3056bcc51f940e884b57b238643cb41306fe). The `o4b-online_ssm` branch described below is now `o4b-online_ssm_mdc26`. `o4b-online_ssm`: created off of `o4b-online` on Nov 13 and rebased on Dec 19. [link](https://git.ligo.org/lscsoft/gstlal/-/commits/o4b-online_ssm). Tagged as [`ssm-o4b-online`](https://git.ligo.org/lscsoft/gstlal/-/tags/ssm-o4b-online), and the container is [here](https://git.ligo.org/gstlal/o4b-containers/-/tree/ssm?ref_type=heads). - commits added: cherry-picked some later o4b-container commits - online monitoring epic is added to here: https://git.ligo.org/groups/gstlal/-/epics/159 - plan is to not process inj online. SSM inj will be available offline but not online. - `PASTRO_READY` and `HAS_SSM` will be needed for sending out SSM alerts online. We will upload empty pastro files. Superevents will be generated, and if the AllSky analyses upload gevents with similar gps times, these will be included in the same superevent. Alerts will be sent out for high significant events. If the AllSky events have high significance, those will be preferred over SSM events. - `HIGH_PROFILE`: we will not have pastro or p_terr - **~~MDC 27: upload to gracedb-test instead of gracedb-playground. Both of the checkerboards will run on AllSky inj.~~** - As per RODA requirements, **MDC 27: upload to gracedb-test instead of gracedb-playground. Both of the checkerboards will run on AllSky inj.** ### Online Resources (MDC) We need 1170 job slots for one checkerboarded MDC, 2340 job slots in total. - NEMO: (by 01/08/2023) total 86 nodes with 38 jobs per nodes, giving us 3268 online slots, there are 9 nodes on NEMO that are currently down (once they are back, we will have 342 more slots, relieving the eviction issue) - ICDS: all nodes together provide 8252 online jobs, 3 MDC are currently running (Rick, 1 MDC for online injection streamer, and 1 SSM checkerboarded MDC), leaving \~2000 slots left (Cort's check, 3724 slots are available/not-busy) - CIT: (by 01/08/2023) all max-out ### Offline `offline_new-workflow_o4ssm`: created off of `offline_new-workflow` on Nov 13 (last git hash: `b670d9ab`). [link](https://git.ligo.org/lscsoft/gstlal/-/commits/offline_new-workflow_o4ssm) - commits added: - ssm dtdphi file: [link](https://git.ligo.org/lscsoft/gstlal/-/commit/2816093f5b350bc01f632b6c5650e4889dff2039) - test SSM offline workflow - actual O4a production - computing estimation: for a week of data, we expect 10 hours for the non-inj dag with 100 K jobs to finish with 2000 CPUs. (See [issue 1144](https://git.ligo.org/gstlal/projects/-/issues/1144) and [issue 1130](https://git.ligo.org/gstlal/projects/-/issues/1130).) ### Offline: - ROTA: https://git.ligo.org/gstlal/projects/-/issues/1192 - NEMO: - runs directory: `/home/gstlalcbc.offline/observing/4/SSM/a/runs` - build: - CIT: - runs directory: `/home/gstlalcbc.offline/observing/4/SSM/a/runs` - build: `/home/gstlalcbc.offline/observing/4/SSM/a/builds` ### Low-Latency Requirement (Online) - `RODA`: https://dcc.ligo.org/DocDB/0191/M2400032/003/RODA_pipeline_participation_O4b.pdf - `Requirement for Search Pipeline to Run online`: https://dcc.ligo.org/DocDB/0185/M2200264/003/Requirements%20for%20pipelines%20to%20run%20online.pdf ### Divya's Response to LLAI (as an summary of MDC code versions, duration, GraceDB query, etc: ``` Dear Roberto, The gstlal SSM uploads that correspond to the code version closest to production was made from Feb 27 to March 16 2024. The analyses continued to run after the MDC cycle ended on 16th march 2024, therefore there are uploads from MDC channels upto March 20th 2024.The query that would result in events from close-to-production code is as follows: PIPELINE = gstlal SEARCH = SSM gpstime = 1393033340 (02/27/2024) to gpstime = 1394582418 (03/16/24, end of MDC cycle) The gstlal branch for SSM analyses is: o4b-online_ssm which was used during MDCs for dev work. The MDC in question was run with hash ee1561. At the start of O4b, the production analysis was launched with version o4b-online_ssm-v1.0.0 which contains the ee1561 and few other commits on top that were tested and added to all gstlal low-latency analyses before production. Currently the SSM analyses are running with o4b-online_ssm-v1.1.1 .Prior to the above mentioned MDC, we ran on the SSM-MDC(SSM injections in O3 Replay data) upto Feb 11 with a code version where empty pastro had not been implemented, so in my understanding data from January upto Feb 22 wouldn't correspond to the appropriate channels for O3Replay+Injection data we want to test with.Please let me know if this is insufficient information, I'll try to provide other required details asap. Thank you! With regards, Divya ```