Skip to content

Changes

Page history
Update 112922 authored by Shio Sakon's avatar Shio Sakon
Show whitespace changes
Inline Side-by-side
West-call/112922.md
View page @ f28a2de0
......@@ -15,33 +15,33 @@ Shio
Char / Minute Taker / Focus Session [Rota](https://git.ligo.org/groups/gstlal/-/wikis/Meetings/West-call/Rota-table)
## Action Items for next week
* [ ] Everyone: Go to R&P call! Rates and Populations call this Thursday/Friday at 14 PT / 17 ET / 23 CET (Dec 1), 9 AET (Dec 2)
## Action Items from last week
* [ ] Prathamesh, Leo: Investigate high RAM usage of offline jobs
- Prathamesh got profiling results (memory usage vs time for inspiral job). Couple of places indicate memory leaking. Haven’t heard back from him. Ongoing. [plot](https://ldas-jobs.ligo.caltech.edu/~prathamesh.joshi/RAM/inspiral_inj/memvstime.png )
* [ ] everyone : look at R&P injection proposal [Slides presented at All-sky](https://docs.google.com/presentation/d/1TQQr6vZZCUW5Pw67qL4w5IHslc42TS58bT9b3YhzWUw/edit?usp=sharing) :track_next:
* Surabhi: R&P discussion. R&P proposing to generating inj after data collection. Asking if inj can be provided before data taking with expected PSD. We need to looking to converting hdf5 to xml or figure out frame generation in low latency.
* Chad: proposing to use existing mdc frame work. It would be waste of everyone’s time.
* Becca: i thought they were gonna make frames for us?
* Prathamesh got profiling results (memory usage vs time for inspiral job). Couple of places indicate memory leaking. Haven’t heard back from him. Ongoing. [plot](https://ldas-jobs.ligo.caltech.edu/~prathamesh.joshi/RAM/inspiral_inj/memvstime.png )
* [x] everyone : look at R&P injection proposal [Slides presented at All-sky](https://docs.google.com/presentation/d/1TQQr6vZZCUW5Pw67qL4w5IHslc42TS58bT9b3YhzWUw/edit?usp=sharing) :track_next:
* Surabhi: R&P discussion. R&P proposing to generate inj after data collection. Asking if inj can be provided before data taking with expected PSD. We need to look at converting hdf5 to xml or figure out frame generation in low latency.
* Chad: proposing to use the existing mdc framework. It would be a waste of everyone’s time.
* Becca: I thought they were gonna make frames for us?
* Surabhi: Chad, maybe that will work out. But in MDC we already have the data. Idk if they are producing frames on the fly. Shawn and Patrick Brokhill and Reed will be the ones to ask.
* Chad: no reason to not be able to make frame data on the fly. Using existing MDC frame work would be less review etc.
* Chad: no reason to not be able to make frame data on the fly. Using the existing MDC framework would be less review etc.
* Jolien: sth might be need to changed to use hdf5 file because that was not what was used before.
* Surabhi: they actually did use hdf5 files in the MDC
* Chad: would not be trouble to produce multiple sets of frames.
* Jolien: there might not be enough computers to run it
* Chad: could arrange inj set to roughly correspond to chirp mass bins. Without much work on our end we could optimize online computation.
* Rachael: from the ppl who were planing to making offline inj, they were surprise that we wanted online inj, and concern for person power. There was some proposed solution.
* Rachael: from the ppl who were planning to make offline inj, they were surprised that we wanted online inj, and concerned for person power. There was some proposed solution.
* Jolien: yeah, I agree ... another idea is to do 5 nearly-hopeless injections between each hopeful one to reduce overlap issues (e.g., for clustering)
* Leo: I think Tom's suggestion is similar to what Chad said
* Chad: we’re not asking for two data sets. We’re asking for inj to measure the pipeline sensitivity prior to data collection.
* Becca: i think they mentioned that they didn't know if they could have the offline set ready in advance of online
* Leo: they need to know PSD to make a injection set
* Becca: I think they mentioned that they didn't know if they could have the offline set ready in advance of online
* Leo: they need to know PSD to make an injection set
* Surabhi: I did float around the idea of using the PSD of the previous month to make inj. They were concerned they had to make two inj sets for online and offline and that they don’t have enough person power. We need to get back to Reed. MBTA and GstLAL are asking for online inj. PyCBC is not interested in doing inj using online.
* Rachael: and I believe they were talking about there producing injections every month based on the PSD from the previous month. Not just having one set at the start of the run used the whole time
* Rachael: I believe they were talking about their producing injections every month based on the PSD from the previous month. Not just having one set at the start of the run used the whole time
* Chad: we don’t need to reevaluate the PSD. Ppl might need to reevaluate SNR. O3 PSD was stable.
* Surabhi: I agree. The low cut they are using it too low. The rates ppl what to lower the low freq cutoff when using not actual PSD. Since only MBTA and GstLAL are interested in using online inj, we could ask for higher network snr cut.
* Chad: online and offline inj should be identical with some sensible cut on what is hopeless to set a baseline sensitivity for pipelines. Would want to do apples-to-apples VT comparison between online and offline. Identical inj would be necessary.
* Surabhi: I agree. The low cut they are using is too low. The rates ppl what to lower the low freq cutoff when using not actual PSD. Since only MBTA and GstLAL are interested in using online inj, we could ask for a higher network snr cut.
* Chad: online and offline inj should be identical with some sensible cut on what is hopeless to set a baseline sensitivity for pipelines. Would want to do an apples-to-apples VT comparison between online and offline. An identical inj would be necessary.
* Surabhi: so are we willing to increase the number of injections? If we were to ask them to generate inj with uncertain PSD, they want to lower SNR cut. We need to get back to them.
* Chad: we should have firm bounds on LIGO sensitivity. Is it their concern that we’ll not have enough recovered inj?
* Leo: if we can justify SNRcut=4 is too low in the first place, can the online frame be produced with SNRcut~4-ish?
......@@ -49,20 +49,20 @@ Char / Minute Taker / Focus Session [Rota](https://git.ligo.org/groups/gstlal/-/
* Jolien: didn’t want to do a lot of inj.
* Chad: that seems extreme.
* Jolien: they think they need to do inj every 3 sec. I agree with you that they should do optimistic PSD, make inj, half of them are hopeless is fine.
* Chad: it makes the bookeeping harder and more error prone
* Chad: it makes the bookkeeping harder and more error-prone
* Chad: Monte Carlo errors should be able to tolerate a factor of two
* Jolien: use previous month’s PSD to generate next month’s PSD.
* Chad: why is this not boil down to some program doing this online?
* Surabhi: Chad I completely agree. There is not enough person power avilable in rates group (pretty much only Reed). Request wan’t taken seriously because only Surabhi was on the call.
* Jolien: use the previous month’s PSD to generate next month’s PSD.
* Chad: why is this not boiled down to some program doing this online?
* Surabhi: Chad I completely agree. There is not enough person power available in the rates group (pretty much only Reed). My request wasn’t taken seriously because only Surabhi was on the call.
* Chad: we should be analyzing online inj so we need a backup plan. Becca had jobs generating inj as it runs but not optimal for online. We might have to get collaboration going with Patrick Brokhill to get this done ourselves if they aren’t doing it.
* Jolien: this week the rates call is later than normal (rates-East call)
* Jolien: always could do subsets of inj online and the remaining ones offline so that you have some result
* Surabhi: FWIW, I think they can definitely produce injection sets that are not identical to the offline (but the same distribution). We can make the case for them to be identical, can try to put something together for next week.
* Chad: that would be ok, just slightly less useful. I just really want us to not have to be the ones doing this...
* ACTION ITEM Rachel: we should all go to rates call and R&D call.
* Surabhi: I think it'll also be useful if someone could present the test suite to the rates group, so they understand what we are doing with it and might also be more invested in seeing these results in low-latency
* Jolien: Rates and Populations call this Thursday/Friday at 14 PT / 17 ET / 23 CET (Dec 1), 9 AET (Dec 2)
* [ ] Prathamesh, Cort : implement dynamical memory request for offline inspiral job
* [x] Prathamesh, Cort : implement dynamical memory request for offline inspiral job
* Leo: got suggestion from Cody after he talked with some osg ppl. Manually changed condition in submit file related to memory release etc. it is working. Could add to regular workflow.
* [ ] Leo, Becca : test Rocky container with Jacob.
* [ ] Bring up new lalsuite and lalapps release on East Call after Becca's dag.
* [ ] Victoria: Create a new LIGO LW table for incorporating new injection file format
......