Online new extinction
How the marginalize_likelihoods_online program worked before is:
- For each registry, get the rankingstat from the inspiral job and run calc_rank_pdfs of it
- Add it to the variable
data
- retry the failed bins
-
-
data
now contains the newly-created noise PDFS from (almost) every bin
-
- get the marginalized zerolag from the trigger counter job and add it to the data variable. This should be the zerolag from the start of the analysis till now
- If a previously saved marg dist_stat_pdf exists, load it and add it to
data
-
data
now contains not just the newly created noise PDFS, but also the noise PDFs from the start of the analysis till now - Save
data
to disk as the new version of the marg dist_stat_pdf file
With these changes, the marginalize_likelihoods_online implements the new extinction model. This involves first-round-extinction of the bin-specific PDFs with the bin-specific zerolag, adding these singly-extincted PDFs together, and adding the clustered zerolag so that it's ready for second-round-extinction. Now, the marginalize_likelihoods_online job works as:
0. Now the marg job also saves the bin-specific dist_stat_pdf to disk since they're required for first-round extinction. The path is determined by DataCache.generate
, and not decided by the user. The filename of the marg dist_stat_pdf is still provided by the user
- For each registry, load the old dist_stat_pdf file from disk if it exists
- Get the rankingstat from the inspiral job and run calc_rank_pdfs of it, and add it to the old dist_stat_pdf
- calc_rank_pdfs is retried 3 times in my version. This is because I found it difficult to coordinate the retries with both 1 and 2. But I think it's still possible to do the retries at the end like before, so I'll make that change soon
- Save the new + old dist_stat_pdf to disk
- Get the bin-specific zerolag from the corresponding inspiral job and add it to the new + old dist_stat_pdf (needed for dirst-round extinction)
- Perform first round extinction on this pdf, and add it to the
data
variable - Since in each iteration of the loop we are adding the old and new dist_stat_pdfs to
data
, it contains the noise PDF from the start of the analysis to now. No need to do step 6 from the previous version of the code - Remove the bin-specific zerolags and add the marginalized zerolag from the trigger counter job. This should be the zerolag from the start of the analysis till now
- Save data to disk as the new version of the marg dist_stat_pdf file
Edited by Prathamesh Joshi