Healpix prior

Add a method to read in and use a healpix prior (e.g. from Fermi/INTEGRAL). I suspect this will need some close interaction with the correlated prior.

added Feature-request Good First Issue labels

I would like to take this on.

assigned to @zoheyr-doctor

unassigned @zoheyr-doctor

assigned to @zoheyr-doctor

I've added myself as the assignee for now until @jonathan.merritt is added to the project.

assigned to @jonathan.merritt

Great, I've added @jonathan.merritt to the project and also assigned him.

A word of caution on this one: this is a special case of a correlated prior (between ra/dec and potentially distance), the implementation of which !332 (merged) is currently still in progress (cc @sylvia.biscoveanu, @moritz.huebner, and @matthew-pitkin who have all variously worked on this). I'm not sure how it is best implemented, but there is lots of potential conflict so I suggest everyone make sure to communicate to avoid code clashes :)

That being said: the reading on of healpix priors by itself might be quite a task so that is where I'd personally start.

Thanks, @gregory.ashton, that's good to know. Just trying to catch up on what's been done on the correlated prior issue, and if I'm understanding correctly, @jonathan.merritt and I would need to plug a healpix map into the correlated prior infrastructure (which I will have to familiarize myself with)?

@moritz.huebner @sylvia.biscoveanu @matthew-pitkin We're looking at the conditional prior issue on branch 270: https://git.ligo.org/lscsoft/bilby/blob/270-introduce-correlated-prior-sets/bilby/core/prior.py Can the prior sampler only check if the samples in the conditional variable satisfy a constraint of the unconditional variable or is there an existing way to feed in a joint probability distribution or p(conditional var|unconditional var)? Thanks.

For the LIGO public-facing skymaps for example, we get a healpix map with columns p(pixel) and p(distance|pixel). Is there a natural way this fits into the conditional prior infrastructure? Perusing the prior.py code made me think that you can't quite do this?

@jonathan.merritt @zoheyr-doctor sorry for the late response. I just came back to Melbourne from Europe. I think p(conditional var|unconditional var) is pretty much what I am implementing right now, so we could definitely handle something like p(distance|pixel).

The only major issue right now is there is no way to save/load these priors to/from files, because the condition is implemented via a function which you can't easily save. I don't think I will implement this feature in the current MR.

I'll present the progress on this MR on the bilby dev call on Monday. I am only adding tests right now and fixing some technical issues that I come across.

assigned to @bruce.edelman

No worries, @moritz.huebner. Perhaps it makes most sense to discuss this on the call on Monday.

Thanks for presenting on the dev call @moritz.huebner. Now I have a sense of what you've implemented. I imagine what would happen would be something like:

my_healpix_file = get_my_healpix_file()
def conditional_func_distance(reference_params,ra,dec):
    return dict(mu=my_healpix_file.distmu(ra,dec),
                sigma=my_healpix_file.distsigma(ra,dec)
               )

conditional_prior_dict['distance']=bilby.core.prior.Gaussian(
                                   condition_func = conditional_func_distance, ...
                                   )

Alternatively, we could skip over using the conditional prior stuff you've built and instead implement a whole new Healpix prior object class. It would be initialized with healpix file and then have sample, prob, ln_prob, and cdf methods.

@zoheyr-doctor I was thinking something like that. I am not too familiar with the nature of the Healpix priors, but if they do indeed follow analytical distributions, it should be as easy as that. But even if we have just a number of points with associated probability, we could manage to do that with Interped (interpolated) priors.

I would nicely put this desired behaviour away into a separate function, so that you only have to write something like:

my_healpix_file = get_my_healpix_file()
priors = ConditionalPriorDict()
priors['ra'], priors['dec'], priors['distance'] = get_prior_from_healpix(my_healpix_file)

I also do apologise for the poor audio earlier.

this is partially a followup of the features I presented in #293 (closed).

I think the HealPIX implementation is the way forward (note that we need to support both 2D and 3D FITS files), but that it could also be useful to have a more simple Gaussian skyBlob implemented, but I think that might be covered more by !332 (merged)?

Hi all, We have been working on this, going with the approach of creating a general JointPrior class for similar types of priors in the future. We started by taking @matthew-pitkin 's MultivariateGaussian prior infrastructure and try to generalize it so that all we have to do to create joint priors is to create a Prior that inherits from BaseJointPriorDist then for each parameter included create a JointPrior that stores the instance of the JointPriorDist that it is a part of. For more details see my branch here:

https://git.ligo.org/bruce.edelman/bilby/blob/joint_prior/bilby/core/prior.py#L2437 https://git.ligo.org/bruce.edelman/bilby/blob/joint_prior/bilby/gw/prior.py#L635

We have only implemented 2D HealPix priors for now but it should be straightforward to extend to 3D (if what is here is correct and working). We were hoping to get some insight from other bilby developers here to see if this approach is satisfactory with everyone in terms of infrastructure to make sure there isn't any large conflicts of infrastructure we aren't thinking about.

Curious to here all your thoughts, Thanks!

added 1 deleted label

mentioned in merge request !651 (merged)

closed via merge request !651 (merged)

mentioned in commit 38d1effc

unassigned @zoheyr-doctor, @bruce.edelman, and @jonathan.merritt