Find correspondences between a set of 3D gaussian distributions (quadrics or ellipsoids) and a set of 2d gaussian distributions (conics or ellipses)

[ttsesm]

Hello,

I don't have a formal background in OT, therefore with this issue I am also trying to understand a bit better whether OT could be an option for my task or not.

So briefly, I am having the following problem. Imagine that I have set of 3D gaussian distributions, i.e quadrics or Ellipsoids, in the 3d space and a set of 2D gaussian distributions, i.e. conics or ellipses, projected on an plane. This number of 3D/2D gaussians in each set could be couple of thousands. Moreover, for each of the sets I have all the information that describe these two sets as gaussian distributions respectively. Thus, the mean, μ, and the covariance matrices Σ.

My question is, based on this information is there an approach that I could possibly obtain some good correspondences so that I can match an ellipsoid with an ellipse. Each distribution in each of the sets can be considered as individual distributions or combined. Moreover, it is not necessary that all distributions in one set match with all distributions on the other set. It can be that only few ellipsoids match with some of the ellipses in the other set. I am trying to figure out whether geometrically or statistically is possible to get something based on OT.

The inspiration comes from these works: Generalized Wasserstein barycenters between
probability measures living on different subspaces and this thesis Generalised Wasserstein Barycentres
where some of the authors (@eloitanguy, @rflamary) from what I've noticed are also contributors of the POT library. The idea as described there is that based on some probability measures, marginals or projections (≥2) using OT it possible to recover the 3d probability or the reconstructed generalized barycenter which apparently describes the geometry of the distribution. In my case though I have only one projection, which of course this projection contains multiple 2D gaussians.

To my understanding the Blind Generalised Wasserstein Barycentre Problem could be an approach to follow but I am not quite sure since the whole concept is totally new to me. Thus, any feedback would be appreciated.

[eloitanguy]

Hi,

With BGWB (which is not implemented in POT), you need discrete distributions. A possible avenue would be to sample from your 3D and 2D gaussians, compute a projection P: R^3 -> R^2 which maps your samples, as well as a transport plan $\pi$ which will tell you which 3d sample matches which 2d sample.

The minimisation problem in question would be $\min_{P}W_2^2(P$# $\mu, \nu)$, where $\mu$ is your 3D samples and $\nu$ is your 2D samples. You could solve this using BCD or SGD (similarly to Generalised Wasserstein Barycentres ), but this could yield poor results numerically.

To use Gaussians and not samples, you could look see each Gaussian as a point on the Bures-Wasserstein manifold. To compute an assignment between your 3D gaussians and 2D gaussians, you could then consider the Gromov-Wasserstein distance, which can correspond elements of different metric spaces.

In general, your problem also makes me think of Domain Adaptation, which has been tackled with OT successfully.

Thanks for the interesting question, I don't think we can give an immediate answer with accessible tools :D
Let us know how your thoughts go along!