Does the correlation calculation of mutual information support complex random variables?

[nighthink]

I am very grateful to the author for organizing and implementing related algorithms! However, I recently needed to calculate the mutual information (or conditional mutual information) between two complex random variables, but I tried several methods in the package and found that they only support real numbers. I would like to know what methods support the calculation of mutual information of complex random vectors?

[kahaaga]

Hey, @nighthink! Good questions. I can't remember encountering any differential mutual information estimators in the literature that explicitly deal with complex data, but I haven't looked for them explicitly, so they may very well exist.

That said, there's nothing preventing you from just symbolizing your complex-valued data points into discrete data points, and then compute mutual information from the symbolized datasets using some discrete estimator. How to do this translation, however, I can't really say anything about from the top of my head - it depends on what properties of your data you're trying to capture.

[kahaaga]

@nighthink See also discussion in the linked issue.

[nighthink]

另请参阅链接问题中的讨论。

Thank you very much for your help, I will try a new solution based on your suggestions. I checked the references in the project and they do not restrict the data type. The reason for this problem is that the third-party package used by CausalityTool does not provide support for calculations related to complex numbers. For example, if I want to get the MI of two complex random variables by:

using CausalityTools
using Distributions
using Statistics

n = 1000
using CausalityTools
x = randn(ComplexF64,1000)
y = rand(ComplexF64,1000) .+ x
mutualinfo(KSG1(k = 5), x, y)
mutualinfo(GaussianMI(), x, y) # defaults to `MIShannon()`

I will get

Stacktrace:
 [1] compute_bbox(data::Vector{SVector{2, ComplexF64}})
   @ NearestNeighbors ~/.julia/packages/NearestNeighbors/dZqbT/src/hyperrectangles.jl:15
 [2] KDTree(data::Vector{…}, metric::Chebyshev; leafsize::Int64, storedata::Bool, reorder::Bool, reorderbuffer::Vector{…})
   @ NearestNeighbors ~/.julia/packages/NearestNeighbors/dZqbT/src/kd_tree.jl:61
 [3] KDTree(data::Vector{SVector{2, ComplexF64}}, metric::Chebyshev)
   @ NearestNeighbors ~/.julia/packages/NearestNeighbors/dZqbT/src/kd_tree.jl:25
 [4] KDTree(D::StateSpaceSet{2, ComplexF64}, metric::Chebyshev; kwargs::@Kwargs{})
   @ StateSpaceSets ~/.julia/packages/StateSpaceSets/3YtJp/src/neighborhoods.jl:9
 [5] estimate(::MIShannon{…}, ::KraskovStögbauerGrassberger1{…}, ::Vector{…}, ::Vector{…})
   @ CausalityTools ~/.julia/packages/CausalityTools/alYDd/src/methods/infomeasures/mutualinfo/estimators/KSG1.jl:63
 [6] estimate(est::KraskovStögbauerGrassberger1{Chebyshev, Chebyshev}, x::Vector{ComplexF64}, y::Vector{ComplexF64})
   @ CausalityTools ~/.julia/packages/CausalityTools/alYDd/src/methods/infomeasures/mutualinfo/mutualinfo.jl:177
 [7] mutualinfo(::KraskovStögbauerGrassberger1{Chebyshev, Chebyshev}, ::Vararg{Any}; kwargs::@Kwargs{})
   @ CausalityTools ~/.julia/packages/CausalityTools/alYDd/src/methods/infomeasures/mutualinfo/mutualinfo.jl:53
 [8] top-level scope
   @ ~/Onedrive/code/Julia/EIT/scripts/get_numerical_solutions.jl:24
Some type information was truncated. Use `show(err)` to see complete types.

This is because an algorithm in hyperrectangles.jl utilizes a function (named typemin) which is not support the complex value input. I think these problems are quite troublesome to deal with because there are many packages whose algorithms are implemented without considering the extension of the complex domain. Thanks again for your help and providing such a great package!!

[kahaaga]

Hey @nighthink,

Sorry for the delay response here. CausalityTools supports something like this (remember to install the latest version (v3) for the syntax below to work:

using CausalityTools
n = 5000
x = rand(ComplexF64, n)
y = rand(ComplexF64, n) .* x
z = rand(ComplexF64, n) .+ y

# Convert to `StateSpaceSet`s consisting of one column each for the real and imaginary parts
X =  StateSpaceSet(real.(x), imag.(x))
Y =  StateSpaceSet(real.(y), imag.(y))
Z =  StateSpaceSet(real.(z), imag.(z))

est_mi = KSG1(MIShannon(); k = 10) # define a mutual information estimator
est_cmi = FPVP(CMIShannon(); k = 10) # define a conditional mutual information estimator

association(est_mi, X, Y), association(est, X, Z), association(est_cmi, X, Z, Y)

This gives

julia> association(est_mi, X, Y), association(est, X, Z), association(est_cmi, X, Z, Y)
(1.254795355430503, 0.4466664929348844, -0.042325931085138165)

Seems like a meaningful thing to me? The association (in terms of mutual information) between X and Y is stronger than for X and Z, which is expected, since there is an intermediate variable between X and Z. Also, when conditioning on Y, the association between X and Z vanishes.

[nighthink]

@kahaaga Thank you very much for providing detailed examples and adding them to the documentation. I believe this will be incredibly helpful, not only for me but also for many others, especially those working in signal processing and related fields!

[kahaaga]

Since this now has another open issue for adding a documentation on complex-valued data, I'm closing this issue.