This provides Mutual Information (mi) functions in Python. Currently, this provides the mi between tensors as described by Kraskov et.al.
It takes in two numpy array files as command line inputs and outputs the mutual information in Mutual_information-k*.dat. The numpy array files should have same number of instances, i.e. same first index size. Those instances can be a tensor.
Usage:
import mutual_information as mi
import numpy as np
X = np.load('<numpy array file 1>') # X.size = (N,...)
Y = np.load('<numpy array file 2>') # Y.size = (N,...)
I = mi.pyMIestimator(X, Y) #default k = 5, base = np.exp(1)./mutual_information.py [-k<k value>] <numpy array file 1> <numpy array file 2> [<index for .dat file>]
Options:
-k<k value>
Number of nearest neighbours to account for, default=5
<index for .dat file>
number or index for easy plotting against Mutual information from .dat file
Theoretical maximum Maximum mutual information can be understood to be of a variable with itself, i.e. I(X,X). This is given by:
I = (mi.digamma(N) - mi.digamma(k+1)) / np.log(base) #default k = 5, base = np.exp(1)This also implies that the number of instances cannot be less than or equal to k+1. Good rule of thumb is to have instances >> k (atleast 10 times).