Kac Independence Measure (KacIM) is bivariate statistical independence measure, which can detect arbitrary statistical dependence between two random vectors (similar to mutual information, Hilbert-Schmidt independence criterion (HSIC), distance covariance/correlation, etc.). The idea of KacIM is to maximimize lenght of difference between joint and product marginal characteristic functions (two complex random variables):
This repository includes basic implementation of KacIM, toy-data demonstrations, which show that KacIM works for high-dimensional data (e.g. 512-dimensional input, 512-dimensional output or similar), and feature extraction example, which demonstrates, that KacIM allows to improve classification accuracy on real data.
Article/preprint is currently being prepared: Article draft.
In this article we identify that KacIM is related to distance correlation in common
Example: This graph show KacIM evaluations during gradient optimization looks like for independent data (blue), dependent data with additive (orange) and multiplicative noise (green) (500 iterations):
In independent case the estimator does not converge, meanwhile in dependent cases it does.