This is the code for the preprint paper "Estimating Transfer Entropy via Copula Entropy" which available at here. A non-parametric method for estimating Transfer Entropy via estimating three Copula Entropy terms is proposed in this paper.
The proposed method is implemented in the R and Python package 'copent', available at
The method is demonstrated in the experiment with the UCI Beijing PM2.5 data. The following conditional independence measures are compared in the experiment:
- Transfer Entropy via Copula Entropy (TE) [1];
- Conditional Distance Correlation (CDC) [2];
- Kernel-based Conditional Independence (KCI) [3];
- COnditional DEpendence Coefficient (CODEC) [4];
- Generalised Covariance Measure (GCM) [5];
- weighted Generalised Covariance Measure (wGCM) [6];
- Kernel Partial Correlation (KPC) [7];
- Partial Correlation (pcor);
- Randomized conditional Correlation Test (RCoT) [8];
- kNN based Conditional Mutual Information Estimators [9,10];
- Fast Conditional Independence Test (fcit) [11];
- Model-Powered Conditional Independence Test (CCIT) [12];
- Predictive Conditional Independence Testing (PCIT) [13];
- Conditional Kendall's Tau (CKT) [14];
- Conditional Mean Dependence (CMD) [15];
- Partial Copula based CI test [16].
For more comparison experiments on conditional independence measures, please refer to our paper "Evaluating Independence and Conditional Independence Measures" at here and the assoicated code at here.
- Ma, J. Estimating Transfer Entropy via Copula Entropy. arXiv preprint arXiv:1910.04375, 2019.
- Wang, X.; Pan, W.; Hu, W.; Tian, Y. & Zhang, H. Conditional distance correlation. Journal of the American Statistical Association, 2015, 110, 1726-1734.
- Zhang, K.; Peters, J.; Janzing, D. & Schölkopf, B. Kernel-based conditional independence test and application in causal discovery. Uncertainty in Artificial Intelligence, 2011, 804-813.
- Azadkia, M. & Chatterjee, S. A simple measure of conditional dependence. arXiv preprint arXiv:1910.12327, 2019.
- Shah, R. D. & Peters, J. The hardness of conditional independence testing and the generalised covariance measure. Annals of Statistics, 2020, 48, 1514-1538.
- Cyrill Scheidegger, Julia Hörrmann, Peter Bühlmann. The Weighted Generalised Covariance Measure. arXiv preprint arXiv:2111.04361, 2021.
- Huang, Z.; Deb, N. & Sen, B. Kernel Partial Correlation Coefficient -- a Measure of Conditional Dependence. arXiv preprint arXiv:2012.14804, 2020.
- Strobl, E. V.; Zhang, K. & Visweswaran, S. Approximate Kernel-based Conditional Independence Tests for Fast Non-Parametric Causal Discovery. arXiv preprint arXiv:1702.03877, 2017.
- Runge, J. (2018). Conditional independence testing based on a nearest-neighbor estimator of conditional mutual information. In AISTATS'18.
- Octavio César Mesner, Cosma Rohilla Shalizi. Conditional Mutual Information Estimation for Mixed Discrete and Continuous Variables with Nearest Neighbors. arXiv preprint arXiv:1912.03387, 2019.
- Krzysztof Chalupka, Pietro Perona, Frederick Eberhardt. Fast Conditional Independence Test for Vector Variables with Large Sample Sizes. arXiv preprint arXiv:1804.02747, 2018.
- Rajat Sen, Ananda Theertha Suresh, Karthikeyan Shanmugam, Alexandros G. Dimakis, Sanjay Shakkottai. Model-Powered Conditional Independence Test. NIPS 2017: 2951-2961.
- Samuel Burkart, Franz J Király. Predictive Independence Testing, Predictive Conditional Independence Testing, and Predictive Graphical Modelling. arXiv preprint arXiv:1711.05869, 2017.
- Alexis Derumigny, Jean-David Fermanian. A classification point-of-view about conditional Kendall’s tau. Computational Statistics & Data Analysis, 135, 70-94, 2019.
- Xiaofeng Shao, Jingsi Zhang. Martingale Difference Correlation and Its Use in High-Dimensional Variable Screening. Journal of the American Statistical Association, 109(507), 1302-1318, 2014.
- Petersen, L., & Hansen, N. R. Testing Conditional Independence via Quantile Regression Based Partial Copulas. Journal of Machine Learning Research, 22, 1-47, 2021.