Implement the improved covariance estimation for large class of metrics from paper 'Random Matrix Improved Covariance Estimation for a Large Class of Metrics, ICML 2019'
Version 1 Implementation of the algorithm 2 for linear shrinkage initialization--> Faster implementation
Version 2 Implementation of the algorithm 1 for general initialization
Implement the improved covariance estimation for large class of metrics
Version 1 Implementation of the algorithm 2 for linear shrinkage initialization--> Faster implementation
Version 2 Implementation of the algorithm 1 for genral initialization
1- The function implementing our method is called \texttt{RMTest.m} which takes as arguments the data matrix \texttt{X}, the \texttt{gradient_check} and \texttt{plot_cost} options, the initial point \texttt{C_0} and the distance under investigation. The function returns the estimated covariance matrix \texttt{C_est} and the cost function related to this estimated covariance.
2- The main script comparing all algorithms for synthetic data is \texttt{CompareEst.m}.
3- The main script comparing the estimation methods for the LDA/QDA application is \texttt{ML_applications.m}.
4- folder \texttt{Manopt}: a Matlab toolbox for optimization on manifolds \cite{manopt}. Some of the functions in this folder were adapted to better suit our present problem.
5- folder \texttt{Othermethods}: containing alternative estimation methods among which the QuEST methods QuEST1 and QuEST2 from \cite{LW15,LW18}, the Rao-Blackwell Ledoit-Wolf estimation methods \cite{chen2010shrinkage} and the Oracle Approximating Shrinkage estimation method \cite{chen2010shrinkage}.
6- folder \texttt{Utilities}: contains supplementary codes for the implementation of LDA,QDA.
The different options proposed to execute the script \texttt{CompareEst.m} comparing the different estimation algorithms are as follows:
1- The range of
2- The covariance matrices metric ``\texttt{distance}'' (among \texttt{Fisher, Battacharrya, KL, log, log1st, t}) or for the precision matrices (among \texttt{Inverse_Fisher, ,Inverse_Battacharrya,Inverse_KL,} \texttt{Inverse_log1st, Inverse_log,Inverse_t})
1- The function implementing our method is called \texttt{RMTest.m} which takes as arguments the data matrix \texttt{X}, the \texttt{gradient_check} and \texttt{plot_cost} options, the initial point \texttt{C_0} and the distance under investigation. The function returns the estimated covariance matrix \texttt{C_est} and the cost function related to this estimated covariance.
2- The main script comparing all algorithms for synthetic data is \texttt{CompareEst.m}.
3- The main script comparing the estimation methods for the LDA/QDA application is \texttt{ML_applications.m}.
4- folder \texttt{Manopt}: a Matlab toolbox for optimization on manifolds \cite{manopt}. Some of the functions in this folder were adapted to better suit our present problem.
5- folder \texttt{Othermethods}: containing alternative estimation methods among which the QuEST methods QuEST1 and QuEST2 from \cite{LW15,LW18}, the Rao-Blackwell Ledoit-Wolf estimation methods \cite{chen2010shrinkage} and the Oracle Approximating Shrinkage estimation method \cite{chen2010shrinkage}.
6- folder \texttt{Utilities}: contains supplementary codes for the implementation of LDA,QDA.
The different options proposed to execute the script \texttt{CompareEst.m} comparing the different estimation algorithms are as follows:
1- The range of
2- The covariance matrices metric ``\texttt{distance}'' (among \texttt{Fisher, Battacharrya, KL, log, log1st, t}) or for the precision matrices (among \texttt{Inverse_Fisher, ,Inverse_Battacharrya,Inverse_KL,} \texttt{Inverse_log1st, Inverse_log,Inverse_t})
3- The target matrix \texttt{Covariance}'' (among \texttt{dirac,Wishart, toeplitz}) and their parameters
\texttt{param}'' if needed (for the Wishart and Toeplitz cases)
4- The initialization point for the gradient descent algorithm denoted \texttt{initialization}'' (linear shrinkage from \cite{LED04} denoted
\texttt{shrinkage}'', shrinkage from \cite{chen2010shrinkage} denoted \texttt{alternative shrinkage}'', QuEST denoted as
\texttt{ledoit-wolf}'') or the identity denoted ``\texttt{manual}''
5- Other binary option can be chosen (among 0/1): (\texttt{gradient_check} to check if the gradient is correct, \texttt{plot_cost} to see the cost/real distance during iterations. Code \texttt{ML_applications.m} The different options proposed to execute the script \texttt{ML_applications.m} are as follows:
1- the data on which the LDA/QDA are applied denoted ``\texttt{dataset}'' for which the options are \texttt{synthetic} for synthetic data and \texttt{eeg} for eeg dataset.
2- The machine learning algorithms denoted ``\texttt{application}'' for which the options are \texttt{LDA} and \texttt{QDA}.
3- For synthetic data, the examples of covariance under investigation. For both the covariance of the first and second class, \texttt{Wishart} and \texttt{toeplitz} are the two options.
The following sections detail the parameter setting to reproduce the figures of the main article.
Script
Script
covariance
n
distance
Initialisation
plot_cost
Script
covariance
n
distance
Initialisation
plot_cost
Script
mu2
application
covariance1
covariance2
p
Script
mu2
application
covariance1
covariance2
dataset
p
eeg dataset
Please contact tiomoko_malik@yahoo.fr for further details