Parametric Entropic Locally Linear Embedding for Small Sample Size Classification Problems
Manifold learning algorithms are powerful non-linear dimensionality reduction mathematical methods for unsupervised metric learning. The locally linear embedding (LLE) method uses the local geometry of the linear neighborhood spaces to estimate optimal reconstruction weights for each sample. In the present paper, we propose the parametric entropic LLE (PELLE) method, which adopts the relative entropy instead of the pointwise Euclidean metric to build local entropic covariance matrices. This methodological improvement increases the robustness of the method regarding noise and outliers. Moreover, state-of-the-art algorithms such as UMAP require a large number of samples for convergence to good results due to numerical optimization methods (gradient descent). Results considering 25 different real-world datasets indicate that the proposed method is capable of generating superior clustering and classification accuracies compared to existing state-of-the-art methods for dimensionality reduction-based metric learning, especially in datasets with a limited number of samples.