Note
Centered Kernel Alignment (CKA) [1] is a similarity index between representations of features in neural networks, based on the Hilbert-Schmidt Independence Criterion (HSIC) [2]. Given a set of examples, CKA compares the representations of examples passed through the layers that we want to compare.
Given two matrices
This project requires python >= 3.9. All the necessary packages can be installed with
pip install -r requirements.txt
Take a look at main.py
for a simple use case.
Model compared with itself | Different models compared |
---|---|
[1] Kornblith, Simon, et al. "Similarity of neural network representations revisited." International Conference on Machine Learning. PMLR, 2019.
[2] Wang, Tinghua, Xiaolu Dai, and Yuze Liu. "Learning with Hilbert–Schmidt independence criterion: A review and new perspectives." Knowledge-based systems 234 (2021): 107567.
This project is also based on the following works:
- https://github.com/google-research/google-research/tree/master/representation_similarity (original implementantion).
- https://github.com/AntixK/PyTorch-Model-Compare (nice PyTorch implementation that employs hooks).
This project is MIT licensed.