zfurman56/svd-interpretability-replication

Replication of "The Singular Value Decompositions of Transformer Weight Matrices are Highly Interpretable"

"The Singular Value Decompositions of Transformer Weight Matrices are Highly Interpretable" Replication

Minimal replication of the core result of Millidge & Black's "The Singular Value Decompositions of Transformer Weight Matrices are Highly Interpretable". This replicates OV circuit interpretability via SVD for GPT-2-medium. See the linked paper for context and the full results.