thesephist/spectre

Sparse autoencoders for Contra text embedding models

Spectre 👻

Spectre is my investigation into using sparse autoencoders to interpret and manipulate text with my Contra family of text embedding models.

Overview

This repo is still a work in progress, but you can find some end-to-end working sample code in `./notebooks that demonstrates how to use a sparse autoencoder to interpret and manipulate text with Contra models.

References

• Towards Monosemanticity: Decomposing Language Models With Dictionary Learning
• &c.