We identify circuits in LLMs based on Edge Attribution Patching, and Automatic Circuit Discovery. We are currently supporting TransformerLens models only. In the future, we may apply Activation Steering to the identified circuits.
See demo notebook. Oscar implemented EAP by having a very big tensor of activation differences between clean and corrupted activations that we write and read from. We calculate the EAP scores during the backward pass to save memory and only store the difference between positive and negative activations instead of storing both.
- Positional EAP: Compute edge attribution scores across different positions (Answering questions like: "Does the token bomb at some middle position in the prompt have a high impact on the final position?")
- Graph plotting (use ACDC functionality or web-based approach?)
- Generalize Metric: Compute general metric from logits or intermediate activations (
- Activation steering: Abstract away the localization of hooks and positions to enable forward passes with
- Activation Steering (at arbitrary layers and sequence positions)
- Ablation (zero / mean)
- Custom Steering vector
- Write unit tests
- check poetry build requirements