Overcoming a Theoretical Limitation of Self-Attention This code accompanies the paper: David Chiang and Peter Cholak, Overcoming a theoretical limitation of self-attention, to appear in Proc. ACL, 2022. Tested with Python 3.9.6 and PyTorch 1.9.1. This code is in the public domain.