Optimality-Guarantees-for-Policy-Gradient-Methods IPython notebooks for code to produce Figure 1 in the paper.