Log-sum-exp and softmax functions

About

logsumexp and softmax evaluate the log-sum-exp function $lse(x) = log ∑_i=1^n e^x_i$ and the softmax function $g(x)$ with $g_j(x) = \frac{e^x_j}{∑_i=1^n e^x_i$, where x is a vector. Softmax is the derivative of log-sum-exp.

Also provided is test, which runs some simple tests of the functions.

Usage

The lines

sm = softmax(x)
[sm,lse] = softmax(x)
lse = logsumexp(x)
[lse,sm] = logsumexp(x)

compute the softmax sm and the log-sum-exp lse at the vector x. Both functions can compute both quantities because there is significant overlap in the computations of the two functions.

Requirements

The code was developed in MATLAB R2020b and works with versions back to at least R2016b.

References

P. Blanchard, D. J. Higham, and N. J. Higham. Computing the Log-Sum-Exp and Softmax Functions. IMA J. Numer. Anal., Advance access, 2020.

License