Add LogSumExp functions
Closed this issue · 3 comments
delphidabbler commented
LogSumExp
LogSumExp or LSE for a sequence x1,...,xn is is defined as the logarithm of the sum of the exponentials of the arguments:
LSE(x1,...,xn) = log(exp(x1) + ... + exp(xn))
Source: Wikipedia
function LSE(const A: array of Extended): Extended;
begin
var Sum: Extended := 0.0;
for var X in A do
Sum := Sum + Exp(X):
Result := Ln(Sum);
end;Strictly Convex LogSumExp
Source: Wikipedia
Since exp(0) = e^0 = 1, we have
LSE(0, x1,...,xn) = log(exp(0) + exp(x1) + ... + exp(xn))
= log(1 + exp(x1) + ... + exp(xn)),
So, in Pascal we can have:
function ConvexLSE(const A: array of Extended): Extended;
begin
var Sum: Extended := 1.0; // for additional Exp(0) term
for var X in A do
Sum := Sum + Exp(X):
Result := Ln(Sum);
end;This issue was extracted from issue #16
delphidabbler commented
The Softmax function is closely related to LSE. This should also be implemented.
delphidabbler commented
Strictly Convex LogSumExp
Decided not to implement this.
delphidabbler commented
Implemented by merge commit 4186f6e
