OPT: Prefer analytical expression of gradient and hessian when provided
AlePalu opened this issue · 2 comments
It would be possible for ScalarField to provide the currently implemented .getGradientApprox() and .getHessianApprox() methods under the more general name .derive() and .deriveTwice().
DifferentiableScalarField and TwiceDifferentiableScalarField can overload the .derive() and .deriveWtice() method of their parent class. In this way derivatives under ScalarField will be implicitly numerically estimated. Differently when the implementation provides an analytical expression for either the gradient or the hessian this will be preferred instead of its numerical approximation.
This mechanism should make optimizers to scale well also when analytical expressions of derivatives are not supplied (i.e. at the moment GradientDescent only works under the DifferentiableScalarField. An user might want to use the gradient descent without knowing the analytical expression of the gradient. In this case is possible to still use the gradient descent but working on a numerical approximation of the gradient).
Moreover this would remove redundant code, i.e. class ExactNewton could be totally absorbed from Newton which will behave as the ExactNewton method once a TwiceDifferentiableScalarField is supplied as objective.
This is partially addressed in 55334be, ScalarField now provides methods .derive() and .deriveTwice(). Both methods have an overload allowing for the choice of the step size used in the approximation of derivatives, if no step size is supplied a fixed one is used (at the moment equal to 0.001).
In case of DifferentiableScalarField and TwiceDifferentiableScalarField a call to .derive() or .deriveTwice() evaluates the exact gradient or hessian respectively.
Since .derive() and .deriveTwice() return both a callable, is still possible to get an approximation of the gradient or hessian at a single point using calls .approxGradient() and .approxHessian() exposed by ScalarField and hence in any of its derived classes.
OPT module still waits to reflect these changes.