Add gradient and hessian as optional standard methods

[jwhhh]

It might make sense to add the gradient and hessian as additional helper methods (as a part of Espresso standard, optional for contributors to implement)

So there are two questions:

• Do we add them?
• Do we adjust the misfit method? Now it takes data and predicted data as input and calculate misfit, should we change it into a more general misfit function that takes in only model as the argument?

What do you think, @valentineap ?

[valentineap]

I don't have a good answer to this.

I think if we're going to make provision for a problem-specific misfit function within Espresso, it makes sense to also provide for its gradient and Hessian. However I am a bit concerned that we create confusion -- we already have a gradient function (jacobian), and our Espresso API is getting ever-bigger.

One option might be to wrap misfit into a class that can have its own gradient/hessian/etc methods. I can see some attraction to doing this, but it does build more complexity...

We could indeed think about making it misfit(data, model) rather than misfit(data1,data2). However, my motivation for keeping it in the data space is that it's more flexible -- the misfit is properly defined in data space, and one might(?) want to use it to compare data vectors that don't come out of the forward code. On the other hand anyone doing that can probably manage to sort it out for themselves.

Perhaps the best thing is the simplest: make it misfit(model, data=data) and allow for misfit_gradient and misfit_hessian?