Is there a way to tell the optimizer the step is too far?

[damienhocking]

I'm thinking of trying SOLNP on a black-box problem. There are some regions where evaluation might fail, even though all variables are within their bounds. Is there a way to ask the optimiser to take a smaller step?

[KristerSJakobsson]

Thank you for raising this issue and apologies for not seeing it until now.

In general SONLP will only find optimum for smooth functions and constraints, so it's not typically suitable for black-box problems as it would get stuck in local optima pretty quickly. For black-box problems I would recommend Genetic Algorithms or similar.

However, GOSOLNP (see my package pygosolnp) does try to solve smooth functions with multiple local optima by running SOLNP from various random restarts, that might work for some black-box problems and would probably get your further than pure SOLNP. For an explanation of both SOLNP and GOSOLNP, see the docs page.

Back to your question, there is no such thing as a simple Step Size in the sense you would see with BFGS (Quasi-newton methods). The closest thing is the delta parameter, which is the Step Size when calculating partial differentiation. There is also a Tolerance parameter, which is used to determine convergence and when to stop calculation. These can be configured, but only to constant values, and won't really be useful for optimizing a problem with 'regions where evaluation might fail'.

For any problem where you have 'regions where evaluation might fail', I think you will have limited success with any derivative-based algorithm like SOLNP. Genetic Algorithms and Heuristic approaches will probably work better for you. I haven't used the Python implementation, but BOBYQA is a good Heuristic approach that might be good for this: pybobyqa