Structural Estimates : "Warning: Maximum number of function evaluations has been exceeded."

[sbenthall]

The Structural Estimates DemARK is running suspiciously wrong. The checked in version runs for 40 minutes; local it's running for 20 minutes with the following message:

https://gist.github.com/sbenthall/1f16a6764b17024eb3be3dc9a68852c0

There's a warning: "Warning: Maximum number of function evaluations has been exceeded."
This is associated with the underlying scipy.optimize.fmin method, which has a default "maxfun" argument for maximum function evaluations. This default is being used by the underlying HARK call.

Some ideas:

• The objective function FagerengObjFunc defined in the notebook may have too many functions inside it because of the new distribution code or other underlying HARK implementations
• The parameters have been reset somehow so that there is no unconstrained minimum for this objective function, causing the fmin method to search endlessly

I'm wondering who has a deep understanding of what this notebook is trying to accomplish such that they can explain what's going on in the Fagereng objective function. The GitHub history indicates that @llorracc and @MridulS are the others who have touched the notebook in the repository.

[mnwhite]

`maxfun` only counts objective function evaluations. But looking at that output, it looks like the minimizer is making no progress after the first few iterations-- it's just getting stuck and going in a weird loop. I wrote the code for this about three years ago and need to look at it carefully to see what's going on.

…

On Mon, Jul 6, 2020 at 5:27 PM Sebastian Benthall ***@***.***> wrote: The Structural Estimates DemARK is running suspiciously wrong. The checked in version runs for 40 minutes; local it's running for 20 minutes with the following message: https://gist.github.com/sbenthall/1f16a6764b17024eb3be3dc9a68852c0 There's a warning: "Warning: Maximum number of function evaluations has been exceeded." This is associated with the underlying scipy.optimize.fmin method, which has a default "maxfun" argument for maximum function evaluations. This default is being used by the underlying HARK call. Some ideas: - The objective function FagerengObjFunc defined in the notebook may have too many functions inside it because of the new distribution code or other underlying HARK implementations - The parameters have been reset somehow so that there is no unconstrained minimum for this objective function, causing the fmin method to search endlessly I'm wondering who has a deep understanding of what this notebook is trying to accomplish such that they can explain what's going on in the Fagereng objective function. The GitHub history indicates that @llorracc <https://github.com/llorracc> and @MridulS <https://github.com/MridulS> are the others who have touched the notebook in the repository. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#123>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADKRAFMPSEIFAPY4AMULEZ3R2I6S5ANCNFSM4OR7NDZQ> .

[MridulS]

I went digging into this and it looks like the notebook broke sometime after the commit econ-ark/HARK@f684667 to HARK master (this was the last time travis passed happily for DemARKs)

Use the following git command to see the changes in code between f68566 and master.

git diff master f68466795f2cb986e6943c9e147568a0866f6c3 HARK/ConsumptionSaving

The change in behaviour of Structural notebook is due to the recent changes here, but I am not able to pin point it right now.

[mnwhite]

Sebastian's hypothesis from today's call was partially correct. The issue was RNG not resetting properly, but not within the notebook, but instead in IndShockConsumerType (and all our other classes). The HARK branch MoreRNGfixes has a commit that makes this notebook run properly, but I need to put in that extended method for other AgentType subclasses as well.

The gist of the fix is that resetRNG() needs to call the reset method of each Distribution instance created by the class.