the "whiteboard problem"
Closed this issue · 9 comments
Throughout HARK, the AgentType object is being used like a whiteboard.
Mathematical objects are assigned as attributes to self
willy-nilly. This leads to an unstructured namespace that is hard to understand.
As a further refactoring step, aligned with #660 and other issues:
- compartmentalizing the math so that more of the board is "erased" between operations
- when possible, the math "inputs" into functions get passed in as arguments rather than getting drawn from
self.
This will encapsulate the code in a way that makes it easier to decompose and refactor.
Another way to address this is to introduce more namespaces.
A good target for this might be the many *Dstn
variables that get created. I.e. IncomeDstn
, RiskyDstn
.
These then get sampled for shocks.
This connects to the idea that maybe these distributions should be passed in directly as parameters: #620
But other namespaces could be introduced for, for example, mathematical objects used for checkConditions
, or those used when computing solutions.
I had an idea that's a little goofy, but which may be a good short term solution.
This is to add a property, "boards", to the AgentType object.
Then a new "whiteboard" can be made for the mathematical work of a particular method by doing this:
self.boards["myContext"] = (board := {})
It can also be written without the Python 3.8 walrus operator but you get the idea.
Then in that method one can assign properties to the board:
board.MPCnow = #whatever
If you really want to inspect the mathematical work, then you can find the whiteboard where it was written and look at it. But this way it won't clutter the top-level namespace.
I would much prefer a functional programming orientation and I think we need to get there for complete mathematization of Frames. But I think this would be an improving to the current HARK model code that would help get us to the Frame architecture by structuring the data little more.
What about a decorator (@etc
) for importing a namespace into the scope of a method so that the mathematical objects are available for use?
Punting to 1.1 for time reasons.
Considering #1410 ---
When a model is initialized, it is passed a parameters dictionary.
The values of this dictionary used to be copied over onto the object as attributes (i.e. using setattr
), and many other computed mathematical values were stored in this way.
This is bad for a number of reasons. For example, it makes it very hard to find all the parameters, since they exist alongside all the other attributes of an object.
So some changes were made: The model was changed to store the parameters
dictionary as an argument. @alanlujan91 then made a dedicated parameters object.
But because there was so much code that depended on referencing arbitrary things on the model object, I didn't bother to go through and change all the cases where parameters were referenced in this way. That's why there's currently both, redundant, ways of storing parameters on a model.
To be clear, this means that on a model object, parameters live in both the self.parameters
dictionary, and the object itself. Also on the object itself are things that have been constructed mathematically.
Never before have things that have been constructed mathemaically been added to the parameters dictionary, because that dictionary was just a copy of the values the model was initialized with.
Often it's been floated that constructed objects should live in a dedicated namespace. I believe @llorracc used bilt
for something like this in his pre-alpha implementation.
I don't know why @mnwhite would like to add 'constructs' to the parameters dictionary in #1410 ; it seems like a bad idea to me. But he seems to have his reasons, which are perhaps in the short term.
In the longer term, I don't think we should have any of these values on the object. Rather, it would be better to have namespaces. Understanding which namespace we actually need would go a long way towards rationalizing our model designs.
It seems I misunderstood what the purpose of the parameters
dictionary was. I thought it was meant to be the replacement for where all of the data/information about the instance's problem was supposed to go, rather than being in the top level of the object. That is, anything that would be used by its solver or simulator, as well as anything that will be used to make something that will be used by the solver or simulator (recursively, ad infinitum), should be in the parameters
dictionary.
That seemed to be the intent of Mateo's PR in #1389 , but maybe I misunderstood that as well.
The reason I put constructed objects in the same place(s) as "raw" parameters is that there's no distinction between them, and no way to tell them apart. As an example, the default format of PermGroFac
for IndShockConsumerType
(and all of our models) is that you pass it a list (or array) of floats, one for each period; it defaults to a expecting raw input ready for use in the various solve_one_period
functions. However, in #1410 , a user could do the following:
MyType.constructors['PermGroFac'] = make_polynomial_PermGroFac
MyType.T_cycle = 40
MyType.PermGroFac_coeffs = [1.002, 0.001, -0.00003]
MyType.construct('PermGroFac')
(All of that could be done in the dictionary passed at initialization as well.) That will construct a 40 period list for PermGroFac
using polynomial coefficients (here, quadratic) on t
. Where should that list be stored? In a dictionary like bilt
or constructs
? Or in parameters
, where it would go if we had passed a hand-written PermGroFac
list... or one that was made by calling make_polynomial_PermGroFac
outside of the agent using those same inputs! That's what I mean by there being no difference between "raw" parameters and constructed ones.
Likewise, I would like for the solver and simulator to look in only one place for the information it needs, rather than checking several dictionaries (etc). Overall, I agree that we should think carefully about the organization of information and data structures, but the taxonomy of objects, and what they'll be used for, isn't always clean.
I have some ideas for how to make parameter information more accessible.
I'm not sure we're using "namespace" in the same way. To me, a "namespace" in Python is more or less synonymous with a scope: it's a level at which object references are defined. A namespace or scope is created every time you call a function; objects and references created inside that namespace have access to references in the namespace in which it was created (recursively), and that namespace vanishes from existence when the function ends. Only the returned objects continue to exist (as well as anything in the function scope that ended up referenced by something outside the scope, but that's a somewhat unusual thing to do).
I think we might be talking about two separate things entirely. I'm only addressing how to store and structure things in the current version of HARK, to make incremental progress at improving it and preparing it for big changes in the semi-near future. I think you're talking about a much different future version of HARK.