Equations in Models #1292

sbenthall · 2023-06-27T22:40:47Z

This PR is aimed at several interrelated issues, including: #479 #857 and #889

The idea is to make the definition of the mathematical relationships between variables, 'equations' logically separate from the operational code that checks conditions, solves problems, and runs simulations.

Currently, the mathematical aspects of the model are blended in with the operational code. We've been trying to get away from this for some time.

This PR has one solution and a partial demonstration of how it would work. The Model class takes an equations argument, which is a dictionary. The equations are then used as appropriate. I've demonstrated how this refactoring could work for the assignments of thorn and aNrm.

This is just one way to do it. Here are two considerations:

It wouldn't be too hard to write a formula resolver, so that resolve('thorn') would:
- look up the equation for thorn
- know from its arguments that it needs the parameters dictionary as input
- pass in that dictionary, and
- get return the value. (Or a dictionary, with thorn assigned to the value).
Ultimately, we want these equations to be composable and built in such a way that they can be compiled by JAX. Maybe a PyTenson implementation would be more efficient down the line. But it would be harder to refactor the current HARK code to do this without doing the more basic Python version first.

Tests for new functionality/models or Tests to reproduce the bug-fix in code.
Updated documentation of features that add new functionality.
Update CHANGELOG.md with major/minor changes.

…rk#479 econ-ark#889

sbenthall · 2023-06-28T14:53:24Z

Linking to Dolo documentation about the Model for inspiration here:
https://dolo.readthedocs.io/en/latest/model_api.html

sbenthall · 2023-06-28T15:42:59Z

Also for reference...

I different way to implement a representation of mathematical equations has already been committed in the frame module. In that work, the equations (called Frame in that code, but really I should rename that to Equation or remove frames entirely in favor of whatever system replaces is) are objects that include a target value, the arguments to its equation, the equation itself (as a Python function), and/or a flag to say if the target value is a control variable (needing a decision rule to be fully specified):

https://github.com/econ-ark/HARK/blob/master/HARK/ConsumptionSaving/ConsPortfolioFrameModel.py#L114-L231

The equations get evaluated in simulation in the scope of the current state/shock/parameter values:

https://github.com/econ-ark/HARK/blob/master/HARK/frame.py#L591-L670

At the time I wrote that, I had no idea how to integrate equations like these into the solver. #1283 provides a way to do that.

But also, there are pros and cons to the implementation in frame.py and the version currently in this PR is an alternative.

sbenthall · 2023-06-30T19:07:44Z

The failing tests in the ConsAggShockModels are due to the fact that the AggShockModel constructor does not call the constructor of its superclass (IndShockConsumerType), but rather the AgentType base class constructor.

https://github.com/econ-ark/HARK/blob/master/HARK/ConsumptionSaving/ConsAggShockModel.py#L135-L156

This is not how other model inheritance in HARK works, where the superclass is called.

Is this intentional? Or just an oversight in how the ConsAggShockModel works? if the latter, I can easily fix it in this PR.

mnwhite · 2023-07-05T15:17:12Z

That looks like an oversight, but there might be some initialization routine for IndShockConsumerType that's not supposed to be run for an AggShockConsumerType. Let's try changing the inheritance and see if it runs properly.

…

On Wed, Jul 5, 2023 at 10:58 AM Sebastian Benthall ***@***.***> wrote: @sbenthall <https://github.com/sbenthall> requested your review on: #1292 <#1292> [WIP] Equations in Models. — Reply to this email directly, view it on GitHub <#1292 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADKRAFOY5HTVAAWDCWQYCM3XOV6JNANCNFSM6AAAAAAZWHG3XU> . You are receiving this because your review was requested.Message ID: ***@***.***>

sbenthall · 2023-07-05T16:24:24Z

Let's try changing the inheritance and see if it runs properly.

Ok, I changed it locally and it got tests passing. This change has been committed to this PR.

codecov · 2023-07-12T14:45:25Z

Codecov Report

Patch coverage: 96.72% and project coverage change: +0.06 🎉

Comparison is base (2209109) 72.55% compared to head (97a1740) 72.61%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1292      +/-   ##
==========================================
+ Coverage   72.55%   72.61%   +0.06%     
==========================================
  Files          78       80       +2     
  Lines       13009    13038      +29     
==========================================
+ Hits         9439     9468      +29     
  Misses       3570     3570

Impacted Files	Coverage Δ
HARK/model.py	`94.11% <94.11%> (ø)`
HARK/ConsumptionSaving/ConsAggShockModel.py	`89.68% <100.00%> (ø)`
HARK/ConsumptionSaving/ConsIndShockModel.py	`87.10% <100.00%> (+0.05%)`	⬆️
HARK/core.py	`88.23% <100.00%> (-0.18%)`	⬇️
HARK/tests/test_model.py	`100.00% <100.00%> (ø)`

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

… KinkedR inheritance

sbenthall · 2023-07-12T17:22:58Z

Tests are now passing. KinkedR had a similar issue with inheritance.

The substance of this PR is now in and ready for review. Requesting review from @mnwhite

I know I need to do tests, docs, and changelog, and will start on that next.

llorracc · 2023-07-20T04:44:14Z

Not sure why this just popped up in my inbox, but it prompted me to look again.

I think thinking about all this is premature. It will really need to be integrated with whatever structure gets created from Pablo's architecture. At most, at this point we could perhaps fruitfully build on what is already in dolo or nolib.

sbenthall · 2023-07-20T13:34:25Z

It's unfortunate that you've been missing the regular HARK meetings @llorracc I think we're quite resolved that this is a good idea.

Essentially, no matter what Pablo's architecture is, the HARK models will need to be moved in this sort of direction in order to integrate with that. This is a necessary incremental improvement to HARK no matter what.

sbenthall · 2024-06-01T20:42:47Z

I'd like to revisit this PR, which is now almost a year old, and perhaps out of date with respect to recent HARK changes. But it was trying to accomplish something that we are still trying to move towards.

The main point of this PR was to try attaching dynamics to the existing ConsumptionSaving models, so that they could be rewritten with a single source of truth about their model equations.

These dynamics, in this PR, looked like this:

IndShockConsumerType_dynamics = {
    **PerfForesightConsumerType_dynamics,
    'G' : lambda gamma, psi : gamma * psi,
    'Rnrm' : lambda R, G : R / G,
    'bNrm' : lambda Rnrm, aNrm : Rnrm * aNrm,
    'mNrm' : lambda bNrm, theta : bNrm + theta,
    'cNrm' : Control(['mNrm']),
    'aNrm' : lambda mNrm, cNrm : mNrm - cNrm
}

This is now essentially the same thing as what's included in the 'block' based consumption model, which was recently merged in:

https://github.com/econ-ark/HARK/blob/master/HARK/models/consumer.py#L26-L44

Now that there's a compact format for model equations already in HARK, it would be a good idea to implement a solution to #1346 and put the equations onto the old models.

llorracc · 2024-06-02T07:04:02Z

Not getting what you mean by adding dynamics. In economists’ use of the word these models already have dynamics. They are dynamic, stochastic, optimization models. I’m keen on making some basic structural decisions first. Sent from Gmail Mobile

…

On Sat, Jun 1, 2024 at 4:43 PM Sebastian Benthall ***@***.***> wrote: I'd like to revisit this PR, which is now almost a year old, and perhaps out of date with respect to recent HARK changes. But it was trying to accomplish something that we are still trying to move towards. The main point of this PR was to try attaching dynamics to the existing ConsumptionSaving models, so that they could be rewritten with a single source of truth about their model equations. These dynamics, in this PR, looked like this: IndShockConsumerType_dynamics = { **PerfForesightConsumerType_dynamics, 'G' : lambda gamma, psi : gamma * psi, 'Rnrm' : lambda R, G : R / G, 'bNrm' : lambda Rnrm, aNrm : Rnrm * aNrm, 'mNrm' : lambda bNrm, theta : bNrm + theta, 'cNrm' : Control(['mNrm']), 'aNrm' : lambda mNrm, cNrm : mNrm - cNrm } This is now essentially the same thing as what's included in the 'block' based consumption model, which was recently merged in: https://github.com/econ-ark/HARK/blob/master/HARK/models/consumer.py#L26-L44 Now that there's a compact format for model equations already in HARK, it would be a good idea to implement a solution to #1346 <#1346> and put the equations onto the old models. — Reply to this email directly, view it on GitHub <#1292 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKCK74KLV3MWV7UU3MWW7LZFIW53AVCNFSM6AAAAAAZWHG3XWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBTGU4DEMBXHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

sbenthall · 2024-06-02T11:53:28Z

Just to clarify what I meant: I meant having a representation of the equations governing how the variables change, which other parts of system can use. On Sun, Jun 2, 2024, 3:04 AM Christopher Llorracc Carroll < ***@***.***> wrote:

…

Not getting what you mean by adding dynamics. In economists’ use of the word these models already have dynamics. They are dynamic, stochastic, optimization models. I’m keen on making some basic structural decisions first. Sent from Gmail Mobile On Sat, Jun 1, 2024 at 4:43 PM Sebastian Benthall ***@***.***> wrote: > I'd like to revisit this PR, which is now almost a year old, and perhaps > out of date with respect to recent HARK changes. But it was trying to > accomplish something that we are still trying to move towards. > > The main point of this PR was to try attaching dynamics to the existing > ConsumptionSaving models, so that they could be rewritten with a single > source of truth about their model equations. > > These dynamics, in this PR, looked like this: > > IndShockConsumerType_dynamics = { > **PerfForesightConsumerType_dynamics, > 'G' : lambda gamma, psi : gamma * psi, > 'Rnrm' : lambda R, G : R / G, > 'bNrm' : lambda Rnrm, aNrm : Rnrm * aNrm, > 'mNrm' : lambda bNrm, theta : bNrm + theta, > 'cNrm' : Control(['mNrm']), > 'aNrm' : lambda mNrm, cNrm : mNrm - cNrm > } > > This is now essentially the same thing as what's included in the 'block' > based consumption model, which was recently merged in: > > > https://github.com/econ-ark/HARK/blob/master/HARK/models/consumer.py#L26-L44 > > Now that there's a compact format for model equations already in HARK, it > would be a good idea to implement a solution to #1346 > <#1346> and put the equations onto > the old models. > > — > Reply to this email directly, view it on GitHub > <#1292 (comment)>, or > unsubscribe > < https://github.com/notifications/unsubscribe-auth/AAKCK74KLV3MWV7UU3MWW7LZFIW53AVCNFSM6AAAAAAZWHG3XWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBTGU4DEMBXHA> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub <#1292 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAQZEHMEYOYVSPVHMJ3CO3ZFK7XRAVCNFSM6AAAAAAZWHG3XWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBTG4ZDMMJXGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

sbenthall · 2024-06-02T14:13:42Z

A reason to lay out the equations for more models is to validate that any basic structural designs generalize beyond the portfolio consumption problem. As is, we risk spending months designing a general solution that works for one specific problem.

…

On Sun, Jun 2, 2024, 7:53 AM Sebastian Benthall ***@***.***> wrote: Just to clarify what I meant: I meant having a representation of the equations governing how the variables change, which other parts of system can use. On Sun, Jun 2, 2024, 3:04 AM Christopher Llorracc Carroll < ***@***.***> wrote: > Not getting what you mean by adding dynamics. In economists’ use of the > word these models already have dynamics. They are dynamic, stochastic, > optimization models. > > I’m keen on making some basic structural decisions first. > > Sent from Gmail Mobile > > > On Sat, Jun 1, 2024 at 4:43 PM Sebastian Benthall ***@***.***> > wrote: > > > I'd like to revisit this PR, which is now almost a year old, and > perhaps > > out of date with respect to recent HARK changes. But it was trying to > > accomplish something that we are still trying to move towards. > > > > The main point of this PR was to try attaching dynamics to the existing > > ConsumptionSaving models, so that they could be rewritten with a single > > source of truth about their model equations. > > > > These dynamics, in this PR, looked like this: > > > > IndShockConsumerType_dynamics = { > > **PerfForesightConsumerType_dynamics, > > 'G' : lambda gamma, psi : gamma * psi, > > 'Rnrm' : lambda R, G : R / G, > > 'bNrm' : lambda Rnrm, aNrm : Rnrm * aNrm, > > 'mNrm' : lambda bNrm, theta : bNrm + theta, > > 'cNrm' : Control(['mNrm']), > > 'aNrm' : lambda mNrm, cNrm : mNrm - cNrm > > } > > > > This is now essentially the same thing as what's included in the > 'block' > > based consumption model, which was recently merged in: > > > > > > > https://github.com/econ-ark/HARK/blob/master/HARK/models/consumer.py#L26-L44 > > > > Now that there's a compact format for model equations already in HARK, > it > > would be a good idea to implement a solution to #1346 > > <#1346> and put the equations > onto > > the old models. > > > > — > > Reply to this email directly, view it on GitHub > > <#1292 (comment)>, > or > > unsubscribe > > < > https://github.com/notifications/unsubscribe-auth/AAKCK74KLV3MWV7UU3MWW7LZFIW53AVCNFSM6AAAAAAZWHG3XWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBTGU4DEMBXHA> > > > . > > You are receiving this because you were mentioned.Message ID: > > ***@***.***> > > > > — > Reply to this email directly, view it on GitHub > <#1292 (comment)>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAAQZEHMEYOYVSPVHMJ3CO3ZFK7XRAVCNFSM6AAAAAAZWHG3XWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBTG4ZDMMJXGY> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

llorracc · 2024-06-02T23:58:01Z

I don't think it will be months. And once we have a satisfactory working example solution of two periods of one particular problem, I am in complete agreement that we should take the lessons learned and try adapting to another model. I'm thinking of the EndogenousRetirement problem, because it is so different from ConsIndShk. On Sun, Jun 2, 2024 at 10:14 AM Sebastian Benthall ***@***.***> wrote:

…

A reason to lay out the equations for more models is to validate that any basic structural designs generalize beyond the portfolio consumption problem. As is, we risk spending months designing a general solution that works for one specific problem. On Sun, Jun 2, 2024, 7:53 AM Sebastian Benthall ***@***.***> wrote: > Just to clarify what I meant: I meant having a representation of the > equations governing how the variables change, which other parts of system > can use. > > On Sun, Jun 2, 2024, 3:04 AM Christopher Llorracc Carroll < > ***@***.***> wrote: > >> Not getting what you mean by adding dynamics. In economists’ use of the >> word these models already have dynamics. They are dynamic, stochastic, >> optimization models. >> >> I’m keen on making some basic structural decisions first. >> >> Sent from Gmail Mobile >> >> >> On Sat, Jun 1, 2024 at 4:43 PM Sebastian Benthall ***@***.***> >> wrote: >> >> > I'd like to revisit this PR, which is now almost a year old, and >> perhaps >> > out of date with respect to recent HARK changes. But it was trying to >> > accomplish something that we are still trying to move towards. >> > >> > The main point of this PR was to try attaching dynamics to the existing >> > ConsumptionSaving models, so that they could be rewritten with a single >> > source of truth about their model equations. >> > >> > These dynamics, in this PR, looked like this: >> > >> > IndShockConsumerType_dynamics = { >> > **PerfForesightConsumerType_dynamics, >> > 'G' : lambda gamma, psi : gamma * psi, >> > 'Rnrm' : lambda R, G : R / G, >> > 'bNrm' : lambda Rnrm, aNrm : Rnrm * aNrm, >> > 'mNrm' : lambda bNrm, theta : bNrm + theta, >> > 'cNrm' : Control(['mNrm']), >> > 'aNrm' : lambda mNrm, cNrm : mNrm - cNrm >> > } >> > >> > This is now essentially the same thing as what's included in the >> 'block' >> > based consumption model, which was recently merged in: >> > >> > >> > >> https://github.com/econ-ark/HARK/blob/master/HARK/models/consumer.py#L26-L44 >> > >> > Now that there's a compact format for model equations already in HARK, >> it >> > would be a good idea to implement a solution to #1346 >> > <#1346> and put the equations >> onto >> > the old models. >> > >> > — >> > Reply to this email directly, view it on GitHub >> > <#1292 (comment)>, >> or >> > unsubscribe >> > < >> https://github.com/notifications/unsubscribe-auth/AAKCK74KLV3MWV7UU3MWW7LZFIW53AVCNFSM6AAAAAAZWHG3XWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBTGU4DEMBXHA> >> >> > . >> > You are receiving this because you were mentioned.Message ID: >> > ***@***.***> >> > >> >> — >> Reply to this email directly, view it on GitHub >> <#1292 (comment)>, or >> unsubscribe >> < https://github.com/notifications/unsubscribe-auth/AAAQZEHMEYOYVSPVHMJ3CO3ZFK7XRAVCNFSM6AAAAAAZWHG3XWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBTG4ZDMMJXGY> >> . >> You are receiving this because you were mentioned.Message ID: >> ***@***.***> >> > — Reply to this email directly, view it on GitHub <#1292 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKCK77ENUQ2BBA5UK7NRE3ZFMSCXAVCNFSM6AAAAAAZWHG3XWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBTHA3TAOBTG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- - Chris Carroll

sbenthall · 2024-11-15T19:38:09Z

PR is out of date, and not needed for HARK 1.0

trying a way to include explicit equations in HARK models. See econ-a…

7b165cf

…rk#479 econ-ark#889

sbenthall mentioned this pull request Jun 28, 2023

Update check_conditions() #1294

Merged

3 tasks

sbenthall mentioned this pull request Jun 28, 2023

replace AgentType.sim_one_period with a generic Monte Carlo simulator that consumes equations #1295

Open

more IndShockConsumerType 'dynamics'. Very lightweight functions

d09eb30

sbenthall requested a review from mnwhite July 5, 2023 14:58

ConsAggShock to call super() init instead of AgentType

4e1f4f0

new parameters input more consistent with past tests

e6db4ef

sbenthall mentioned this pull request Jul 12, 2023

Parameters #1240

Merged

3 tasks

sbenthall added 2 commits July 12, 2023 11:27

split PF and IndShock dynamics; load PF dynamics in PF AgentType; fix…

1a6a0dc

… KinkedR inheritance

moving Model to model.py

b9d6fb6

sbenthall changed the title ~~[WIP] Equations in Models~~ Equations in Models Jul 12, 2023

sbenthall added 6 commits July 19, 2023 10:31

Merge branch 'master' into eqmodel

be3b189

changelog update for equations in models, PR econ-ark#1292

4b0c3ed

documentatin and automated tests for equations in models

32c7cf3

adding docs to RST autodocs

d7296f7

import abstract types from typing to pass Python 3.9 tests

4af0ec2

reference/tools/model.rst for docs

97a1740

sbenthall mentioned this pull request Jul 20, 2023

Generic monte carlo simulation #1296

Merged

3 tasks

This was referenced Aug 28, 2023

library of ConsumptionSaving model definitions #1346

Open

perfect foresight model data #1347

Merged

sbenthall mentioned this pull request Jun 3, 2024

documentation for 'new' models; better docs for 'old' models #1439

Open

sbenthall closed this Nov 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Equations in Models #1292

Equations in Models #1292

sbenthall commented Jun 27, 2023 •

edited

Loading

sbenthall commented Jun 28, 2023

sbenthall commented Jun 28, 2023

sbenthall commented Jun 30, 2023

mnwhite commented Jul 5, 2023 via email

sbenthall commented Jul 5, 2023

codecov bot commented Jul 12, 2023 •

edited

Loading

sbenthall commented Jul 12, 2023

llorracc commented Jul 20, 2023

sbenthall commented Jul 20, 2023

sbenthall commented Jun 1, 2024

llorracc commented Jun 2, 2024 via email

sbenthall commented Jun 2, 2024 via email

sbenthall commented Jun 2, 2024 via email

llorracc commented Jun 2, 2024 via email

sbenthall commented Nov 15, 2024

Equations in Models #1292

Equations in Models #1292

Conversation

sbenthall commented Jun 27, 2023 • edited Loading

sbenthall commented Jun 28, 2023

sbenthall commented Jun 28, 2023

sbenthall commented Jun 30, 2023

mnwhite commented Jul 5, 2023 via email

sbenthall commented Jul 5, 2023

codecov bot commented Jul 12, 2023 • edited Loading

Codecov Report

sbenthall commented Jul 12, 2023

llorracc commented Jul 20, 2023

sbenthall commented Jul 20, 2023

sbenthall commented Jun 1, 2024

llorracc commented Jun 2, 2024 via email

sbenthall commented Jun 2, 2024 via email

sbenthall commented Jun 2, 2024 via email

llorracc commented Jun 2, 2024 via email

sbenthall commented Nov 15, 2024

sbenthall commented Jun 27, 2023 •

edited

Loading

codecov bot commented Jul 12, 2023 •

edited

Loading