Generic single-agent MDP classes

[sbenthall]

As a 2.0 direction (see planning document), HARK can have more generic code for its core logic, separating the economic modeling metaphors from the software for modeling and solving classes of problems.

My understanding is that the following kinds of representations would be useful for solving the range of problems currently in HARK:

• (A) Mixed Continuous/Discrete MDP
• (B) Discrete MDP
• (C) Bellman form of Discrete MDP

In addition to these, there are cases for multi-agent problems with market equilibrium assumptions; I'll leave these out of scope of this issue for now.

One way to solve this problem is to allow the following functionality:

• Specifying a problem of type (A)
• Converting it via discretization into a problem of type (B)
• Use generic DP and RL algorithms to solve problems of type (B)
• Assist in the conversion of type (B) problem into type (C) problem, opening up a different range of algorithms

This is different from what is being done in HARK and Dolo, which is interpolation between the discrete values.

[sbenthall]

Interpolation is a key distinguishing figure of what's being done in HARK.