release procedure for DemARK

[sbenthall]

I wanted to make issue in DemARK for the version control and dependency issues of this repository.

This can extend the conversation here:
econ-ark/HARK#527 (comment)

The question is: what happens when a change to HARK master would break a DemARK?

[sbenthall]

Normally, when developing software, if you have software package A that depends on software package B, you:

• Do regular releases of B that are stable
• Have A depend on the latest release of B

Here, A is DemARK and B is HARK.

Travis is currently pulling in master from HARK not the latest stable release.

Naturally, that is going to lead to DemARK being unstable. I think this should change.

[llorracc]

I'm not sure this is the right way to think about it. I'd say that we are developing one overall suite of software -- both the tools and the examples/demonstrations of those tools. A "suite" of software can reasonably have a single release number.

The reason to think about it this way is that we want to know immediately if some change to HARK breaks a DemARK. If we wait for weeks and only test the DemARKs with Travis when we think we are ready for a new release, it will be much harder to run down the exact point, among many commits, at which there was a change that broke the DemARK.

[sbenthall]

I'm not entirely familiar with the DemARK use case.
My understanding is that you use them in your classes.

It does not appear that there are any releases of the DemARK repository.

I think it would be better if there were cut stable releases of DemARK, and that your courses used the latest stable release.

[sbenthall]

We're talking about this at the weekly meeting. I think the consensus is:

• HARK travis should run the DemARK test script if at all possible ( @MridulS is this feasible? )
• DemARK will have stable releases
  • They will be timed with the HARK releases
  • A DemARK release will be pegged to the HARK release.
• DemARK developers working on master will be working with HARK master in order to catch bugs as they come up, before the release.

I'll pipe in again on this issue once I've been able to file a PR fixing the DemARKs, which is a high priority item for me.

[sbenthall]

I want to revise this based on two things that have come up in the past 24 hours:

• If DemARKs sometimes get build-breaking commits, then HARK travis should not run all the DemARKs.
  • In my view it's bad if it becomes impossible to do a pull request into DemARKs because of some breaking change in HARK, or vice versa.
• Some DemARKs depend on QuARK? This is a further dependency that probably should be taken into account in the release procedure.

[llorracc]

Let me add one point to this discussion: If a change to HARK breaks a DemARK, one option for handling that, at least in the short run, is to move that DemARK from the "master" branch to a specially created branch named something like "commit####-breaks-this" so that we do not have to make sure that every change we make works with every DemARK. We can exercise judgment about the circs in which this is the right thing to do -- clearly it is NOT if the DemARK breaks as a result of an accidental bug introduced by the HARK commit -- then we should fix HARK. But if it's just an incompatibility, and not easy to see how to resolve quickly, we could adopt the workflow described above.

[sbenthall]

@llorracc This would be highly irregular. I've never seen anything like that.

I think one thing that makes this tricky is that it's not clear who is responsible for maintaining DemARKs.

Normally, somebody who contributes code to a software repository takes some responsibility for maintaining it. But many of the DemARKs seem to have been contributed by people who are no longer active in the community.

[llorracc]

Maybe I should investigate GitHub Classroom again. I looked into it maybe 18 months ago and it seemed both cumbersome and not really capable of doing some of the things I wanted to do. But maybe some of those problems have been overcome by now, or people have developed workarounds for them.

https://education.github.community/t/classroom-vs-custom-solution-for-jupyter-notebook-assignments/36620/2

[MridulS]

I like to think of DemARK as part of the HARK library (or the econ-ark project). They are just more detailed examples as compared to the documentation examples we have in HARK. Irrespective of who adds a DemARK, I think it’s the responsibility of HARK maintainers (us) to make sure DemARKs are always synced up with HARK. Think of DemARK as more elaborate documentation examples. If there is a major breaking change in HARK that should reflect in DemARK too. One way of clearing this up is marking releases of DemARK when we release HARK, so we have a tag available in DemARKs to jump around versions, exactly like documentation.

…

On 20-Mar-2020, at 9:58 PM, Sebastian Benthall ***@***.***> wrote: @llorracc <https://github.com/llorracc> This would be highly irregular. I've never seen anything like that. I think one thing that makes this tricky is that it's not clear who is responsible for maintaining DemARKs. Normally, somebody who contributes code to a software repository takes some responsibility for maintaining it. But many of the DemARKs seem to have been contributed by people who are no longer active in the community. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#108 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABI5RFBUEMMEJ3R7NGU6POTRIOKTBANCNFSM4LO2GDYA>.

[llorracc]

One way of clearing this up is marking releases of DemARK when we release

HARK, so we have a tag available in DemARKs to jump around versions, exactly like documentation. Yes, Seb and I had reached the same conclusion: The DemARK repo needs to have "release" numbers that are sync'd with HARK releases. On Fri, Mar 20, 2020 at 1:14 PM Mridul Seth <[email protected]> wrote:

…

I like to think of DemARK as part of the HARK library (or the econ-ark project). They are just more detailed examples as compared to the documentation examples we have in HARK. Irrespective of who adds a DemARK, I think it’s the responsibility of HARK maintainers (us) to make sure DemARKs are always synced up with HARK. Think of DemARK as more elaborate documentation examples. If there is a major breaking change in HARK that should reflect in DemARK too. One way of clearing this up is marking releases of DemARK when we release HARK, so we have a tag available in DemARKs to jump around versions, exactly like documentation. > On 20-Mar-2020, at 9:58 PM, Sebastian Benthall ***@***.***> wrote: > > > @llorracc <https://github.com/llorracc> This would be highly irregular. I've never seen anything like that. > > I think one thing that makes this tricky is that it's not clear who is responsible for maintaining DemARKs. > > Normally, somebody who contributes code to a software repository takes some responsibility for maintaining it. But many of the DemARKs seem to have been contributed by people who are no longer active in the community. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub < #108 (comment)>, or unsubscribe < https://github.com/notifications/unsubscribe-auth/ABI5RFBUEMMEJ3R7NGU6POTRIOKTBANCNFSM4LO2GDYA >. > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#108 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKCK76IDJJADNLVPO3KRPLRIOQARANCNFSM4LO2GDYA> .

-- - Chris Carroll

[sbenthall]

I like to think of DemARK as part of the HARK library (or the econ-ark project). They are just more detailed examples as compared to the documentation examples we have in HARK.

If that's true, then I think it would make more sense to put them into the HARK repository.
That would simplify testing, versioning, etc.

However, my (recent) understanding is that some DemARKs are actually sourced from QuARKS? I don't really understand what QuARKs are. I just learned about them today. But I gather that they are tied to specific classroom-related use cases.

I think the code that is for classroom-related use cases should be decoupled from the library code.

[llorracc]

"QuARKs" are "Questions using the ARK" -- basically, student exercises for teaching the how to use the toolkit to answer economic questions. These are the bread and butter of my PhD teaching material.

Basically, they are just augmented versions of DemARKs, where the user is asked to solve specific questions or make specific computations related to substantive points being made in the DemARK. Like, after some point has been demonstrated under default parameter values, the student is asked to do the same kind of exercise but for a variety of other parameter values and explain why the different parameter choices yield different quantitative answers.

[llorracc]

Resolved: DemARKs (and examples) should all work before we release any new version of HARK.