You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have a suggestion on how to further improve the integration test situation:
Currently, we always run against the lates (go) tendermint docker image. On one hand this is really cool because we get early warnings for things that will break compatibility with the next release. On the other hand, we can't guarantee compatibility against a current release (e.g. we only want compatibility with 0.33.X or even only with 0.33.4 because that's what the hub is using or goz or whatever reasons there might be to want compatibility to a particular version).
I think tests would be even more useful if:
they were run against a fixed and chosen tendermint version and were made mandatory (required on gh) for this version
additionally integration tests for the latest image run regularly too (but much less frequently; e.g. nightly/weekly)
Regarding 1): we currently do not merge PRs unless integration tests pass anyways. The tools could better reflect this already established process and only allow merging if intgration tests pass too. Also, sometimes integration tests pass on the branch but fail on master (#249#233 and many more). This happens because there was a new docker image released between last check on branch and merge to master. This would be mitigated if we made sure, we commit to compatibility of one tendermint version.
And 2) would help us to be learn about upcoming changes on a regular basis.
I have a suggestion on how to further improve the integration test situation:
Currently, we always run against the lates (go) tendermint docker image. On one hand this is really cool because we get early warnings for things that will break compatibility with the next release. On the other hand, we can't guarantee compatibility against a current release (e.g. we only want compatibility with 0.33.X or even only with 0.33.4 because that's what the hub is using or goz or whatever reasons there might be to want compatibility to a particular version).
I think tests would be even more useful if:
latest
image run regularly too (but much less frequently; e.g. nightly/weekly)Regarding 1): we currently do not merge PRs unless integration tests pass anyways. The tools could better reflect this already established process and only allow merging if intgration tests pass too. Also, sometimes integration tests pass on the branch but fail on master (#249 #233 and many more). This happens because there was a new docker image released between last check on branch and merge to master. This would be mitigated if we made sure, we commit to compatibility of one tendermint version.
And 2) would help us to be learn about upcoming changes on a regular basis.
Originally posted by @liamsi in #120 (comment)
The text was updated successfully, but these errors were encountered: