Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run integration tests in CI #68

Closed
6 of 7 tasks
KtorZ opened this issue Mar 14, 2019 · 1 comment
Closed
6 of 7 tasks

Run integration tests in CI #68

KtorZ opened this issue Mar 14, 2019 · 1 comment
Assignees

Comments

@KtorZ
Copy link
Member

KtorZ commented Mar 14, 2019

Context

After we figured out #67, #55 and #56 we should have all the keys in hand in order to run integration tests for our wallet, testing the end-2-end connectivity between a cluster of node, our wallet backend and an http client making request to the wallet server.

Decision

Implement simple example of integration tests which illustrate a complete setup running tests against a local cluster. We may get inspiration from cardano-wallet-legacy

Then, make sure those tests runs in CI as expected, and provide a correct report coverage. For the coverage report, we may look into hpc combine in order to combine tix files from the unit tests and the integration tests (allowing both jobs to run in parallel rather in sequence).

Acceptance Criteria

  • We must be able to run integration tests through stack test cardano-wallet:integration
  • Integration tests must run in the CI and, block code from being merged when failed
  • Integration tests may be run in parallel (through a separate job) with unit tests
  • Integration tests results should also be part of the coverage reports

Development Plan

  • Define a basic structure for integration tests
  • Add some integration tests connecting to the cardano-http-bridge on testnet
  • Add necessary steps in Travis and find a way to cache (part of) the testnet blockchain data

PR

Number Base
#80 master
#85 master
#86 master
#97 master

QA

  • We now have a folder and cabal target for integration tests (stack test cardano-wallet:integration). A few gotchas however:

    • Integration tests require the cardano-http-bridge to be available in scope
    • The HTTP Bridge should have been (partially -- at least the first 15 epochs) synced with testnet
  • The CI has been adjusted to now download an extra cache containing the blockchain data for testnet (for the first 15 epochs) and integration tests are now ran as part of the standard test-suite (which means that they also count for the coverage report). We may split them into two separate jobs later if you judge it necessary.

  • I've also reviewed a bit our deployment stage on Travis for it wasn't really working well. I've made a few test on a dummy repository of my own to make sure that we get the following behavior to work:

    • Upon releasing, one can create a git tag on git tag --sign -m "release v.X.Y.Z" vX.Y.Z master
    • Pushing a tag to CI will now trigger a deployment git push origin --tags, which will run all automated tests and, deploy the launcher and wallet server executables to our github releases
    • Github releases are made in a "draft" mode, such that they aren't visible by anyone outside of the team and can be edited (adding a description or CHANGELOG).
    • When ready, we can simply press the "publish release" button in Github to make the release available to the public.
@piotr-iohk
Copy link
Contributor

lgtm 👍

KtorZ added a commit that referenced this issue Mar 20, 2019
Fix typo in HttpBridgeSpec test (QA review of #68)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants