-
Notifications
You must be signed in to change notification settings - Fork 16
ci: run SDK tests against latest version of API #59
Conversation
btw the tests here are running against the server in CI, the issue is that some tests change state (i.e. deleting sources/submissions) - that's the remaining work here before this is ready for review |
131905c
to
6d3a6cb
Compare
6b1d764
to
ce95c86
Compare
9f9bda0
to
568c72b
Compare
OK this is working and ready for review. Once this is approved I'll update the required CI jobs for merge. Note that the diff looks big here, but this is mostly VCR cassettes getting updated. What I think we want to do next - ideally in a followup since what's here so far is a concrete step forward testing-wise - is:
|
- run: | ||
name: Wait for server to be up and for test sources to load | ||
command: | | ||
dockerize -wait tcp://127.0.0.1:8080 -timeout 15m |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This might not be enough time. Building the dev container is slooooooooow.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
haha, it feels like it takes a long time but it "only" takes around 4-5 minutes
.circleci/config.yml
Outdated
only: | ||
- master | ||
jobs: | ||
- test-against-latest-api |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: missing newline at EOF
The follow up should also include an env var / CLI opt to ignore the cassettes that way we don't have to do an |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Other than the one question, rest looks great. Thank you 👍 Approved.
], | ||
"markers": "python_version >= '3.4'", | ||
"version": "==1.3.0" | ||
"version": "==1.2.0" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are we going back to an older release?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We specify in our Pipfile Python 3.5 (and in my local dev env I use pyenv to use only that version). However in yarl 1.2.1 they pinned the minimum Python version to be 3.5.3. So this can cause confusing local failures for developers if they're using Python 3.5.0, so I set it back to 1.2.0 for compatibility with all Python 3.5.* since there were no security updates in the diff from yarl 1.2.0 to yarl 1.3.0.
also adding it as a nightly job in case changes are made on the API side
1.3.0 does not install on python 3.5 aio-libs/yarl@v1.2.0...v1.3.0
568c72b
to
3157be8
Compare
the idea in this PR is to add a circle CI job that runs nightly and (I can be convinced the latter is a bad idea) on each PR. the circle CI job will run the API tests against the server container
that way if there's a discrepancy between API and SDK, we can detect it (ref: #55)
(btw I did this in a fork so that my frequent force pushing doesn't create a billion notifications)