Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All tests failing in github CI #41

Closed
bwohlberg opened this issue Oct 13, 2021 · 3 comments · Fixed by #43
Closed

All tests failing in github CI #41

bwohlberg opened this issue Oct 13, 2021 · 3 comments · Fixed by #43
Assignees
Labels
bug Something isn't working priority: high High priority tests Pertaining to SCICO tests
Milestone

Comments

@bwohlberg
Copy link
Collaborator

All tests are currently failing with an error ValueError: 'shape' is not in list. This appears to be related to the line

shape_ind = list(inspect.signature(fun).parameters.keys()).index("shape")

in scico.random. Since most tests are passing when pytest is run locally, this is presumably due to a recent update of the version of some dependency package in the github CI run environment.

@bwohlberg bwohlberg added bug Something isn't working tests Pertaining to SCICO tests priority: high High priority labels Oct 13, 2021
@bwohlberg bwohlberg added this to the Release 0.1.0 milestone Oct 13, 2021
@Michael-T-McCann
Copy link
Contributor

It looks to me like the CI just grabs the latest jax: jax>=0.2.19. This is bad, right? Given that the jax API is changing, I would suggest we pin SCICO to a specific version of jax.

@Michael-T-McCann Michael-T-McCann self-assigned this Oct 13, 2021
@Michael-T-McCann
Copy link
Contributor

I see how to fix the immediate problem in random, will handle it now.

@lukepfister
Copy link
Contributor

lukepfister commented Oct 13, 2021

Yes, definitely pin it.

It wouldn't be a bad idea to set up a test matrix and run the tests on jax==whatever we pin to and jax upstream.

See here: https://docs.github.com/en/actions/learn-github-actions/managing-complex-workflows#using-a-build-matrix

I've seen this error before. My best guess: a new helper function got added to jax.random; one that doesn't take shape as an argument. We are running the scico.random wrapper over all functions that aren't explicitly prohibited. This happened with threefry_2x32 and I fixed it in this commit: https://github.com/scici/scico/commit/3cb896d547f28862e3cc82d253ab15eaf4332cde. I'd suggest looking at the functions here and seeing if any of them are the problem:

https://github.com/google/jax/blob/573943125bc575feb16fc8eeaceb624b07c19941/jax/random.py#L109-L115

To keep this from happening in the future:
Maybe we should change to an allow_list model and only wrap the functions that we intend to?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working priority: high High priority tests Pertaining to SCICO tests
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants