-
Notifications
You must be signed in to change notification settings - Fork 406
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: canonicalize config keys #2206
Conversation
ee0f328
to
472525c
Compare
Looks like the failure is just in some tests which are not expecting some environment variables to be set. Our option I think is to either have these new tests reset their environment variables, or update the other tests. |
Head branch was pushed to by a user without write access
I introduced a small utility to all of the tests to restore the environment to how it was when they captured it. Let me know if you think this approach is reasonable? |
Head branch was pushed to by a user without write access
☝️ I briefly flirted with changing the tests to be a lot more careful about which environment variables they touched. This was because the integration tests workflow itself actually sets some extra environment vars, which was messing with the test. However on reflection on doing so, by being canoncalizing the keys under test, I managed to partly defeat the purpose of the test... In the end I went for the simpler solution from the other tests of just unsetting those variables. |
f912d25
to
453a706
Compare
Description
When merging supplied storage options with environment variables, we need to canonicalize the items in the storage options first. This is because for some keys there is multiple valid definitions (
AWS_ENDPOINT_URL
,AWS_ENDPOINT
) and also casing (AWS_ACCESS_KEY_ID
,aws_access_key_id
). It seems like the 'more canonical' lowercase versions win, which can cause surprise if you have one set of access keys in your environment and one in your storage options. This is exacerbated as theS3StorageOptions
construction also sets envrionment variables, and it always does so into the canonical lowercase versions. I found this when opening two buckets with different credentials, and the credentials for the first were being used to open the second.