Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

As a library, this should not pin to a specific version of botocore or boto3 #971

Closed
tleonhardt opened this issue Sep 26, 2022 · 7 comments

Comments

@tleonhardt
Copy link

tleonhardt commented Sep 26, 2022

As a library, aiobotocre should not require a specific version of any of these in the setup.py:

  • botocore
  • boto3
  • awscli

If this were an application, it would be different. But in order to use this library with a wide variety of other libraries, you want to use >= not a combination of >= and < which is effectively an ==.

I have had to regretfully remove this and things that depend on it from a JupyterHub/JupyterLab system I maintain because it wouldn't allow me to upgrade to the latest version of botocore and boto3 and hence prevented other tools from working properly.

@mathewlee11
Copy link

Yeah especially with the new pip resolver that fails on conflicts the current setup makes it very difficult to include aiobotocore in anything

@thehesiod
Copy link
Collaborator

this is due to how this module overrides many classes in botocore, please see history: https://github.com/aio-libs/aiobotocore/issues?q=is%3Aissue+is%3Aclosed+pin. Otherwise we'd need to vendor botocore, which means it wouldn't match the botocore installed next to it. If you'd like a newer botocore please open a ticket or if you have any other suggestions please feel free to re-open.

@tleonhardt
Copy link
Author

We'll just tell our customers to use awswrangler instead so we can stick with sane dependency management.

@alvaro-ponce
Copy link

alvaro-ponce commented Sep 30, 2022

Same problem here, it is impossible to install s3fs and boto3 at the same time because of the dependency generated by this strict policy of installing a specific version of botocore.
At this point I am forced to do not use s3fs nor aiobotocore and use awswrangler or create my own s3filesystem handler.

@bnavigator
Copy link

aiobotocore is about to drop out of openSUSE Tumbleweed because it can't keep up with botocore and boto3

@thehesiod
Copy link
Collaborator

so ya if it needs to match botocore/boto3 then we'll have to vendor botocore ;(

@thehesiod
Copy link
Collaborator

I'm not even sure how that would work, because when in botocore it does an import it will need to import from the vendored copy as well, which I think means we have to re-write all the import statements

bmwiedemann pushed a commit to bmwiedemann/openSUSE that referenced this issue Oct 26, 2022
https://build.opensuse.org/request/show/1010922
by user bnavigator + dimstar_suse
pins to aiobotocore which itself pins to a specific botocore version. Impossible to maintain in a rolling distro

gh#fsspec/s3fs#615, gh#aio-libs/aiobotocore#971

see also sr#1010918, sr#1010920
bmwiedemann pushed a commit to bmwiedemann/openSUSE that referenced this issue Oct 26, 2022
https://build.opensuse.org/request/show/1010923
by user bnavigator + dimstar_suse
The pinning of boto3 and botocore versions makes this impossible to maintain in a rolling distro.

gh#fsspec/s3fs#615, gh#aio-libs/aiobotocore#971

see also sr#1010918, sr#1010920, sr#1010922
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants