Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide py38-py311 and py312 compatible way for subclassing #172

Closed
3 of 5 tasks
ap-- opened this issue Jan 28, 2024 · 0 comments · Fixed by #173
Closed
3 of 5 tasks

Provide py38-py311 and py312 compatible way for subclassing #172

ap-- opened this issue Jan 28, 2024 · 0 comments · Fixed by #173
Assignees
Labels
bug 🐛 Something isn't working compatibility 🤝 Compatibility with stdlib pathlib
Milestone

Comments

@ap--
Copy link
Collaborator

ap-- commented Jan 28, 2024

When #152 is merged, user subclasses that override upath.core._FSSpecAccessor will very likely not work correctly with Python3.12, because the 3.12 implementation doesn't use the accessor.

There should be a recommended way for users to subclass UPath for Python3.12 AND Python3.8-Python3.11 without having to maintain two code paths.

Searching GitHub, I could collect an example set of user subclasses that can be split into 3 4 intended use cases:

(1) customize fsspec filesystem creation

here the user wants to change the way the fsspec filesystem instance is created:

upath.UPath should offer a classmethod _make_fs() (?) to allow fsspec filesystem instance customization.

It should be possible too to support old code on python3.12 by detecting when the _default_accessor attribute is present and use that to make a compatible _make_fs() (?) fallback method.

(2) customize path formatting

here the user wants to change the way the path is formatted. Examples:

(3) fix behavior of some accessor methods

here the behavior of a fsspec filesystem method is fixed by fixing the accessor method. Example:

(4) customize storage_options extraction

Not really specific to the Accessor subclassing, because the implementation is overriding __new__ in the UPath subclass. The intention is to extract additional fsspec storage_options from the provided args when instantiating UPath(). Example:

TODO

While this list in not complete, it does cover 3 4 reasonably easy to solve cases for making the transition to 3.12 less bumpy for custom subclasses.

  • (ALL) detect custom _accessor in py3.12 classes and provide warning with instructions on how to migrate.
  • (1) provide customizable fsspec filesystem factory classmethod
  • (2) provide instructions for how to migrate to .path (with potential customization)
  • (3) recommend overriding UPath.mkdir, etc... directly.
  • (4) provide customizable fsspec storage_option extraction classmethod
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🐛 Something isn't working compatibility 🤝 Compatibility with stdlib pathlib
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant