Drop Python 3.7 and add Python 3.12 #112
Conversation
BREAKING CHANGE: This drops support for python 3.7 which has reached its end of life
- dirty first pass, validating against the FS adapter test suite. - 38 fail / 92 passed so far - test_pathy, test_pure_pathy, test_cli left for FS adapter BREAKING CHANGE: Pathy.key returns a str rather than a Pathy instance There was nowhere in the code where I wasn't casting the .key access to string. This removes all those casts and saves a little effort. It's possible that some user expects .key to be a Path still, so add a breaking change.
- still need to check tests against cloud adapters - this disabled some tests and functionality, but nothing in terms of the major public API - old Pathy inherited some local file behaviors from pathlib.Path and those are now gone. This is a breaking change, but is unlikely to impact most use-cases. BREAKING CHANGE: Pathy no longer inherits from pahtlib.Path This means Pathy does not support directly accepting and working with file system paths. You must use Pathy.fluid or pathlib.Path to construct your file system paths. Pathy will continue to interoperate with them as needed to accomodate its public API.
- remove as_uri for gcs paths. - fix sep access since _flavour is no more :sun:
justindujardin
changed the title
- auto format removed apparently unused posixpath imports. Alias them directly - implement is_symlink to stop the base class from continuing down a bad path
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #112 +/- ##
==========================================
- Coverage 99.91% 99.91% -0.01%
==========================================
Files 6 7 +1
Lines 1159 1122 -37
==========================================
- Hits 1158 1121 -37
Misses 1 1 ☔ View full report in Codecov by Sentry. |
- use a more foolproof invocation of bdist_wheel: https://stackoverflow.com/a/70459530/287335 - resolve build package warning about wanting underscore case keys
- derp couldn't ever be hit given the preceeding code
- oof, thanks cli
- I'll reapply it in another PR where it doesn't skew the diff stats so much
- Pathy wants to provide a consistent API when using pathlib Paths or Bucket paths. To that end we extend pathlib.Path for all POSIX/Windows paths returned from pathy (usually in .fluid()) to add ls() - drop some dead code and remove some todos
- we can update it as needed
|
|
||
| _flavour = _PathyFlavour() | ||
| pathmod = pathy_pathmod |
There was a problem hiding this comment.
@barneygale this will be a problem, right?
There was a problem hiding this comment.
@merwok thanks for the review!
Can you tell me why you think this will be a problem? I'm happy to fix something if it's wrong.
This code is exercised a bunch by the test suite, and the behaviors are as expected across Python (3.8-3.12), so hopefully it's not too wrong.
I assumed that the introduction of pathmod here was to make it more easily accessible to the path classes (e.g., you can say some_path.pathmod.sep and get the proper separator to use.
There was a problem hiding this comment.
Oh, because this attribute is not public in pathlib anymore: python/cpython#100502 (comment)
There was a problem hiding this comment.
I think I understand, thanks. 🙇
I removed the assignment of pathmod with Pathy's specific module, but it caused cascading failures in my test suite.
The trouble is that the pathmod defines how to split the string for root/drv/tail, and without it, my bucket paths get mangled.
If we shouldn't override pathmod, we'll need another way to control the path-splitting logic.
There was a problem hiding this comment.
Because this is functional and fixes our broken main branch build, I will merge this now.
Please feel free to comment here or in another issue and we can refine this implementation where needed.
There was a problem hiding this comment.
@merwok pathmod is still public! You were right to urge me to keep it, as it rather nicely abstracts away all the pure functionality of the path using composition rather than inheritance.
I'm thinking of adding a PathModuleBase to define its interface: #113893
There was a problem hiding this comment.
It is? Thought it was added in 3.13 then removed (so renamed to _pathmod in the end)
Edit: mixed up the commit on your repo and PRs to upstream!
There was a problem hiding this comment.
I considered renaming it to _pathmod but decided against it, given your advice to wait: python/cpython#100502 (comment)
# [0.11.0](v0.10.3...v0.11.0) (2024-01-11) ### Features * Drop Python 3.7 and add Python 3.12 ([#112](#112)) ([97ebaa1](97ebaa1)) ### BREAKING CHANGES * This drops support for python 3.7 which has reached its end of life * feat(Pathy): integrate pathlib_abc - replace base pathlib.Path class with abstract base class from a future version of python. * Pathy.key returns a str rather than a Pathy instance * feat(ci): add python 3.12 * Pathy no longer inherits from pahtlib.Path This means Pathy does not support directly accepting and working with file system paths. You must use Pathy.fluid or pathlib.Path to construct your file system paths. Pathy will continue to interoperate with them as needed to accommodate its public API.
|
🎉 This PR is included in version 0.11.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
The circle of life is beautiful. Python 3.7 has reached its end, and Python 3.12 rises to carry on its name.
Motivation
Pathy has relied on the internals of pathlib.Path to support complex things like glob matching and the differences between Windows and Posix paths. Because of this when the python developers rightly refactor their internals, Pathy breaks. 😭
We need to stop depending on the internals of pathlib to avoid the yearly "new version of python breakage". It was on my list to inline the parts of pathlib needed and call it good, but then the absolute legend @barneygale published a pypi package pathlib_abc that's synced from CPython and implements the logic we depended on inside of pathlib.
Now, we can have a proper public API to work from and not lose sleep every time a new version of Python is released. 😎
This PR updates Pathy to use pathlib_abc and supports Python 3.12 while deprecating Python 3.7, which reached end-of-life in the middle of last year.
Changes
pathlib.Pathwithpathlib_abc.BasePathBreaking Changes
Pathyno longer inherits frompathlib.Pathchanging type checks looking at a Pathy comparing it for pathlib.Path. This is better overall but could break user code in rare cases.Pathyno longer raises a ValueError when given an invalid bucket path. It's unclear how to cleanly differentiate file system paths from chunks of paths used for constructing bucket paths, except in the absolute path case.Pathyno longer automatically collapses relative paths. BeforePathy('gs://bucket/foo/../bar')would automatically becomePathy('gs://bucket/bar'). Now you have to useresolve()to get the same outputPathy('gs://bucket/foo/../bar').resolve()