drop python 3.8, enable python 3.13, and enable full linting for 3.12 #2194
Conversation
✅ Deploy Preview for dlt-hub-docs canceled.
|
sh-rp
force-pushed
the
tmp/test_py_versions
branch
3 times, most recently
from
368e04d to
d0de684
Compare
sh-rp
changed the title
sh-rp
changed the title
sh-rp
force-pushed
the
tmp/test_py_versions
branch
from
d0de684 to
8cc3b7a
Compare
sh-rp
changed the title
sh-rp
changed the title
sh-rp
force-pushed
the
tmp/test_py_versions
branch
2 times, most recently
from
5e4f829 to
fc4006c
Compare
| requests = ">=2.26.0" | ||
| pendulum = ">=2.1.2" | ||
| # TODO: pin to tag on dlt-pendulum repo | ||
| dlt-pendulum = [ |
There was a problem hiding this comment.
NOTE: this still requires cargo at this point
There was a problem hiding this comment.
@rudolfix I have another question about this: should we not have a different package name for our pendulum? The way it is currently set up, dlt-pendulum is always used, also for python versions before 3.13, this means there will be package name collissions if users also have the regular pendulum in their dependencies already in a project. I'm not sure what happens in this case.
Also you mentioned that you made some changes they will not like and there is a change in the code where you removed the casting of pendulum.now() to a string, are the libraries now incompatible?
There was a problem hiding this comment.
We are now using a properly build dlt-pendulum for python 3.13, so that part is solved. The tests will verify wether this string removal still works or not.
sh-rp
force-pushed
the
tmp/test_py_versions
branch
2 times, most recently
from
7f22437 to
e46f21c
Compare
sh-rp
changed the title
| @@ -26,7 +26,7 @@ jobs: | |||
| matrix: | |||
| os: | |||
| - ubuntu-latest | |||
| python-version: ["3.8.x", "3.9.x", "3.10.x", "3.11.x"] | |||
| python-version: ["3.9.x", "3.10.x", "3.11.x", "3.12.x"] | |||
There was a problem hiding this comment.
make dev works for 3.12 now!
sh-rp
force-pushed
the
tmp/test_py_versions
branch
from
cca2c57 to
b3372de
Compare
| @@ -1,4 +1,4 @@ | |||
| FROM alpine:3.15 | |||
| FROM python:3.11.11-alpine | |||
There was a problem hiding this comment.
Note: switched to a python image, I can probably revert this if we want to use a vanilla alpine here.
|
|
||
| # install arrow 17.0.0, usually we would need apache-arrow-dev=17.0.0 but it is not available in alpine 3.20 | ||
| # adapt this version to the arrow version you need | ||
| RUN git clone --no-checkout https://github.com/apache/arrow.git /arrow \ |
There was a problem hiding this comment.
as noted in the comment, alpine always seems to come with one version of each package, i remember something like this from our last company, so I need to build arrow 17 here if I want to use it..
| # poetry export -f requirements.txt --output _gen_requirements.txt --without-hashes --extras gcp --extras redshift | ||
| # grep `cat compiled_packages.txt` _gen_requirements.txt > compiled_requirements.txt | ||
| poetry install --no-interaction -E gcp -E redshift -E duckdb | ||
| poetry run pip freeze > _gen_requirements.txt |
There was a problem hiding this comment.
this works well, poetry export was failing on several dependencies..
Description
This PR does the following:
Based on #2047
ToDo: