Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Apple silicon & Python 3.12 #20

Merged
merged 18 commits into from
May 4, 2024
Merged

Apple silicon & Python 3.12 #20

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
da009e7
Update test.yml
martinellimarco May 21, 2023
97c100c
Update test.yml
martinellimarco May 21, 2023
ac8a112
Update test.yml
martinellimarco May 21, 2023
e1f3875
Update test.yml
martinellimarco May 21, 2023
34f0eca
Update test.yml
martinellimarco May 4, 2024
f68130e
Update test.yml
martinellimarco May 4, 2024
bd735ff
Update test.yml
martinellimarco May 4, 2024
58136af
Update test.yml
martinellimarco May 4, 2024
dea791b
Update test.yml
martinellimarco May 4, 2024
b37a1dc
Update test.yml
martinellimarco May 4, 2024
68efee9
Update test.yml
martinellimarco May 4, 2024
5a92b30
Update test.yml
martinellimarco May 4, 2024
ffd7120
Update test.yml
martinellimarco May 4, 2024
cd1c4fb
Update test.yml
martinellimarco May 4, 2024
f2805fd
Update test.yml
martinellimarco May 4, 2024
8375f26
Update _get_zstd.py
martinellimarco May 4, 2024
78fdd3a
Update test.yml
martinellimarco May 4, 2024
d60020b
Update publish.yml
martinellimarco May 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions .github/workflows/publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ jobs:

strategy:
matrix:
os: [macos-latest, ubuntu-latest, windows-latest]
os: [macos-13, ubuntu-latest, windows-latest]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you want both: os: [macos-13, macos-14, ubuntu-latest, windows-latest]
macos-13 for building for x86-64 and macos-14 for building with ARM. At least that's how I did it in my CI.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried macos-14 but for some reason it's not finding zstd.h. It works fine in macos-13 with no other change.
I have CIBW_ARCHS_MACOS: "x86_64 arm64" which seems to be working fine to build both x86-64 and arm64 wheels.
See this test under check wheels. Does it seem ok to you?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah nice, looks good and seems more reliable and self-explanatory than running on different macos versions.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good, I'm going ahead and publishing v1.6.1 :) Let's hope for the best.

manylinux-image: [manylinux2014]

steps:
Expand All @@ -23,12 +23,12 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.11'
python-version: '3.12'

- name: Install Dependencies
run: |
python3 -m pip install --upgrade pip
python3 -m pip install --upgrade-strategy eager --upgrade cython twine cibuildwheel
python3 -m pip install --upgrade-strategy eager --upgrade cython twine cibuildwheel setuptools

- name: Build Tarball
if: matrix.os == 'ubuntu-latest'
Expand All @@ -39,8 +39,9 @@ jobs:
CIBW_MANYLINUX_X86_64_IMAGE: ${{ matrix.manylinux-image }}
CIBW_MUSLLINUX_X86_64_IMAGE: musllinux_1_1
CIBW_ARCHS: auto64
CIBW_ARCHS_MACOS: "x86_64 arm64"
CIBW_BEFORE_ALL_LINUX: yum install -y libzstd-devel || apk add zstd-dev
CIBW_BEFORE_ALL_MACOS: brew install zstd
CIBW_BEFORE_ALL_MACOS: brew update && brew reinstall zstd
CIBW_BEFORE_BUILD_WINDOWS: "python {package}/libzstd/_get_zstd.py && pip install delvewheel"
CIBW_REPAIR_WHEEL_COMMAND_WINDOWS: "delvewheel repair -w {dest_dir} {wheel}"
run: python -m cibuildwheel --output-dir dist
Expand All @@ -64,7 +65,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.11'
python-version: '3.12'

- name: Install Dependencies
run: |
Expand Down
23 changes: 16 additions & 7 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,10 @@ jobs:
# 3.9 was released 2020-10-05 and end-of-life will be 2025-10-25
# 3.10 was released 2021-10-04 and end-of-life will be 2026-10-25
# 3.11 was released 2022-10-24 and end-of-life will be 2027-10-24
python-version: ['3.7', '3.11']
os: [macos-latest, ubuntu-latest, windows-latest]
# 3.12 was released 2023-10-02 and end-of-life will be 2028-10
python-version: ['3.8', '3.12']
#os: [macos-13, macos-14, ubuntu-latest, windows-latest]
os: [macos-13, ubuntu-latest, windows-latest]

steps:
- uses: actions/checkout@v2
Expand All @@ -28,6 +30,11 @@ jobs:
with:
python-version: ${{ matrix.python-version }}

- name: Install Dependencies (macos)
if: matrix.os == 'macos-13' || matrix.os == 'macos-14'
run: |
brew update && brew reinstall zstd

- name: Install Dependencies (ubuntu)
if: matrix.os == 'ubuntu-latest'
run: |
Expand All @@ -43,7 +50,7 @@ jobs:
- name: Install Python Modules
run: |
python3 -m pip install --upgrade pip
python3 -m pip install --upgrade-strategy eager --upgrade cython twine cibuildwheel
python3 -m pip install --upgrade-strategy eager --upgrade cython twine cibuildwheel setuptools

- name: Test Installation From Tarball
shell: bash
Expand All @@ -65,7 +72,8 @@ jobs:

strategy:
matrix:
os: [macos-latest, ubuntu-latest, windows-latest]
#os: [macos-13, macos-14, ubuntu-latest, windows-latest]
os: [macos-13, ubuntu-latest, windows-latest]
manylinux-image: [manylinux2014]

steps:
Expand All @@ -76,12 +84,12 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.11'
python-version: '3.12'

- name: Install Python Dependencies
run: |
python3 -m pip install --upgrade pip
python3 -m pip install --upgrade-strategy eager --upgrade cython twine cibuildwheel
python3 -m pip install --upgrade-strategy eager --upgrade cython twine cibuildwheel setuptools

- name: Build Tarball
if: matrix.os == 'ubuntu-latest'
Expand All @@ -92,8 +100,9 @@ jobs:
CIBW_MANYLINUX_X86_64_IMAGE: ${{ matrix.manylinux-image }}
CIBW_MUSLLINUX_X86_64_IMAGE: musllinux_1_1
CIBW_ARCHS: auto64
CIBW_ARCHS_MACOS: "x86_64 arm64"
CIBW_BEFORE_ALL_LINUX: yum install -y libzstd-devel || apk add zstd-dev
CIBW_BEFORE_ALL_MACOS: brew install zstd
CIBW_BEFORE_ALL_MACOS: brew update && brew reinstall zstd
CIBW_BEFORE_BUILD_WINDOWS: "python {package}/libzstd/_get_zstd.py && pip install delvewheel"
CIBW_REPAIR_WHEEL_COMMAND_WINDOWS: "delvewheel repair -w {dest_dir} {wheel}"
run: python -m cibuildwheel --output-dir dist
Expand Down
15 changes: 11 additions & 4 deletions libzstd/_get_zstd.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,13 +36,20 @@


with zipfile.ZipFile(io.BytesIO(zip_data)) as zf:
INCLUDE_DIR.joinpath("zstd.h").write_bytes(zf.read("include/zstd.h"))
LIBRARY_DIR.joinpath("libzstd.dll").write_bytes(zf.read("dll/libzstd.dll"))
files = zf.namelist()
zstd_h = next((f for f in files if "include/zstd.h" in f), None)
libzstd_dll = next((f for f in files if "dll/libzstd.dll" in f), None)
libzstd_lib = next((f for f in files if "dll/libzstd.lib" in f), None)
libzstd_dll_a = next((f for f in files if "dll/libzstd.dll.a" in f), None)

INCLUDE_DIR.joinpath("zstd.h").write_bytes(zf.read(zstd_h))
LIBRARY_DIR.joinpath("libzstd.dll").write_bytes(zf.read(libzstd_dll))
try:
_libzstd_lib = zf.read("dll/libzstd.lib")
_libzstd_lib = zf.read(libzstd_lib)
except KeyError:
_libzstd_lib = zf.read("dll/libzstd.dll.a")
_libzstd_lib = zf.read(libzstd_dll_a)
# this renames libzstd.dll.a to libzstd.lib for setuptools to work
LIBRARY_DIR.joinpath("libzstd.lib").write_bytes(_libzstd_lib)


print("success")
Loading