Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Release 2.60.0] Cherry-pick #32725 into release branch #32761

Merged
merged 1 commit into from
Oct 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/build_release_candidate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -310,11 +310,12 @@ jobs:
uses: actions/setup-node@v4
with:
node-version: '16'
- name: Install Java 11
# TODO(https://github.com/apache/beam/issues/32726) switch to Java11
- name: Install Java 8
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: '11'
java-version: '8'
- name: Remove default github maven configuration
# This step is a workaround to avoid a decryption issue of Beam's
# net.linguica.gradle.maven.settings plugin and github's provided maven
Expand Down
34 changes: 10 additions & 24 deletions .github/workflows/build_wheels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,6 @@ jobs:
if: needs.check_env_variables.outputs.gcp-variables-set == 'true'
steps:
- name: Download compressed sources from artifacts
# Pinned to v3 because of https://github.com/actions/download-artifact/issues/249
uses: actions/[email protected]
with:
name: source_zip
Expand All @@ -229,18 +228,16 @@ jobs:
{"os": "windows-latest", "runner": "windows-latest", "python": "${{ needs.check_env_variables.outputs.py-versions-test }}", arch: "auto" },
{"os": "ubuntu-20.04", "runner": [self-hosted, ubuntu-20.04, main], "python": "${{ needs.check_env_variables.outputs.py-versions-test }}", arch: "aarch64" }
]
# Keep in sync with PY_VERSIONS_FULL env var abvove - if changed, change that as well.
py_version: ["cp38-*", "cp39-*", "cp310-*", "cp311-*", "cp312-*"]
# Keep in sync (remove asterisks) with PY_VERSIONS_FULL env var above - if changed, change that as well.
py_version: ["cp38-", "cp39-", "cp310-", "cp311-", "cp312-"]
steps:
- name: Download python source distribution from artifacts
# Pinned to v3 because of https://github.com/actions/download-artifact/issues/249
uses: actions/[email protected]
with:
name: source
path: apache-beam-source
- name: Download Python SDK RC source distribution from artifacts
if: ${{ needs.build_source.outputs.is_rc == 1 }}
# Pinned to v3 because of https://github.com/actions/download-artifact/issues/249
uses: actions/[email protected]
with:
name: source_rc${{ needs.build_source.outputs.rc_num }}
Expand All @@ -260,7 +257,7 @@ jobs:
if: ${{ contains(matrix.os_python.python, matrix.py_version) }}
working-directory: apache-beam-source
env:
CIBW_BUILD: ${{ matrix.py_version }}
CIBW_BUILD: ${{ matrix.py_version }}*
# TODO: https://github.com/apache/beam/issues/23048
CIBW_SKIP: "*-musllinux_*"
CIBW_BEFORE_BUILD: pip install cython==0.29.36 numpy --config-settings=setup-args="-Dallow-noblas=true" && pip install --upgrade setuptools
Expand All @@ -279,17 +276,16 @@ jobs:
shell: bash
- name: Upload wheels as artifacts
if: ${{ contains(matrix.os_python.python, matrix.py_version) }}
# Pinned to v3 because of https://github.com/actions/upload-artifact?tab=readme-ov-file#breaking-changes
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: wheelhouse-${{ matrix.os_python.os }}${{ (matrix.os_python.arch == 'aarch64' && '-aarch64') || '' }}
name: wheelhouse-${{ matrix.py_version }}${{ matrix.os_python.os }}${{ (matrix.os_python.arch == 'aarch64' && '-aarch64') || '' }}
path: apache-beam-source/wheelhouse/
- name: Build RC wheels
# Only build wheel if it is one of the target versions for this platform, otherwise no-op
if: ${{ needs.build_source.outputs.is_rc == 1 && contains(matrix.os_python.python, matrix.py_version) }}
working-directory: apache-beam-source-rc
env:
CIBW_BUILD: ${{ matrix.py_version }}
CIBW_BUILD: ${{ matrix.py_version }}*
# TODO: https://github.com/apache/beam/issues/23048
CIBW_SKIP: "*-musllinux_*"
CIBW_BEFORE_BUILD: pip install cython==0.29.36 numpy --config-settings=setup-args="-Dallow-noblas=true" && pip install --upgrade setuptools
Expand All @@ -305,10 +301,9 @@ jobs:
shell: bash
- name: Upload RC wheels as artifacts
if: ${{ needs.build_source.outputs.is_rc == 1 }}
# Pinned to v3 because of https://github.com/actions/download-artifact/issues/249
uses: actions/upload-artifact@v4
with:
name: wheelhouse-rc${{ needs.build_source.outputs.rc_num }}-${{ matrix.os_python.os }}${{ (matrix.arch == 'aarch64' && '-aarch64') || '' }}
name: wheelhouse-rc${{ needs.build_source.outputs.rc_num }}-${{ matrix.py_version }}${{ matrix.os_python.os }}${{ (matrix.os_python.arch == 'aarch64' && '-aarch64') || '' }}
path: apache-beam-source-rc/wheelhouse/

upload_wheels_to_gcs:
Expand All @@ -318,21 +313,12 @@ jobs:
- check_env_variables
runs-on: [self-hosted, ubuntu-20.04, main]
if: needs.check_env_variables.outputs.gcp-variables-set == 'true' && github.event_name != 'pull_request'
strategy:
matrix:
# Temporarily pin to macos-13 because macos-latest breaks this build
# TODO(https://github.com/apache/beam/issues/31114)
os : [ubuntu-20.04, macos-13, windows-latest]
arch: [auto]
include:
- os: ubuntu-20.04
arch: aarch64
steps:
- name: Download wheels from artifacts
# Pinned to v3 because of https://github.com/actions/upload-artifact?tab=readme-ov-file#breaking-changes
uses: actions/download-artifact@v3
uses: actions/download-artifact@v4
with:
name: wheelhouse-${{ matrix.os }}${{ (matrix.arch == 'aarch64' && '-aarch64') || '' }}
pattern: wheelhouse-*
merge-multiple: true
path: wheelhouse/
- name: Copy wheels to GCS bucket
run: gsutil cp -r -a public-read wheelhouse/* ${{ env.GCP_PATH }}
Expand Down
Loading