Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PLT-0] Improve Overall Dev-X By Using Rye #1492

Merged
merged 6 commits into from
Apr 8, 2024
Merged

[PLT-0] Improve Overall Dev-X By Using Rye #1492

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
226ba48
Initial move
adrian-chang Mar 15, 2024
68d47ae
always create build artifacts
adrian-chang Apr 3, 2024
81ee779
remove pytest ini
adrian-chang Apr 5, 2024
725533f
fix
adrian-chang Apr 6, 2024
ebd7c29
fix
adrian-chang Apr 6, 2024
9b710b4
reduce workers
adrian-chang Apr 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
28 changes: 28 additions & 0 deletions .github/actions/python-package-shared-setup/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: Labelbox Python SDK Shared Setup

inputs:
rye-version:
required: true
python-version:
required: true

runs:
using: "composite"
steps:
- name: Install the latest version of rye
uses: eifinger/setup-rye@v2
with:
version: ${{ inputs.rye-version }}
enable-cache: true
- name: Rye Setup
shell: bash
run: |
rye config --set-bool behavior.use-uv=true
- name: Python setup
shell: bash
run: rye pin ${{ inputs.python-version }}
- name: Environment setup
working-directory: libs/labelbox
shell: bash
run: |
rye sync -f --update-all
80 changes: 0 additions & 80 deletions .github/workflows/publish.yaml
View file
Open in desktop

This file was deleted.

173 changes: 173 additions & 0 deletions .github/workflows/publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
name: Labelbox Python SDK Publish

on:
workflow_dispatch:
inputs:
tag:
description: 'Release Tag'
required: true
skip-tests:
description: 'Skip PROD Test (Do not do this unless there is an emergency)'
default: 'false'
required: true
type: choice
options:
- false
- true


concurrency:
group: ${{ github.workflow }}
cancel-in-progress: false

# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
token: ${{ secrets.ACTIONS_ACCESS_TOKEN }}
ref: ${{ inputs.tag }}
- name: Install the latest version of rye
uses: eifinger/setup-rye@v2
with:
version: ${{ vars.RYE_VERSION }}
enable-cache: true
- name: Rye Setup
run: |
rye config --set-bool behavior.use-uv=true
- name: Create build
working-directory: libs/labelbox
run: |
rye sync
rye build
- uses: actions/upload-artifact@v4
with:
name: build
path: ./dist
test-build:
if: ${{ inputs.skip-tests }} != "true"
needs: ['build']
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- python-version: 3.8
prod-key: PROD_LABELBOX_API_KEY_2
da-test-key: DA_GCP_LABELBOX_API_KEY
- python-version: 3.9
prod-key: PROD_LABELBOX_API_KEY_3
da-test-key: DA_GCP_LABELBOX_API_KEY
- python-version: "3.10"
prod-key: PROD_LABELBOX_API_KEY_4
da-test-key: DA_GCP_LABELBOX_API_KEY
- python-version: 3.11
prod-key: LABELBOX_API_KEY
da-test-key: DA_GCP_LABELBOX_API_KEY
- python-version: 3.12
prod-key: PROD_LABELBOX_API_KEY_5
da-test-key: DA_GCP_LABELBOX_API_KEY
steps:
- uses: actions/checkout@v4
with:
token: ${{ secrets.ACTIONS_ACCESS_TOKEN }}
ref: ${{ inputs.tag }}
- name: Install the latest version of rye
uses: eifinger/setup-rye@v2
with:
version: ${{ vars.RYE_VERSION }}
enable-cache: true
- name: Rye Setup
run: |
rye config --set-bool behavior.use-uv=true
- name: Python setup
run: rye pin ${{ matrix.python-version }}
- uses: actions/download-artifact@v4
with:
name: build
path: ./dist
- name: Prepare package and environment
run: |
rye sync
rye run toml unset --toml-path pyproject.toml tool.rye.workspace
rye sync -f --update-all
- name: Integration Testing
env:
adrian-chang marked this conversation as resolved.
Show resolved Hide resolved
PYTEST_XDIST_AUTO_NUM_WORKERS: 32
LABELBOX_TEST_API_KEY: ${{ secrets[matrix.prod-key] }}
DA_GCP_LABELBOX_API_KEY: ${{ secrets[matrix.da-test-key] }}
LABELBOX_TEST_ENVIRON: prod
run: |
rye add labelbox --path ./$(find ./dist/ -name *.tar.gz) --sync
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just adding it as a review comment so that it doesnt get missed out

I havent ever used rye before.. couple of qns.

Is it an alternative to poetry? why not poetry?
On their website they say "Rye is still a very experimental tool" ..so is it stable and used widely in the industry?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://alpopkes.com/posts/python/packaging_tools/

^ look at the comparison, I personally don't want to deal with inconsistent environments (poetry doesn't solve the python-version problem, you'd have to use pyenv, something else)

https://astral.sh/blog/uv -> for corporate sponsorship

Star History Chart](https://star-history.com/#astral-sh/rye&python-poetry/poetry&Date) -> popularity, well I mean the creator did make Flask (if that is meaningful reputation wise).

https://rye-up.com/philosophy/

You can use poetry since, by definition, the pyproject.toml files are pep standard. Not against poetry, in fact, I've used / use it in other projects outside of LB.

example (open ai): https://github.com/openai/openai-python/blob/main/requirements.lock, codeinterpreter-api (https://github.com/shroominic/codeinterpreter-api/blob/main/requirements.lock)

To your point, there is some language (last year) indicating a more experimental attitude towards the work but the recent website updates are starting to eliminate this language.

cd libs/labelbox
rm pyproject.toml
rye run pytest tests/integration
- name: Data Testing
env:
PYTEST_XDIST_AUTO_NUM_WORKERS: 32
LABELBOX_TEST_API_KEY: ${{ secrets[matrix.prod-key] }}
DA_GCP_LABELBOX_API_KEY: ${{ secrets[matrix.da-test-key] }}
LABELBOX_TEST_ENVIRON: prod
run: |
rye add labelbox --path ./$(find ./dist/ -name *.tar.gz) --sync --features data
cd libs/labelbox
rye run pytest tests/data
pypi-publish:
runs-on: ubuntu-latest
needs: ['test-build']
environment:
name: publish
url: 'https://pypi.org/project/labelbox/'
permissions:
# IMPORTANT: this permission is mandatory for trusted publishing
id-token: write
steps:
- uses: actions/download-artifact@v4
with:
name: build
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
packages-dir: artifact/
container-publish:
runs-on: ubuntu-latest
needs: ['test-build']
env:
CONTAINER_IMAGE: "ghcr.io/${{ github.repository }}"
steps:
- name: downcase CONTAINER_IMAGE
run: |
echo "CONTAINER_IMAGE=${CONTAINER_IMAGE,,}" >> ${GITHUB_ENV}

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build and push
uses: docker/build-push-action@v5
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
push: true

platforms: |
linux/amd64
linux/arm64
adrian-chang marked this conversation as resolved.
Show resolved Hide resolved

tags: |
${{ env.CONTAINER_IMAGE }}:latest
${{ env.CONTAINER_IMAGE }}:${{ inputs.tag }}

# Note that the build and pypi-publish jobs are split so that the additional permissions are only granted to the pypi-publish job.
Loading