Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI Overhaul #1656

Merged
merged 47 commits into from
Apr 24, 2024
Merged
Changes from 1 commit
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
3963f4a
test ci
pingsutw Apr 22, 2024
af61e2c
test ci
pingsutw Apr 22, 2024
32452bc
test pre
pingsutw Apr 22, 2024
481e6fe
test
pingsutw Apr 22, 2024
8a16857
test
pingsutw Apr 22, 2024
7ce9331
test ci
pingsutw Apr 22, 2024
1d39345
test airflow
pingsutw Apr 22, 2024
0faa014
test
pingsutw Apr 23, 2024
003747d
Add requirements.in
pingsutw Apr 23, 2024
5bd8b27
Add more examples
pingsutw Apr 23, 2024
95bf416
test
pingsutw Apr 23, 2024
e42c5ba
Update chatgpt example
pingsutw Apr 23, 2024
e1196b5
test
pingsutw Apr 23, 2024
469640f
test
pingsutw Apr 23, 2024
eaccdb6
test
pingsutw Apr 23, 2024
baccfdb
test
pingsutw Apr 23, 2024
8a2c7e9
test
pingsutw Apr 23, 2024
5f06346
test
pingsutw Apr 23, 2024
f07c854
test
pingsutw Apr 23, 2024
90c4804
test
pingsutw Apr 23, 2024
41085e9
test
pingsutw Apr 23, 2024
e3697d6
test
pingsutw Apr 23, 2024
82a3587
test
pingsutw Apr 23, 2024
5380b6a
test
pingsutw Apr 23, 2024
c7c0f89
test
pingsutw Apr 23, 2024
3162b1b
test
pingsutw Apr 23, 2024
ce7b807
test
pingsutw Apr 23, 2024
6384e78
test
pingsutw Apr 23, 2024
dd96f17
test
pingsutw Apr 23, 2024
21143ed
test
pingsutw Apr 23, 2024
5a91a8c
test
pingsutw Apr 23, 2024
9ae8db3
test
pingsutw Apr 23, 2024
c9e0047
tesat
pingsutw Apr 23, 2024
3c857ed
test
pingsutw Apr 23, 2024
5c79466
test
pingsutw Apr 23, 2024
489d145
test
pingsutw Apr 23, 2024
67d9b61
test
pingsutw Apr 23, 2024
28d4fa5
test
pingsutw Apr 23, 2024
d9fa22d
test
pingsutw Apr 23, 2024
0b35a97
nit
pingsutw Apr 23, 2024
2d1a623
tet
pingsutw Apr 23, 2024
d2f70a6
test
pingsutw Apr 23, 2024
40d6ad2
update decks example
pingsutw Apr 23, 2024
36aacfb
nit
pingsutw Apr 23, 2024
4bd8fb9
lint
pingsutw Apr 23, 2024
337fdf0
nit
pingsutw Apr 23, 2024
5ff5cda
nit
pingsutw Apr 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
test
Signed-off-by: Kevin Su <pingsutw@apache.org>
pingsutw committed Apr 23, 2024
commit 469640f025f0166a2c1e548d5aaf2bc4baf9837b
39 changes: 14 additions & 25 deletions .github/workflows/checks_new.yml
Original file line number Diff line number Diff line change
@@ -15,37 +15,25 @@ concurrency:
cancel-in-progress: true

jobs:
# This is the build system for the new example directory structure
list_examples:
runs-on: ubuntu-latest
name: "Create a list of example packages"
steps:
- uses: actions/checkout@v2
- name: "Provide the list"
id: create-example-list
run: echo "PACKAGES=$(find examples -mindepth 1 -maxdepth 2 -type f -name Dockerfile -exec dirname '{}' \; | sort | sed 's/examples\///' | jq --raw-input . | jq --slurp . | jq -c .)"
outputs:
packages: "${{ steps.create-example-list.outputs.PACKAGES }}"
serialize_examples:
needs: [list_examples]
runs-on: "ubuntu-latest"
name: Serialize
strategy:
fail-fast: false
matrix:
example:
- advanced_composition
- airflow_agent
- k8s_spark_plugin
- athena_plugin
- aws_batch_plugin
- basics
- bigquery_agent
- bigquery_plugin
- blast
- chatgpt_agent
- customizing_dependencies
- data_types_and_io
- dbt_plugin
- development_lifecycle
- dolt_plugin
- duckdb_plugin
- exploratory_data_analysis
- extending
- feast_integration
- forecasting_sales
- greatexpectations_plugin
- hive_plugin
- house_price_prediction
- k8s_dask_plugin
example: "${{ fromJson(needs.list_examples.outputs.packages) }}"
steps:
- uses: actions/checkout@v4
- name: Set up Python
@@ -61,6 +49,7 @@ jobs:
key: ${{ format('{0}-pip-{1}', runner.os, hashFiles('dev-requirements.in', 'requirements.in')) }}
- name: Install dependencies
run: |
sudo apt install python3-packaging
pip install uv
uv venv
source .venv/bin/activate