Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pruning support in the pipeline for CI to aid with speed #604

Merged
merged 5 commits into from
Jan 2, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/wicked-planes-talk.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@tokens-studio/graph-engine-nodes-design-tokens": patch
---

Fixes the incorrectly typed Tokenset Schema
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this also include some info of the pipeline change?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, nothing within the packages changes from the pipeline change, this is only for customer facing info

82 changes: 80 additions & 2 deletions .github/workflows/pr-ci.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
name: CI

on: pull_request
on:
pull_request:
types: [opened, synchronize]

env:
HUSKY: 0
Expand All @@ -16,9 +18,82 @@ concurrency:
cancel-in-progress: true

jobs:


changes:
runs-on: ubuntu-latest
env:
TURBO_RUN_FILTER: ${{ github.event_name == 'pull_request' && '...[origin/master]' || '...[HEAD^]' }}
# Required permissions
permissions:
pull-requests: read
# Set job outputs to values from filter step
outputs:
flat: ${{ steps.flat.outputs.result }}
steps:
- name: Check out code
uses: actions/checkout@v3
with:
fetch-depth: 0
lfs: true
- name: Setup Node.js environment
uses: actions/setup-node@v4
with:
cache: 'npm'
node-version: 20
- name: Install [email protected] & glob
run: |
npm i -g [email protected] glob
echo "PATH=$(npm config get config):$PATH" >> $GITHUB_ENV

- name: Verify turbo installation
shell: bash
run: |
source $GITHUB_ENV
turbo --version
- name: Debug TURBO_RUN_FILTER
run: echo "TURBO_RUN_FILTER is $TURBO_RUN_FILTER"
- name: Validate JSON Output
run: |
content=$(turbo build --filter=$TURBO_RUN_FILTER --dry-run=json)
echo $content | jq .
- name: Debug turbo command
run: |
set -x
turbo build --filter=$TURBO_RUN_FILTER --dry-run=json
set +x
- name: Verify turbo installation
run: which turbo

- name: Debug content
run: |
content=`turbo build --filter=$TURBO_RUN_FILTER --dry-run=json`
echo "Content is: $content"
- name: Changeset
id: changeset
run: |
content=`turbo build --filter=$TURBO_RUN_FILTER --dry-run=json`
content="${content//'%'/'%25'}"
content="${content//$'\n'/'%0A'}"
content="${content//$'\r'/'%0D'}"
echo "::set-output name=result::$content"
- name: Flat output
id: flat
run: |
filteredPkgs=`turbo build --filter=$TURBO_RUN_FILTER --dry-run=json | node ./scripts/diff.js`
echo "::set-output name=result::$filteredPkgs"
echo "$filteredPkgs"


verify:
name: Verify changes
needs: changes
runs-on: ubuntu-latest
if: ${{ needs.changes.outputs.flat != '[]' }}
defaults:
run:
shell: bash
working-directory: ./out
steps:
- name: Check out code
uses: actions/checkout@v3
Expand All @@ -32,7 +107,10 @@ jobs:
cache: "yarn"

- name: Install turbo
run: yarn global add [email protected]
run: yarn global add [email protected]
- name: prune
run: turbo prune ${{ needs.changes.outputs.flat }}
working-directory: ./

- name: Turbo Cache
id: turbo-cache
Expand Down
Empty file added out/.gitkeep
Empty file.
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@
"husky": "^8.0.3",
"lint-staged": "^13.2.1",
"listr2": "5.0.8",
"turbo": "^1.9.8"
"turbo": "^1.13.3"
},
"packageManager": "[email protected]"
}
6 changes: 1 addition & 5 deletions packages/nodes-design-tokens/src/schemas/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -89,11 +89,7 @@ export const TOKEN_SET = 'https://schemas.tokens.studio/tokenSet.json';
export const TokenSetSchema: SchemaObject = {
$id: TOKEN_SET,
title: 'Token Set',
type: 'object',
properties: {
name: StringSchema,
token: StringSchema
}
type: 'object'
};

export const LEONARDO_COLOR =
Expand Down
24 changes: 24 additions & 0 deletions scripts/diff.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
const stdin = process.openStdin();

let data = '';

stdin.on('data', function (chunk) {
data += chunk;
});

stdin.on('end', function () {
const input = JSON.parse(data);
const flatpack = input.tasks.reduce((acc, task) => {
if (task.command == '\u003cNONEXISTENT\u003e') {
return acc;
}

if (task.cache.status == 'MISS') {
acc.push(task.package);
}
return acc;
}, []);

const uniq = [...new Set(flatpack)];
console.log(uniq.join(' '));
});
2 changes: 1 addition & 1 deletion yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -21482,7 +21482,7 @@ [email protected]:
resolved "https://registry.yarnpkg.com/turbo-windows-arm64/-/turbo-windows-arm64-1.13.4.tgz#0adfddb0d6c167a46ecff646666e347a7d372d76"
integrity sha512-u5A+VOKHswJJmJ8o8rcilBfU5U3Y1TTAfP9wX8bFh8teYF1ghP0EhtMRLjhtp6RPa+XCxHHVA2CiC3gbh5eg5g==

turbo@^1.9.8:
turbo@^1.13.3:
version "1.13.4"
resolved "https://registry.yarnpkg.com/turbo/-/turbo-1.13.4.tgz#06767fff53f0aae43f78e12e5ac7d5e7652d40d0"
integrity sha512-1q7+9UJABuBAHrcC4Sxp5lOqYS5mvxRrwa33wpIyM18hlOCpRD/fTJNxZ0vhbMcJmz15o9kkVm743mPn7p6jpQ==
Expand Down
Loading