Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce transfer size of checkout mode transfers #4

Merged
merged 5 commits into from
May 26, 2022
Merged

Reduce transfer size of checkout mode transfers #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
b8b851f
Transfer only the `.git` directory to save data
ps-jay May 26, 2022
06c260f
Fix logic for paths & improve checkout
ps-jay May 26, 2022
1f9d003
Update README
ps-jay May 26, 2022
54d4d6e
Improve tests by moving the action out of the way
ps-jay May 26, 2022
df96f6d
Add test for non-checkout transfers
ps-jay May 26, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 53 additions & 6 deletions .github/workflows/test-and-release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,15 +28,41 @@ jobs:
- id: checkout
name: Checkout code 🛒
uses: actions/checkout@v3
with:
path: transfer-action

- id: create-file
name: Create a file 🆕
shell: bash
run: |
set -euo pipefail
touch foo

- id: given-upload
name: Upload a given file 📤
uses: ./transfer-action/
with:
direction: 'upload'
name: 'foo'
path: './foo'

- id: checkout-upload
name: Checkout and upload 📦
uses: ./
uses: ./transfer-action/
with:
checkout: 'true'
direction: 'upload'

- id: checkout-again
# have to do this because the `checkout-upload` step cleans the working dir
name: Checkout code (again) 🛒
uses: actions/checkout@v3
with:
path: transfer-action

outputs:
name: ${{ steps.checkout-upload.outputs.name }}
checkout-name: ${{ steps.checkout-upload.outputs.name }}
given-name: ${{ steps.given-upload.outputs.name }}

test-down:
runs-on: ubuntu-latest
Expand All @@ -45,21 +71,42 @@ jobs:
- id: checkout
name: Checkout code 🛒
uses: actions/checkout@v3
with:
path: transfer-action

- id: checkout-download
name: Download checkout 📥
uses: ./
uses: ./transfer-action/
with:
checkout: 'true'
direction: 'download'
name: ${{ needs.test-up.outputs.name }}
path: 'download/'
name: ${{ needs.test-up.outputs.checkout-name }} # not strictly needed, but tests the output was set

- id: test-git
name: Git status 🤷
run: |
set -euo pipefail
cd download
git status
if [[ ! -f action.yaml ]] ; then
echo "🛑 'action.yaml' is not found!"
exit 1
fi

- id: given-download
name: Download given file 📨
uses: ./transfer-action/
with:
direction: 'download'
name: ${{ needs.test-up.outputs.given-name }}

- id: test-file
name: Did the file download? 🔎
run: |
set -euo pipefail
if [[ ! -f foo ]] ; then
echo "🛑 'foo' is not found!"
exit 1
fi

update-doc:
if: ${{ github.ref_name != github.event.repository.default_branch }}
Expand Down
28 changes: 14 additions & 14 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ This action transfers data to/from artefact storage
runs-on: [self-hosted]
steps:
- name: Upload repo
uses: reecetech/transfer-action@v1
uses: reecetech/transfer-action@v2
with:
checkout: 'true'
direction: 'upload'
Expand All @@ -27,18 +27,18 @@ This action transfers data to/from artefact storage
- checkout
steps:
- name: Download repo
uses: reecetech/transfer-action@v1
uses: reecetech/transfer-action@v2
with:
checkout: 'true'
direction: 'download'
name: 'checkout'
```

### Upload specific build results

```yaml
- id: upload
name: Upload build output
uses: reecetech/transfer-action@v1
uses: reecetech/transfer-action@v2
with:
direction: 'upload'
name: 'build-result'
Expand All @@ -49,7 +49,7 @@ This action transfers data to/from artefact storage

```yaml
- name: Download build output
uses: reecetech/transfer-action@v1
uses: reecetech/transfer-action@v2
with:
direction: 'download'
name: 'build-result'
Expand All @@ -61,15 +61,15 @@ This action transfers data to/from artefact storage

<!-- AUTO-DOC-INPUT:START - Do not remove or modify this section -->

| INPUT | TYPE | REQUIRED | DEFAULT | DESCRIPTION |
|----------------------|--------|----------|-----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| checkout | string | false | `"false"` | Do a Git checkout before<br>transferring, either: `true` or `false`;<br>defaults to `false` |
| checkout-fetch-depth | string | false | `"1"` | Number of commits to fetch.<br>0 indicates all history for<br>all branches and tags |
| checkout-lfs | string | false | `"false"` | Whether to download Git-LFS files<br> |
| direction | string | true | | Direction of transfer, either: `upload`<br>or `download` |
| if-no-files-found | string | false | `"error"` | The desired behavior if no<br>files are found using the<br>provided path. Available Options: warn:<br>Output a warning but do<br>not fail the action error:<br>Fail the action with an<br>error message ignore: Do not<br>output any warnings or errors,<br>the action does not fail<br> |
| name | string | false | | Artifact name: defaults to `checkout`<br>if checkout input is `true`,<br>otherwise, please provide a value<br> |
| path | string | false | `"./"` | Defaults to `./` - additionally,<br>the meaning of `path` is:<br>if direction is `upload`: a<br>file, directory or wildcard pattern<br>that describes what to upload;<br>if direction is `download`: destination<br>path |
| INPUT | TYPE | REQUIRED | DEFAULT | DESCRIPTION |
|----------------------|--------|----------|-----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| checkout | string | false | `"false"` | Do a Git checkout before<br>transferring, either: `true` or `false`;<br>defaults to `false` |
| checkout-fetch-depth | string | false | `"1"` | Number of commits to fetch.<br>0 indicates all history for<br>all branches and tags |
| checkout-lfs | string | false | `"false"` | Whether to download Git-LFS files<br> |
| direction | string | true | | Direction of transfer, either: `upload`<br>or `download` |
| if-no-files-found | string | false | `"error"` | The desired behavior if no<br>files are found using the<br>provided path. Available Options: warn:<br>Output a warning but do<br>not fail the action error:<br>Fail the action with an<br>error message ignore: Do not<br>output any warnings or errors,<br>the action does not fail<br> |
| name | string | false | | Artifact name: defaults to `checkout`<br>if checkout input is `true`,<br>otherwise, please provide a value<br> |
| path | string | false | | ⚠️ Conditional defaults! - if<br>`checkout` is `true`; then defaults<br>to `./.git/` - if `checkout`<br>is `false`; then defaults to<br>`./` Additionally, the meaning of<br>`path` is: - if direction<br>is `upload`: a file, directory<br>or wildcard pattern that describes<br>what to upload; - if<br>direction is `download`: destination path<br> |

<!-- AUTO-DOC-INPUT:END -->

Expand Down
59 changes: 47 additions & 12 deletions action.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,13 @@ inputs:
description: 'Artifact name: defaults to `checkout` if checkout input is `true`, otherwise, please provide a value'
path:
description: |
Defaults to `./` - additionally, the meaning of `path` is:
if direction is `upload`: a file, directory or wildcard pattern that describes what to upload;
if direction is `download`: destination path
default: './'
⚠️ Conditional defaults!
- if `checkout` is `true`; then defaults to `./.git/`
- if `checkout` is `false`; then defaults to `./`

Additionally, the meaning of `path` is:
- if direction is `upload`: a file, directory or wildcard pattern that describes what to upload;
- if direction is `download`: destination path

outputs:
name:
Expand All @@ -53,9 +56,16 @@ runs:
CHECKOUT: ${{ inputs.checkout }}
DIRECTION: ${{ inputs.direction }}
NAME: ${{ inputs.name }}
PATH: ${{ inputs.path }}
LFS: ${{ inputs.checkout-lfs }}
run: |
set -euo pipefail

if [[ "${LFS}" != 'false' ]] ; then
echo "🛑 sorry, LFS functionality not yet implemented"
exit 255
fi

if [[ "${CHECKOUT}" != 'true' && "${CHECKOUT}" != 'false' ]] ; then
echo "🛑 checkout input set to '${CHECKOUT}', should be either 'true' or 'false'"
exit 1
Expand All @@ -77,28 +87,53 @@ runs:
fi
fi

- id: checkout
if: inputs.checkout == 'true'
name: Checkout repo
if [[ -n "${PATH:-}" ]] ; then
echo "::set-output name=path::${PATH}"
else
if [[ "${CHECKOUT}" == 'true' ]] ; then
echo "::set-output name=path::./.git/"
else
echo "::set-output name=path::./"
fi
fi

- id: clone
if: inputs.checkout == 'true' && inputs.direction == 'upload'
name: Clone repo ⧉
uses: actions/checkout@v3
with:
fetch-depth: ${{ inputs.checkout-fetch-depth }}
lfs: ${{ inputs.checkout-lfs }}

- id: prepare-dot-git
if: inputs.checkout == 'true' && inputs.direction == 'upload'
name: Remove index 🗑
shell: bash
run: |
set -euo pipefail
rm -f .git/index

- id: transfer-up
if: inputs.direction == 'upload'
name: Transfer data to GitHub
name: Transfer data to GitHub 🆙
uses: actions/upload-artifact@v3
with:
if-no-files-found: ${{ inputs.if-no-files-found }}
name: ${{ steps.verify.outputs.name }}
path: ${{ inputs.path }}
path: ${{ steps.verify.outputs.path }}
retention-days: 1

- id: transfer-down
if: inputs.direction == 'download'
name: Transfer data from GitHub
name: Transfer data from GitHub ⬇️
uses: actions/download-artifact@v3
with:
name: ${{ steps.verify.outputs.name }}
path: ${{ inputs.path }}
path: ${{ steps.verify.outputs.path }}

- id: checkout
if: inputs.checkout == 'true' && inputs.direction == 'download'
name: Checkout repo 🛒
shell: bash
run: |
set -euo pipefail
git checkout --progress --force -B "${{ github.ref_name }}" "${{ github.ref }}"