Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AWS CLI: Copy local file to remote bucket started failing #2791

Closed
1 of 7 tasks
Avaq opened this issue Feb 25, 2021 · 2 comments
Closed
1 of 7 tasks

AWS CLI: Copy local file to remote bucket started failing #2791

Avaq opened this issue Feb 25, 2021 · 2 comments
Assignees
@miketimofeev
Labels
Area: Scripting and command line investigate Collect additional information, like space on disk, other tool incompatibilities etc. OS: Ubuntu

Comments

@Avaq
Copy link

Avaq commented Feb 25, 2021 •
edited
Loading

Description

I have a GitHub action that runs something along the lines of the following command:

aws s3 cp ./local.zip 's3://remote-bucket'

This has been working for a long time, but started failing a few weeks ago (the last successful build was on Feb 1, 2021; the first failing build on Feb 8, 2021). The error message it fails with gives me the impression that it's a python dependency issue with a dependency installed for the awscli package.

Area for Triage:
Scripting and command line; Packages

Question, Bug, or Feature?:
Bug

Virtual environments affected

  • Ubuntu 16.04
  • Ubuntu 18.04
  • Ubuntu 20.04
  • macOS 10.15
  • macOS 11.0
  • Windows Server 2016 R2
  • Windows Server 2019

Image version
20210208.0

Expected behavior
The command succeeds and the AWS bucket now contains the zip file. The same command does work on my local machine, where I have used it with several versions of the awscli package.

Actual behavior
The command immediately exits with the following output:

<botocore.awsrequest.AWSRequest object at 0x7fb6fc4e6668>
Error: Process completed with exit code 255.

Repro steps
A description with steps to reproduce the issue. If your have a public example or repo to share,
please provide the link.

  1. Create a github workflow containing a job with a step with a run with aws s3 cp ./local.zip 's3://MY_BUCKET'
  2. Create local.zip: zip -FI local.zip <(echo 'hi')
  3. Commit and push it to GitHub
@miketimofeev miketimofeev added Area: Scripting and command line investigate Collect additional information, like space on disk, other tool incompatibilities etc. OS: Ubuntu and removed needs triage labels Feb 25, 2021
@miketimofeev miketimofeev self-assigned this Feb 25, 2021
@miketimofeev
Copy link
Contributor

Hi @Avaq!
I assume you use the ubuntu-latest image label, which is now being transitioned from Ubuntu-18 to Ubuntu-20 and thus from AWS cli v1 to v2 — #1816
For AWS cli v2 there is an issue when the region is not specified — aws/aws-cli#5623
Could you please try either specify the region or set AWS_EC2_METADATA_DISABLED=true environmental variable?

@Avaq
Copy link
Author

Avaq commented Feb 25, 2021

@miketimofeev Very helpful! Thank you. Setting the region fixed the issue. I should probably use an image label that doesn't cover multiple major versions to avoid confusion in the future. 😳

@Avaq Avaq closed this as completed Feb 25, 2021
@dphang dphang mentioned this issue Mar 4, 2021
Merged
eliezio added a commit to go-sari/sari-web that referenced this issue Apr 13, 2021
@eliezio
Add AWS region to "aws s3 cp"
df16e3c 
See actions/runner-images#2791 (comment)
sio added a commit to sio/gitlab-runners-fleet that referenced this issue Apr 7, 2023
@sio
Fix S3 upload with awscli v2
0cc0d17 
https://florian.ec/blog/github-actions-awscli-errors/
aws/aws-cli#5623
actions/runner-images#2791
@mbh329 mbh329 mentioned this issue Apr 21, 2023
Merged
vvarg229 added a commit to vvarg229/neofs-node that referenced this issue Jul 18, 2023
@vvarg229
Disable aws ec2 metadata
3239f5f 
Fixed bug nspcc-dev/neofs-s3-gw#798
This bug is only reproducible on ubuntu-latest in github actions.
This is a bug specifically of github acions, not the neofs-s3-gw:
actions/runner-images#2791
aws/aws-cli#5623

Signed-off-by: Oleg Kulachenko <[email protected]>
vvarg229 added a commit to vvarg229/neofs-node that referenced this issue Jul 18, 2023
@vvarg229
Disable aws ec2 metadata
4201d47 
Fixed bug nspcc-dev/neofs-s3-gw#798
This bug is only reproducible on ubuntu-latest in github actions.
This is a bug specifically of github acions, not the neofs-s3-gw:
actions/runner-images#2791
aws/aws-cli#5623

Signed-off-by: Oleg Kulachenko <[email protected]>
@vvarg229 vvarg229 mentioned this issue Jul 18, 2023
Closed
vvarg229 added a commit to vvarg229/neofs-testcases that referenced this issue Jul 19, 2023
@vvarg229
Disable aws ec2 metadata
ed80ed1 
Fixed bug nspcc-dev/neofs-s3-gw#798
This is a test environment error, not neofs-s3-gw:
actions/runner-images#2791
aws/aws-cli#5623

Signed-off-by: Oleg Kulachenko <[email protected]>
@vvarg229 vvarg229 mentioned this issue Jul 19, 2023
Merged
roman-khimov added a commit to nspcc-dev/neofs-testcases that referenced this issue Jul 19, 2023
@roman-khimov
Disable aws ec2 metadata (#586)
2ad7951 
Fixed bug nspcc-dev/neofs-s3-gw#798
This is a test environment error, not neofs-s3-gw:
actions/runner-images#2791
aws/aws-cli#5623

Tests results:

https://github.com/vvarg229/neofs-node/actions/runs/5601200752/jobs/10244709153

https://http.t5.fs.neo.org/86C4P6uJC7gb5n3KkwEGpXRfdczubXyRNW5N9KeJRW73/216-1689781099/index.html#
vvarg229 added a commit to vvarg229/neofs-testcases that referenced this issue Jul 31, 2023
@vvarg229
Disable aws ec2 metadata
b4b1914 
Fixed bug nspcc-dev/neofs-s3-gw#798
This is a test environment error, not neofs-s3-gw:
actions/runner-images#2791
aws/aws-cli#5623

Signed-off-by: Oleg Kulachenko <[email protected]>
joverlee521 added a commit to nextstrain/docker-base that referenced this issue Jun 6, 2024
@joverlee521
tests/envdir-url: Add AWS_DEFAULT_REGION envvar
9b233f9 
AWS CLI v2 runs into errors if the region is not defined¹

The AWS_DEFAULT_REGION variable is available in the GH Action CI after
the aws-actions/configure-aws-credentials@v4 step.

¹ <actions/runner-images#2791 (comment)>
joverlee521 added a commit to nextstrain/docker-base that referenced this issue Jun 6, 2024
@joverlee521
tests/envdir-url: Add AWS_DEFAULT_REGION envvar
1591ad7 
AWS CLI v2 runs into errors if the region is not defined¹

The AWS_DEFAULT_REGION variable is available in the GH Action CI after
the aws-actions/configure-aws-credentials@v4 step.

¹ <actions/runner-images#2791 (comment)>
xquery-go added a commit to xquery-go/gitlab-fleet that referenced this issue Nov 27, 2024
@xquery-go
Fix S3 upload with awscli v2
f6a5e5f 
https://florian.ec/blog/github-actions-awscli-errors/
aws/aws-cli#5623
actions/runner-images#2791
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@miketimofeev miketimofeev

Labels
Area: Scripting and command line investigate Collect additional information, like space on disk, other tool incompatibilities etc. OS: Ubuntu
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Avaq @miketimofeev