Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync doesn't allow whitspaces in --cache-control #541

Closed
7rulnik opened this issue Jan 12, 2023 · 0 comments · Fixed by #636
Closed

Sync doesn't allow whitspaces in --cache-control #541

7rulnik opened this issue Jan 12, 2023 · 0 comments · Fixed by #636
Labels

Comments

@7rulnik
Copy link

7rulnik commented Jan 12, 2023

I guess this is a problem with parsing and passing --cache-control option.

This example doesn't work:

s5cmd --dry-run sync --cache-control 'public, max-age=31536000, immutable' ./dist/selene-static/ s3://selene-static/selene-static-s5cmd/

But this one does:

s5cmd --dry-run sync --cache-control 'public,max-age=31536000,immutable' ./dist/selene-static/ s3://selene-static/selene-static-s5cmd/

And for cp command both variants work

sonmezonur pushed a commit that referenced this issue Aug 7, 2023
Resolves #541

While running the sync command it starts to generate cp commands in the
generateCommand function and writes it to a pipe where it is read by the
Run function of the Run command. In the Run function, there is a line

`fields, err := shellquote.Split(line)`

to split the generated cp command into its fields.



 For a command



 
```
 
s5cmd sync --cache-control ‘public, max-age=31536000, immutable’ /Users/ataberk/desktop/test s3://s5cmd-test2
```

We would have expected the generateCommand function to write into the
pipe a command



```
cp --cache-control='public, max-age=31536000, immutable' --raw='true' \"/Users/ataberk/desktop/test/hello.txt\" \"s3://s5cmd-test2/test/hello.txt\"
```

But instead, it writes the command



```
cp --cache-control=public, max-age=31536000, immutable --raw=true \"/Users/ataberk/desktop/test/hello.txt\" \"s3://s5cmd-test2/test/hello.txt\"
```

![Screenshot 2023-08-05 at 13 45
52](https://github.com/peak/s5cmd/assets/30214288/da4db89a-49c2-41ca-9020-89d12484e72c)

This causes the `shellquote.Split(line)` to split fields incorrectly.
This causes `flagset.Parse(fields)` to populate flagset incorrectly and
as a result we end up with an error.

![Screenshot 2023-08-05 at 05 53
43](https://github.com/peak/s5cmd/assets/30214288/6777a474-a220-4b95-b158-16e2848402b5)

The main problem occurs in the generateCommand function while appending
flags without quotes. If we add quotes around the flagValue while
generating a command. The `shellquote.Split(line)` acts as intended and
successfully sync with the given cache-control metadata in S3

![Screenshot 2023-08-05 at 13 58
28](https://github.com/peak/s5cmd/assets/30214288/534aaa13-31e9-46bc-959f-85506d66917d)

![Screenshot 2023-08-05 at 13 59
31](https://github.com/peak/s5cmd/assets/30214288/64817fa7-7467-4046-85d3-f3e694426f63)

Co-authored-by: Ataberk Gürel <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants