Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 publishing endpoint appears to reupload everything #619

Closed
MMartyn opened this issue Aug 15, 2017 · 1 comment · Fixed by #622
Closed

S3 publishing endpoint appears to reupload everything #619

MMartyn opened this issue Aug 15, 2017 · 1 comment · Fixed by #622
Labels

Comments

@MMartyn
Copy link

MMartyn commented Aug 15, 2017

I am investigating slowness around publishing new snapshots to s3. I turned on debugging and the ListObjects call appeared to double up on the prefix ie
GET /?max-keys=1000&prefix={prefix}%2F{prefix}%2F.

As far as I can tell I have the s3 endpoint configured correctly and I am not sending a prefix in the command but do have the prefix configured in the endpoint.

Unless I am reading the code incorrectly it looks like that is happening as a result of these 2 lines:
https://github.com/smira/aptly/blob/master/s3/public.go#L270
https://github.com/smira/aptly/blob/master/s3/public.go#L325

@smira
Copy link
Contributor

smira commented Aug 15, 2017

thanks, nice catch!

there's no error but it gets empty listing and decides to re-upload everything :(

smira added a commit that referenced this issue Aug 16, 2017
Original PR: #621
Fixes: #619

I've added unit-test to Martyn's PR.

Without this fix, if `prefix` is set on S3 publish endpoint,
aptly would incorrectly build path cache and re-upload every object
on publish.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants