Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replace BucketReader.ObjectSize() with .Attributes() #2613

Merged
merged 6 commits into from
May 21, 2020
Merged

Replace BucketReader.ObjectSize() with .Attributes() #2613

merged 6 commits into from
May 21, 2020

Conversation

pracucci
Copy link
Contributor

@pracucci pracucci commented May 15, 2020 •
edited
Loading

  • I added CHANGELOG entry for this change.
  • Change is not relevant to the end user.

Changes

As first step to address #2585, in this PR I've replaced BucketReader.ObjectSize() with Attributes() in order to also be able to read the object last modified timestamp via the BucketReader.

/cc @pstibrany

Verification

Unit tests.

povilasv
povilasv approved these changes May 16, 2020
View reviewed changes
Copy link
Member

@povilasv povilasv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

pkg/store/cache/caching_bucket.go Outdated Show resolved Hide resolved
@pracucci pracucci force-pushed the add-attributes-to-objstore-clients branch from 4ff388f to 39b249d Compare May 18, 2020 12:50
@pstibrany pstibrany mentioned this pull request May 18, 2020
Merged
2 tasks
pstibrany
pstibrany approved these changes May 18, 2020
View reviewed changes
Copy link
Contributor

@pstibrany pstibrany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

bwplotka
bwplotka approved these changes May 18, 2020
View reviewed changes
Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing! Tiny comments only and LGTM (: 💪

pkg/objstore/clientutil/parse.go Outdated Show resolved Hide resolved
pkg/objstore/clientutil/parse.go Outdated Show resolved Hide resolved
pkg/objstore/clientutil/parse.go Show resolved Hide resolved
pkg/objstore/inmem.go Show resolved Hide resolved
povilasv
povilasv reviewed May 19, 2020
View reviewed changes
pkg/objstore/objstore.go Show resolved Hide resolved
pkg/objstore/inmem.go Show resolved Hide resolved
pkg/block/indexheader/binary_reader.go Show resolved Hide resolved
GiedriusS
GiedriusS reviewed May 20, 2020
View reviewed changes
docs/components/store.md Outdated Show resolved Hide resolved
pkg/objstore/inmem.go Show resolved Hide resolved
GiedriusS
GiedriusS reviewed May 20, 2020
View reviewed changes
Copy link
Member

@GiedriusS GiedriusS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good modulo the current comments/suggestions

@pracucci
Added Attributes() to BucketReader
65aa0d7 
Signed-off-by: Marco Pracucci <[email protected]>
@pracucci
Removed ObjectSize() from BucketReader
784efdd 
Signed-off-by: Marco Pracucci <[email protected]>
@pracucci
Added copyright header
0be28e2 
Signed-off-by: Marco Pracucci <[email protected]>
@pracucci
Fixes after rebase
e8b17f5 
Signed-off-by: Marco Pracucci <[email protected]>
@pracucci
Fixed config
e7d0d6d 
Signed-off-by: Marco Pracucci <[email protected]>
@pracucci
Addressed review comments
001bbf7 
Signed-off-by: Marco Pracucci <[email protected]>
@pracucci pracucci force-pushed the add-attributes-to-objstore-clients branch from c95fd62 to 001bbf7 Compare May 21, 2020 07:54
@pracucci pracucci merged commit 48c80d1 into thanos-io:master May 21, 2020
@pracucci pracucci deleted the add-attributes-to-objstore-clients branch May 21, 2020 08:38
@pstibrany
Copy link
Contributor

pstibrany commented Jun 30, 2020 •
edited
Loading

This PR has changed chunk_object_size_ttl (Thanos 0.13) to chunk_object_attrs_ttl (on master) in cache config, which is a user-visible change, not mentioned in the changelog.

@pracucci
Copy link
Contributor Author

This PR has changed chunk_object_size_ttl (Thanos 0.13) to chunk_object_attrs_ttl (on master) in cache config, which is a user-visible change, not mentioned in the changelog.

Thanks for the heads up. I'm going to open a PR to fix it.

@pracucci pracucci mentioned this pull request Jun 30, 2020
Merged
2 tasks
@SabaPing
Copy link
Contributor

SabaPing commented Jul 21, 2020 •
edited
Loading

The method clientutil.ParseLastModified failed when called by oss.Attributes.

Aliyun oss api returns Last-Modified header in RFC1123 format. But the method clientutil.ParseLastModified parses all timestamp in RFC3339 format.

See oss api doc
image

This bug cause the store fails to load buckets from aliyun oss.

Could I submit a PR to fix this?

Thanks.

@bwplotka
Copy link
Member

Yes please @SabaPing ❤️

@SabaPing SabaPing mentioned this pull request Jul 21, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@GiedriusS GiedriusS GiedriusS left review comments

@pstibrany pstibrany pstibrany approved these changes

@bwplotka bwplotka bwplotka approved these changes

@povilasv povilasv povilasv approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@pracucci @pstibrany @SabaPing @bwplotka @GiedriusS @povilasv