Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

curvefs/metaserver: recover s3ChunkInfoRemove field for GetOrModifyS3… #1404

Merged
merged 1 commit into from
May 14, 2022
Merged

curvefs/metaserver: recover s3ChunkInfoRemove field for GetOrModifyS3… #1404

merged 1 commit into from
May 14, 2022

Conversation

Wine93
Copy link
Contributor

@Wine93 Wine93 commented May 7, 2022

What problem does this PR solve?

Issue Number: #1304, #1308
Problem Summary:

  • guarantee consistent of s3chunkinfo in inode
  • recover s3ChunkInfoRemove field for GetOrModifyS3ChunkInfo PRC request

}

auto list2del = item.second;
S3ChunkInfoList dummy;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's wired that you add a empty list, even now it's correct because ModifyInodeS3ChunkInfoList will just return ok if list is empty.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's wired that you add a empty list, even now it's correct because ModifyInodeS3ChunkInfoList will just return ok if list is empty.

yep, i will use pointer to improve it.

std::unordered_map<uint64_t, bool> deleted;
for (const auto& item : map2add) {
uint64_t chunkIndex = item.first;
auto list2add = item.second;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

avoid meaningless copy, const auto& list2add

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

avoid meaningless copy, const auto& list2add

yep, i will improve it.

for (const auto& item : map2add) {
uint64_t chunkIndex = item.first;
auto list2add = item.second;
S3ChunkInfoList list2del;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto, use a pointer

const S3ChunkInfoList* list2del = nullptr;

line 333:   list2der = &iter->second;

after line 341: line2del = nullptr;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto, use a pointer

const S3ChunkInfoList* list2del = nullptr;

line 333:   list2der = &iter->second;

after line 341: line2del = nullptr;

fixed.

continue;
}

auto list2del = item.second;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

fixed.

if (rc != MetaStatusCode::OK) {
return rc;
}
std::unordered_map<uint64_t, bool> deleted;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to use a map in here, a set is enough, and just store the chunk index that has been deleted.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to use a map in here, a set is enough, and just store the chunk index that has been deleted.

yep, i will improve it.

Comment on lines 262 to 263
bool succ = SplitS3ChunkInfoList(
iterator->Value(), delFirstChunkId, &list);
Copy link
Contributor

@wu-hanqing wu-hanqing May 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why pass an empty list?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why pass an empty list?

fixed.


for (const auto& item : map2del) {
uint64_t chunkIndex = item.first;
if (deleted[chunkIndex]) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deleted[chunkIndex] will add an element (key:chunkIndex, value:false)to map, which it's strange there...
using deleted.count(chunkIndex) instead

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deleted[chunkIndex] will add an element (key:chunkIndex, value:false)to map, which it's strange there... using deleted.count(chunkIndex) instead

yep, i will improve it.

std::unordered_map<uint64_t, bool> deleted;
for (const auto& item : map2add) {
uint64_t chunkIndex = item.first;
auto list2add = item.second;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const auto& ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const auto& ?

fixed.

continue;
}

auto list2del = item.second;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const auto&?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const auto&?

fixed, i will use pointer to avoid memory copying.

bool returnS3ChunkInfoMap,
bool compaction);
std::shared_ptr<Iterator>* iterator);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"iterator" should be more meaningful name. it's too generic to understand

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"iterator" should be more meaningful name. it's too generic to understand

fixed.

// firstChunkId < minChunkId
key2del.push_back(skey);
*size4del += key.size;
// delete list range : [ ]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic need change maybe

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic need change maybe

I adjusted the logic according based on our discussion.

@Wine93
Copy link
Contributor Author

Wine93 commented May 13, 2022

recheck

…ChunkInfo PRC request

and guarantee consistent of s3chunkinfo in inode. (#1304) (#1308)
@Wine93 Wine93 merged commit e3aef89 into opencurve:master May 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants