ft: ZENKO-433 add item count support incremental update and refresh #499

alexanderchan-scality · 2018-06-07T01:28:38Z

Old version: #497

bert-e · 2018-06-07T01:28:39Z

Hello alexanderchan-scality,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get
information on this process.

Status report is not available.

bert-e · 2018-06-07T01:28:41Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

2 peers

ironman-machine · 2018-06-07T08:37:02Z

PR has been updated. Reviewers, please be cautious.

alexanderchan-scality · 2018-06-09T00:07:23Z

@bert-e help

bert-e · 2018-06-09T00:07:24Z

Help page

The following options and commands are available at this time.

Options

name	description	privileged
➡️ bypass_tester_approval	Bypass the pull request testers' approval	⭐
➡️ wait	Instruct Bert-E not to run until further notice.
➡️ no_octopus	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
➡️ bypass_commit_size	Bypass the check on the size of the changeset `TBA`	⭐
➡️ bypass_author_approval	Bypass the pull request author's approval	⭐
➡️ bypass_build_status	Bypass the build and test status	⭐
➡️ bypass_peer_approval	Bypass the pull request peers' approval	⭐
➡️ create_pull_requests	Allow the creation of integration pull requests.
➡️ after_pull_request	Wait for the given pull request id to be merged before continuing with the current one.
➡️ bypass_jira_check	Bypass the Jira issue check	⭐
➡️ bypass_incompatible_branch	Bypass the check on the source branch prefix	⭐
➡️ unanimity	Change review acceptance criteria from `one reviewer at least` to `all reviewers`

Commands

name	description	privileged
➡️ retry	Re-start a fresh build `TBA`
➡️ force_reset	Delete integration branches & pull requests, and restart merge process from the beginning.
➡️ help	Print Bert-E's manual in the pull request.
➡️ clear	Remove all comments from Bert-E from the history `TBA`
➡️ status	Print Bert-E's current status in the pull request `TBA`
➡️ reset	Try to remove integration branches unless there are commits on them which do not appear on the source branch.
➡️ build	Re-start a fresh build `TBA`

bert-e · 2018-06-09T00:07:27Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

2 peers

Do it again human slave!:point_right: :runner: (Oh and the pull request has been updated, by the way.)

ironman-machine · 2018-06-20T16:56:53Z

PR has been updated. Reviewers, please be cautious.

ironman-machine · 2018-06-20T17:01:13Z

PR has been updated. Reviewers, please be cautious.

ploki · 2018-06-20T17:26:38Z

lib/storage/metadata/mongoclient/DataCounter.js

+const PREV = 'prev';
+
+function deepCopyObject(obj) {
+    return JSON.parse(JSON.stringify(obj));


that looks costly, is it the best practice when it comes to deep copy of objects attributes without all the methods?

Suprising as it is, the native JSON parser is pretty efficient, more efficient than recursively iterating over object properties and array elements (which creates temp JS-side objects in the process).

ploki · 2018-06-20T17:33:46Z

lib/storage/metadata/mongoclient/DataCounter.js

+            this.objects = setVal.objects;
+            this.versions = setVal.versions;
+            this.buckets = setVal.buckets;
+            this.bucketList = [...setVal.bucketList];


what is the benefit of the spread syntax instead of a direct affectation of this.bucketList? deep copy?

It's a shallow copy and is equivalent to loop-pushing into an empty new array. Since this is setting a new counter snapshot it allows being "consistent" and not have a dangling reference to an array that can still be pushed into by who knows what callback is in the queue.

ploki · 2018-06-20T17:49:21Z

lib/storage/metadata/mongoclient/MongoClientInterface.js

+        const { objName, versionId } = params;
+        if (isUserBucket(collectionName)) {
+            if (isUpdate) {
+                return c.find({


What is the complexity of this find operation? it looks like it perform a database request.

It helps maintain counters accurate (because the update can have a different size than the version it's overwriting) and does a get on a version id. It's either this with full-scan every hour or full-scan way more often. So PUT/DELETE operations on versioned buckets go from 1 read + 1 write to 2 read + 1 write.

bert-e · 2018-06-20T22:05:51Z

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

✔️ development/8.0

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of Release Engineering will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

Any commit you add on the source branch will trigger a new cycle after the
current queue is merged.
Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of Release Engineering now.

bert-e · 2018-06-20T22:09:23Z

I have successfully merged the changeset of this pull request
into targetted development branches:

✔️ development/8.0

Please check the status of the associated issue ZENKO-433.

Goodbye alexanderchan-scality.

alexanderchan-scality mentioned this pull request Jun 7, 2018

ft: ZENKO-433 add item count support incremental update and refresh #497

Closed

alexanderchan-scality force-pushed the feature/ZENKO-433/countIncUpdateandRefresh branch from dd6da47 to 13e4f2d Compare June 7, 2018 08:37

philipyoo previously approved these changes Jun 8, 2018

View reviewed changes

dora-korpar previously approved these changes Jun 12, 2018

View reviewed changes

ploki self-requested a review June 12, 2018 23:17

alexanderchan-scality force-pushed the feature/ZENKO-433/countIncUpdateandRefresh branch from 13e4f2d to 70c7349 Compare June 20, 2018 16:56

ft: ZENKO-433 add item count support incremental update and refresh

2ce9db4

alexanderchan-scality force-pushed the feature/ZENKO-433/countIncUpdateandRefresh branch from 70c7349 to 2ce9db4 Compare June 20, 2018 17:01

ploki reviewed Jun 20, 2018

View reviewed changes

ploki approved these changes Jun 20, 2018

View reviewed changes

philipyoo approved these changes Jun 20, 2018

View reviewed changes

scality deleted a comment from alexanderchan-scality Jun 20, 2018

bert-e merged commit 2ce9db4 into development/8.0 Jun 20, 2018

alexanderchan-scality deleted the feature/ZENKO-433/countIncUpdateandRefresh branch June 27, 2018 20:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ft: ZENKO-433 add item count support incremental update and refresh #499

ft: ZENKO-433 add item count support incremental update and refresh #499

alexanderchan-scality commented Jun 7, 2018 •

edited

Loading

bert-e commented Jun 7, 2018

bert-e commented Jun 7, 2018

ironman-machine commented Jun 7, 2018

alexanderchan-scality commented Jun 9, 2018

bert-e commented Jun 9, 2018

bert-e commented Jun 9, 2018

ironman-machine commented Jun 20, 2018

ironman-machine commented Jun 20, 2018

ploki Jun 20, 2018

rachedbenmustapha Jun 20, 2018

ploki Jun 20, 2018

rachedbenmustapha Jun 20, 2018

ploki Jun 20, 2018

rachedbenmustapha Jun 20, 2018

bert-e commented Jun 20, 2018

bert-e commented Jun 20, 2018

ft: ZENKO-433 add item count support incremental update and refresh #499

ft: ZENKO-433 add item count support incremental update and refresh #499

Conversation

alexanderchan-scality commented Jun 7, 2018 • edited Loading

bert-e commented Jun 7, 2018

Hello alexanderchan-scality,

bert-e commented Jun 7, 2018

Waiting for approval

ironman-machine commented Jun 7, 2018

alexanderchan-scality commented Jun 9, 2018

bert-e commented Jun 9, 2018

Help page

Options

Commands

bert-e commented Jun 9, 2018

Waiting for approval

ironman-machine commented Jun 20, 2018

ironman-machine commented Jun 20, 2018

ploki Jun 20, 2018

Choose a reason for hiding this comment

rachedbenmustapha Jun 20, 2018

Choose a reason for hiding this comment

ploki Jun 20, 2018

Choose a reason for hiding this comment

rachedbenmustapha Jun 20, 2018

Choose a reason for hiding this comment

ploki Jun 20, 2018

Choose a reason for hiding this comment

rachedbenmustapha Jun 20, 2018

Choose a reason for hiding this comment

bert-e commented Jun 20, 2018

In the queue

bert-e commented Jun 20, 2018

alexanderchan-scality commented Jun 7, 2018 •

edited

Loading