Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adjust meta client ctor && implement heartbeat logic inside. #383

Merged
merged 3 commits into from
May 20, 2019

Conversation

dangleptr
Copy link
Contributor

@dangleptr dangleptr commented May 15, 2019

  1. Force meta client to reuse IOThreadPool outside.
  2. Move meta_server_addrs flag outside, so we don't check the flag in MetaClient ctor any more.
  3. Implement heartbeat logic.

Subtask of #283

After the repo to be public, changes could NOT be rebased on the original pr #379
So Let's use the new one to track it.

@nebula-community-bot
Copy link
Member

Unit testing passed.

1 similar comment
@nebula-community-bot
Copy link
Member

Unit testing passed.

<< ", status:" << result.status();
return EXIT_FAILURE;
}
uint32_t localIP;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a log to print localIP?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point.


if (FLAGS_meta_server_addrs.empty()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a regular to check format of FLAGS_meta_server_addrs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good proposal. We could check the format of server_address inside NetWorkUtils::toHosts

Copy link
Contributor

@dutor dutor May 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI. gflags supports some kind of validator that will be checked against whenever a specific flag is being modified. We could utilize that at the first place for most of our flags. We could do that in future.

@@ -22,7 +24,11 @@ ExecutionEngine::~ExecutionEngine() {


Status ExecutionEngine::init(std::shared_ptr<folly::IOThreadPoolExecutor> ioExecutor) {
metaClient_ = std::make_unique<meta::MetaClient>();
if (FLAGS_meta_server_addrs.empty()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

@nebula-community-bot
Copy link
Member

Unit testing passed.


auto metaClient = std::make_unique<nebula::meta::MetaClient>();
auto ioThreadPool = std::make_shared<folly::IOThreadPoolExecutor>(FLAGS_io_handlers);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could share the same thread pool with the one in ThriftServer

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have done it. Please note line 151

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent.

@@ -24,7 +26,7 @@ using nebula::cpp2::ValueType;
using apache::thrift::FragileConstructor::FRAGILE;

TEST(MetaClientTest, InterfacesTest) {
FLAGS_load_data_interval_second = 1;
FLAGS_load_data_interval_secs = 1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As for these tests, I think there are two ways to make them faster. First, we could have shared the same MockServer instead of launch one per test. Second, seconds is appropriate for a polling interval, but too long for unit testing, while I cannot come up with a better way. But you still could impove it by usleep(FLAGS_load_data_interval_secs * 1000000 + 100000/*100ms*/) to save 0.9s per sleep.

Of course, we could leave these work to the next PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good

@dangleptr
Copy link
Contributor Author

Jenkins go

@nebula-community-bot
Copy link
Member

Unit testing passed.

dutor
dutor previously approved these changes May 17, 2019
dutor
dutor previously approved these changes May 17, 2019
darionyaphet
darionyaphet previously approved these changes May 17, 2019
@dangleptr
Copy link
Contributor Author

Jenkins go

@nebula-community-bot
Copy link
Member

Unit testing failed.

@dangleptr dangleptr dismissed stale reviews from darionyaphet and dutor via c8bd872 May 17, 2019 08:40
@dangleptr
Copy link
Contributor Author

Damn it! Conflicts again.

@dangleptr
Copy link
Contributor Author

Jenkins go

@nebula-community-bot
Copy link
Member

Unit testing failed.

1 similar comment
@nebula-community-bot
Copy link
Member

Unit testing failed.

@dangleptr
Copy link
Contributor Author

Jenkins go

@nebula-community-bot
Copy link
Member

Unit testing failed.

@nebula-community-bot
Copy link
Member

Unit testing passed.

Copy link
Contributor

@laura-ding laura-ding left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@laura-ding laura-ding merged commit afb907a into vesoft-inc:master May 20, 2019
@dangleptr dangleptr deleted the hb3 branch May 20, 2019 04:38
yixinglu pushed a commit to yixinglu/nebula that referenced this pull request Feb 16, 2020
…inc#383)

* Adjust meta client ctor  && implement heartbeat logic inside.

* Address laura-ding's comments

* Fix conflicts
tong-hao pushed a commit to tong-hao/nebula that referenced this pull request Jun 1, 2021
…inc#383)

* Adjust meta client ctor  && implement heartbeat logic inside.

* Address laura-ding's comments

* Fix conflicts
yixinglu pushed a commit to yixinglu/nebula that referenced this pull request Mar 21, 2022
* add cachelib and storage cache class

include new files

change cmakelist

update thrift

fix build

fix build

fix build

add constructor for struct

clean and add test

add cmake related

fix test

trying cmake

add more test and cmake change

cmake trying out failed and revert
fix test bug

now fix cmake
add a test

fix poolname bug
delete a test

add lock

update conf

* update format and change name

* fix typo in cmake

* remove unneeded lib in cmake

* fix cmake

* fix comment
change to use & in put

* add some log and update signature of functions in storage cache

* add function getting cache hit count

* update config

* update config

* change the vertex pool ttl default value

* format

* fix typo

* change to use error code

* fix typo

* typo

* update error code value

* add lrt

Co-authored-by: Doodle <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-for-testing PR: ready for the CI test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants