Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hugegraph-tool 备份请教 #342

Closed
alegjk opened this issue Jan 24, 2019 · 5 comments
Closed

hugegraph-tool 备份请教 #342

alegjk opened this issue Jan 24, 2019 · 5 comments
Labels

Comments

@alegjk
Copy link

alegjk commented Jan 24, 2019

image

hugegraph-tools执行备份操作的时候, 异常, 提示顶点/边的数目过大

Caused by: com.baidu.hugegraph.rest.ClientException: Exception occurred while backing up edges at com.baidu.hugegraph.base.RetryManager.retry(RetryManager.java:76) at com.baidu.hugegraph.manager.BackupManager.lambda$backupEdges$3(BackupManager.java:117) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: class com.baidu.hugegraph.exception.LimitExceedException: Too many records(must <=800000) for the query: Query for EDGE_OUT offset=0, limit=9223372036854775807, order by {} where id in [] and [ID scan Shard{start=2317021520, end=2322672792, length=0}] at com.baidu.hugegraph.exception.ServerException.fromResponse(ServerException.java:44) at com.baidu.hugegraph.client.RestClient.checkStatus(RestClient.java:63) at com.baidu.hugegraph.rest.RestClient.get(RestClient.java:169) at com.baidu.hugegraph.api.traverser.EdgesAPI.scan(EdgesAPI.java:66) at com.baidu.hugegraph.driver.TraverserManager.edges(TraverserManager.java:189) at com.baidu.hugegraph.manager.BackupManager.lambda$null$2(BackupManager.java:118) at com.baidu.hugegraph.base.RetryManager.retry(RetryManager.java:73) ... 6 more
https://hugegraph.github.io/hugegraph-doc/quickstart/hugegraph-tools.html
上以及命令行上的操作提示都并未提及这一限制, 请问如果想要做全量数据备份, 应该如何处理?
我的操作命令如下:
bin/hugegraph backup -t all -d /data/hugegraph_backup
顶点在400W, 边在2000万左右

@zhoney
Copy link
Contributor

zhoney commented Feb 19, 2019

导致原因是一个shard中的数据量大于80w导致的。

hugegraph-0.9版本中shard的scan API将支持paging #360 ,对应的client和tools中都进行了支持,解决了单个shard数据量大以及超级顶点的backup问题。

scan with paging的代码已提交,还未合入,参见issue #360

@alegjk
Copy link
Author

alegjk commented Feb 20, 2019

请问HugeGraph 0.9什么时候发布?

@zhoney
Copy link
Contributor

zhoney commented Feb 25, 2019

@hklmuaqmn 预计3月下旬发布

@suesunss
Copy link

@zhoney 请问 hugegraph 分布式方案年内有计划吗?

@github-actions
Copy link

Due to the lack of activity, the current issue is marked as stale and will be closed after 20 days, any update will remove the stale label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants