enhance: Decouple shard client manager from shard cache #37371
Conversation
mergify
bot
added
dco-passed
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #37371 +/- ##
===========================================
- Coverage 80.58% 68.13% -12.46%
===========================================
Files 1356 290 -1066
Lines 190009 25395 -164614
===========================================
- Hits 153128 17302 -135826
+ Misses 31475 8093 -23382
+ Partials 5406 0 -5406
|
|
@weiliu1031 E2e jenkins job failed, comment |
weiliu1031
force-pushed
the
decouple_shard_client
branch
from
70bc109 to
5d498f1
Compare
|
@weiliu1031 go-sdk check failed, comment |
|
@weiliu1031 cpp-unit-test check failed, comment |
|
@weiliu1031 E2e jenkins job failed, comment |
|
/run-cpu-e2e |
|
@weiliu1031 E2e jenkins job failed, comment |
|
/run-cpu-e2e |
|
@weiliu1031 E2e jenkins job failed, comment |
weiliu1031
force-pushed
the
decouple_shard_client
branch
from
5d498f1 to
f976f00
Compare
|
@weiliu1031 go-sdk check failed, comment |
|
@weiliu1031 cpp-unit-test check failed, comment |
|
rerun go-sdk |
|
@weiliu1031 E2e jenkins job failed, comment |
weiliu1031
force-pushed
the
decouple_shard_client
branch
from
f976f00 to
86d7446
Compare
|
@weiliu1031 cpp-unit-test check failed, comment |
|
@weiliu1031 E2e jenkins job failed, comment |
|
@weiliu1031 go-sdk check failed, comment |
weiliu1031
force-pushed
the
decouple_shard_client
branch
from
86d7446 to
2239ddb
Compare
|
@weiliu1031 E2e jenkins job failed, comment |
|
/run-cpu-e2e |
the old implementation update shard cache and shard client manager at same time, which causes lots of conor case due to concurrent issue without lock. This PR decouple shard client manager from shard cache, so only shard cache will be updated if delegator changes. and make sure shard client manager will always return the right client, and create a new client if not exist. in case of client leak, shard client manager will purge client in async for every 10 minutes. Signed-off-by: Wei Liu <[email protected]>
weiliu1031
force-pushed
the
decouple_shard_client
branch
from
2239ddb to
557b161
Compare
|
@weiliu1031 go-sdk check failed, comment |
|
@weiliu1031 E2e jenkins job failed, comment |
Signed-off-by: Wei Liu <[email protected]>
weiliu1031
force-pushed
the
decouple_shard_client
branch
from
557b161 to
ead8b31
Compare
Signed-off-by: Wei Liu <[email protected]>
|
@weiliu1031 E2e jenkins job failed, comment |
|
/run-cpu-e2e |
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: czs007, weiliu1031 The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Related to previous pr: milvus-io#37371 Signed-off-by: Congqi Xia <[email protected]>
Related to previous pr: #37371 Signed-off-by: Congqi Xia <[email protected]>
) issue: milvus-io#37115 the old implementation update shard cache and shard client manager at same time, which causes lots of conor case due to concurrent issue without lock. This PR decouple shard client manager from shard cache, so only shard cache will be updated if delegator changes. and make sure shard client manager will always return the right client, and create a new client if not exist. in case of client leak, shard client manager will purge client in async for every 10 minutes. --------- Signed-off-by: Wei Liu <[email protected]>
…s-io#37646) Related to previous pr: milvus-io#37371 Signed-off-by: Congqi Xia <[email protected]> Signed-off-by: Wei Liu <[email protected]>
) issue: #37115 pr: #37371 #37646 #37729 the old implementation update shard cache and shard client manager at same time, which causes lots of conor case due to concurrent issue without lock. This PR decouple shard client manager from shard cache, so only shard cache will be updated if delegator changes. and make sure shard client manager will always return the right client, and create a new client if not exist. in case of client leak, shard client manager will purge client in async for every 10 minutes. --------- --------- Signed-off-by: Wei Liu <[email protected]> Signed-off-by: Congqi Xia <[email protected]> Co-authored-by: congqixia <[email protected]>
issue: #37115
the old implementation update shard cache and shard client manager at same time, which causes lots of conor case due to concurrent issue without lock.
This PR decouple shard client manager from shard cache, so only shard cache will be updated if delegator changes. and make sure shard client manager will always return the right client, and create a new client if not exist. in case of client leak, shard client manager will purge client in async for every 10 minutes.