Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

enhance: optimize some cache to reduce memory usage #33534

Merged
merged 1 commit into from
Jun 4, 2024
Merged

enhance: optimize some cache to reduce memory usage #33534

merged 1 commit into from
Jun 4, 2024

Conversation

zhagnlu
Copy link
Contributor

@zhagnlu zhagnlu commented Jun 1, 2024

@sre-ci-robot sre-ci-robot added the size/L Denotes a PR that changes 100-499 lines. label Jun 1, 2024
@mergify mergify bot added dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement labels Jun 1, 2024
@zhagnlu zhagnlu force-pushed the optimize_memory branch from 2b4c2c9 to bd65aac Compare June 1, 2024 14:44
@mergify Mergify
Copy link
Contributor

mergify bot commented Jun 1, 2024

@zhagnlu E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@codecov Codecov
Copy link

codecov bot commented Jun 1, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 76.92308% with 3 lines in your changes missing coverage. Please review.

Project coverage is 82.02%. Comparing base (0382628) to head (3e5eb5b).
Report is 11 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #33534      +/-   ##
==========================================
+ Coverage   81.94%   82.02%   +0.07%     
==========================================
  Files        1014     1006       -8     
  Lines      130109   130290     +181     
==========================================
+ Hits       106624   106870     +246     
+ Misses      19499    19437      -62     
+ Partials     3986     3983       -3
Files Coverage Δ
internal/core/src/mmap/Column.h 90.52% <100.00%> (-0.52%) ⬇️
internal/core/src/segcore/InsertRecord.h 90.94% <100.00%> (+0.11%) ⬆️
internal/core/src/segcore/SegmentInterface.h 100.00% <ø> (ø)
internal/core/src/segcore/SegmentGrowingImpl.cpp 74.23% <0.00%> (+0.12%) ⬆️
internal/core/src/segcore/SegmentSealedImpl.cpp 77.66% <71.42%> (+0.54%) ⬆️

... and 220 files with indirect coverage changes

@zhagnlu
Copy link
Contributor Author

zhagnlu commented Jun 1, 2024

/run-cpu-e2e

@mergify Mergify
Copy link
Contributor

mergify bot commented Jun 2, 2024

@zhagnlu E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@zhagnlu
Copy link
Contributor Author

zhagnlu commented Jun 2, 2024

/run-cpu-e2e

@mergify Mergify
Copy link
Contributor

mergify bot commented Jun 2, 2024

@zhagnlu E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

1 similar comment
@mergify Mergify
Copy link
Contributor

mergify bot commented Jun 2, 2024

@zhagnlu E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@zhagnlu
Copy link
Contributor Author

zhagnlu commented Jun 3, 2024

/run-cpu-e2e

@mergify Mergify
Copy link
Contributor

mergify bot commented Jun 3, 2024

@zhagnlu E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@zhagnlu zhagnlu force-pushed the optimize_memory branch from bd65aac to 3e5eb5b Compare June 3, 2024 07:15
@xiaofan-luan
Copy link
Collaborator

/lgtm
/approve

@zhagnlu
Copy link
Contributor Author

zhagnlu commented Jun 3, 2024

rerun ut

@zhagnlu zhagnlu mentioned this pull request Jun 3, 2024
Merged
@xiaofan-luan
Copy link
Collaborator

/l g t m

@xiaofan-luan
Copy link
Collaborator

/lgtm
/approve

@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: xiaofan-luan, zhagnlu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

longjiquan
longjiquan reviewed Jun 4, 2024
View reviewed changes
internal/core/src/segcore/InsertRecord.h
@@ -285,13 +286,13 @@ class OffsetOrderedArray : public OffsetMap {

private:
bool is_sealed = false;
std::vector<std::pair<T, int64_t>> array_;
std::vector<std::pair<T, int32_t>> array_;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch.

longjiquan
longjiquan reviewed Jun 4, 2024
View reviewed changes
Copy link
Contributor

@longjiquan longjiquan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@sre-ci-robot sre-ci-robot merged commit c6f8a73 into milvus-io:master Jun 4, 2024
14 of 15 checks passed
sre-ci-robot pushed a commit that referenced this pull request Jun 5, 2024
@zhagnlu
enhance: optimize some cache to reduce memory usage (#33560)
f94a722 
pr: #33534
pr: #33599

Signed-off-by: luzhang <[email protected]>
Co-authored-by: luzhang <[email protected]>
yellow-shine pushed a commit to yellow-shine/milvus that referenced this pull request Jul 2, 2024
@zhagnlu @yellow-shine
enhance: optimize some cache to reduce memory usage (milvus-io#33534)
9c66f67 
milvus-io#33533

Signed-off-by: luzhang <[email protected]>
Co-authored-by: luzhang <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@longjiquan longjiquan longjiquan left review comments

@bigsheeper bigsheeper Awaiting requested review from bigsheeper

@godchen0212 godchen0212 Awaiting requested review from godchen0212

Assignees

@longjiquan longjiquan

@xiaofan-luan xiaofan-luan

Labels
approved ci-passed dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement lgtm size/L Denotes a PR that changes 100-499 lines.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@zhagnlu @xiaofan-luan @sre-ci-robot @longjiquan @czs007