Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

significant increase in network and memory footprint after upgrading to 1.1.x from 1.0.x #1327

Closed
aylei opened this issue Dec 11, 2019 · 3 comments · Fixed by #1504
Closed

significant increase in network and memory footprint after upgrading to 1.1.x from 1.0.x #1327

aylei opened this issue Dec 11, 2019 · 3 comments · Fixed by #1504
Assignees
@aylei
Labels
needs-cherry-pick-1.1 type/bug Something isn't working type/question Further information is requested

Comments

@aylei
Copy link
Contributor

aylei commented Dec 11, 2019 •
edited
Loading

Question

After upgrade tidb-operator from v1.0.3 to the latest master (a3f4b8d), there is a significant increase in network and memory footprint:

image
image

cluster scale:

  • Node: 60+
  • TiDB clusters: 70+
@aylei aylei added type/bug Something isn't working type/question Further information is requested labels Dec 11, 2019
@aylei aylei added this to the v1.1.0 milestone Dec 11, 2019
@aylei aylei self-assigned this Dec 11, 2019
@aylei
Copy link
Contributor Author

aylei commented Dec 11, 2019

image

@aylei
Copy link
Contributor Author

aylei commented Dec 11, 2019

The problem is introduced by ConfigMapInformer, we should remove the configmap cache.

Meanwhile, as @cofyc mentioned, cache for Secret has the same risk and should also be removed.

@aylei aylei mentioned this issue Jan 8, 2020
Merged
@github-actions
Copy link

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 15 days

@cofyc cofyc removed this from the v1.1.0 milestone Feb 27, 2020
@Yisaer Yisaer mentioned this issue Apr 13, 2020
Merged
@cofyc cofyc mentioned this issue Apr 23, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@aylei aylei

Labels
needs-cherry-pick-1.1 type/bug Something isn't working type/question Further information is requested
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Remove unnecessary informer caches aylei/tidb-operator
2 participants
@cofyc @aylei