Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store : OOMKilled - exit code 137 #5577

Closed
chalut01 opened this issue Aug 9, 2022 · 2 comments
Closed

Store : OOMKilled - exit code 137 #5577

chalut01 opened this issue Aug 9, 2022 · 2 comments
Labels

Comments

@chalut01
Copy link

chalut01 commented Aug 9, 2022

Thanos, Prometheus and Golang version used: 0.26.0

Hi,
From the following issue #3665. I get an error "OOMKilled - exit code 137" many time.
Do you have any suggestions?

My args
--store.grpc.series-sample-limit=0'
--log.level=info'
--data-dir=/var/thanos/store'
--index-cache.config-file=/thanos-conf/thanos-store-redis.yaml'
--chunk-pool-size=8GB'
--store.grpc.series-max-concurrency=10'
--sync-block-duration=5m'
--block-sync-concurrency=10'
--objstore.config-file=/thanos-conf/s3.yaml'
--store.enable-index-header-lazy-reader'
--min-time=-360d'
--max-time=-120d'

resources limit
cpu: '4'
memory: 25Gi

log
thanos-store-1h-5b44d8c479-dr8fj.log

@chalut01 chalut01 changed the title OOMKilled - exit code 137 Store : OOMKilled - exit code 137 Aug 9, 2022
@oronsh
Copy link
Contributor

oronsh commented Aug 15, 2022

I think it's the label values, when passed a mathcer, if external label it would try to fetch all series to match on.
I opened a PR with a fix:
#5596

@stale
Copy link

stale bot commented Nov 13, 2022

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

@stale stale bot added the stale label Nov 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants