Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Postgres memleak caused by prepared statements growing forever when prepareCache is overflowing #1143

Open
silles79 opened this issue Nov 5, 2024 · 0 comments

Comments

@silles79
Copy link

silles79 commented Nov 5, 2024

Our Postgres db keeps running out of RAM.

We have an infinity stream of events from Kafka, each we groupBy(number = 1000) then go to the db and fetch in batch

We have a few queries which are parametrized with lists like

sql """ select ...
where id in (${SomeCodecs.id.list(ids.size)});
"""

We have a session with a pool, prepareCacheSize = 1024
We have about 25 queries

I think what happens is that we have more prepared statements ( lets say about 25 x 1000 ) than the cache size ,
and when the local cache runs out of space it just disposed it from the local cache but not from the service with deallocate.

When the same query with same number of params to list comes up, skunk just prepares again since it's no longer in the cache, so the number prepared statement prepared statement on the postgres server just keeps growing until it either dies or we restart the app

There are a few thing I could/will do at our side but I thought it make sense to report this as a bug

@silles79 silles79 changed the title Postgres memleak cause by prepared statements growing forever. Postgres memleak caused by prepared statements growing forever. Nov 6, 2024
@silles79 silles79 changed the title Postgres memleak caused by prepared statements growing forever. Postgres memleak caused by prepared statements growing forever when prepareCache is overflowing Nov 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant