Memory caches have an infinite lifetime and grow to consume infinite RAM #14310
Labels
⚠️bug?
This might be a bug
packages/backend
Server side specific issue/PR
🐢Performance
Efficiency related issue/PR
💡 Summary
Several memory caches do not have an eviction policy and can grow to contain millions of entries per active process. This is highly inefficient and is a partial cause of the slow memory leak that affects some instances. In particular, these caches have an infinite eviction time:
As a solution, caches should define a maximum capacity and reasonable eviction time. If the capacity is exceeded, then the oldest (least recently used) entry should be evicted to make room. This will prevent caches from growing indefinitely while still ensuring a high cache hit rate.
This bug was first discovered in the Sharkey fork, but exists in Misskey as well. For additional details, see Sharkey issue #600 and Sharkey MR #580.
Related to #14311.
🥰 Expected Behavior
Caches should have a limited size and reasonable eviction method to prevent infinite memory usage.
🤬 Actual Behavior
Memory caches are unbounded and consume infinite server memory.
📝 Steps to Reproduce
Run a federated instance for an extended period of time and observe memory usage. Background memory will increase indefinitely as caches are filled with remote users, public keys, and other data.
💻 Frontend Environment
🛰 Backend Environment (for server admin)
Do you want to address this bug yourself?
The text was updated successfully, but these errors were encountered: