feat: add Redis configuration for improved memory management

[Hassanzadeh-sd]

As Sentry continues to evolve, effective resource management becomes crucial for maintaining performance and stability. This update includes configurations that will help optimize Redis's memory usage, ensuring that the system runs efficiently under varying loads.

Key Changes:

• Added maxmemory Directive: Configured Redis to limit its memory usage to a specified size. This prevents excessive memory consumption and helps maintain system stability.
• Set maxmemory-policy to allkeys-lru: This policy allows Redis to evict the least recently used keys when it reaches the memory limit, ensuring that frequently accessed data remains available while older, less-used data is removed.

Thank you for considering this enhancement to improve Redis's performance within the Sentry ecosystem!

[BYK]

Thank you so much for the PR!

Requested a change and left a question. After that I think this is good to merge but I'll also consult our SRE team to make sure.

Finally, if you could add this boilerplate to your PR description, we'd be able to merge at ease: https://github.com/getsentry/self-hosted/blob/master/.github/PULL_REQUEST_TEMPLATE.md

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.01%. Comparing base (be66069) to head (6d33bdc).
Report is 1 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #3427   +/-   ##
=======================================
  Coverage   99.01%   99.01%           
=======================================
  Files           3        3           
  Lines         203      203           
=======================================
  Hits          201      201           
  Misses          2        2           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[BYK]

Got word from a colleague: "For a self hosted Relay itself, that config would be fine."

[BYK]

Note to followers: if the eviction policy kicks in, there might be data loss as Redis is not just used as a cache but also a processing queue/storage.

[Hassanzadeh-sd]

thanks. @BYK

[mwarkentin]

In prod our redis are split into many different clusters and instances, so I don't know what the best setting would be for a deployment where everything is running within a single redis. It might be worth considering volatile-lru instead of allkeys-lru:

volatile-lru: Evict the least recently used keys that have the expire field set to true.

The volatile-lru and volatile-random policies are mainly useful when you want to use a single Redis instance for both caching and for a set of persistent keys. However, you should consider running two separate Redis instances in a case like this, if possible.