"Connection closed by server."

[mdierksen]

Version: redis-py 4.5.5 and Redis 5.0.7-2ubuntu0.1

Platform: Python 3.10.11 on Ubuntu 20.04.6 LTS (Focal Fossa)

Description:
I am using the redis-py asyncio client in a service application. After several hours of running fine occasionally the error "Connection closed by server." is logged. This happens both for local Redis instances connected via TCP as well as for remote Redis instances connected via network TCP.

The settings for Redis are

timeout 0
tcp-keepalive 300

When I check the process with lsof I can see the number of Redis connections slowly rise over time capping out at around 20 or so. The program should not need that many simultaneous connections so I would guess a lot of these connection are stale.

There seem to be several parameters in redis-py available to influence the connection handling and the most promising to me seem to be retry_on_timeout=True, socket_keepalive=True and health_check_interval=X. However documentation for these parameters is very limited.

What would be the best practice to solve such an issue? I would like to avoid unneeded overhead.

[RZachLamberty]

I believe we are seeing the same thing in our application and would similarly love some advice

[finlaysawyer]

We're experiencing similar issues and would also like some advice. I've tried adding exponential backoff and retries but that isn't working, I suspect we need some combination of the parameters above.

from redis.asyncio import ConnectionPool, Redis
from redis.asyncio.retry import Retry
from redis.backoff import ExponentialBackoff

pool = ConnectionPool.from_url(...)
redis_client: Redis = Redis(
    connection_pool=pool, retry=Retry(ExponentialBackoff(), 5)
)

[achimnol]

I'm also experiencing frequent failures in GitHub Actions where I spawn multiple Redis containers for parallelized test executions with different port numbers, even though I configured the intrinsic retry and backoff configurations.

[mykolasolodukha]

TL;DR

Init redis with these 4 parameters:

r = redis.from_url(
  'rediss://default:<password>@XXXXXX.stackhero-network.com:6380',
  health_check_interval=10,
  socket_connect_timeout=5,
  retry_on_timeout=True,
  socket_keepalive=True
)

---

FYI I had the same issue using Heroku's Redis, and they have a doc on how to fix it: https://devcenter.heroku.com/articles/ah-redis-stackhero#:~:text=The%20error%20%E2%80%9Credis.,and%20the%20connection%20closes%20automatically.

[geonyoro]

My errors were happening on first connection, AND subsequent connections. Published a package subclassing redis that wraps the underlying execute_command and uses the retries.

Will switch to using this in my projects, awaiting their update on this.

https://pypi.org/project/redis-exec-retry/0.0.2/

[KundaPanda]

We are experiencing the same issue. Setting the previously mentioned parameters didn't really help since the connections were being closed randomly all the time. I tried using a single connection client, but that didn't help at all.

The first thing that helped was re-creating the client for every request (which is, of course, suboptimal), but the thing that actually solved this issue was switching the connection's parser_class to PythonParser. This suggests that there is some kind of a problem with hiredis or the hiredis parser client.
EDIT: This client is called _AsyncRESP2Parser or _AsyncRESP3Parser in the new version of redis-py.

Nevermind, this is actually fixed by manually creating the connection pool and passing it to the redis client regardless of the parser.

[github-actions]

This issue is marked stale. It will be closed in 30 days if it is not updated.