Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Twemproxy " Connection refused " in logs on failure of one redis instance #530

Closed
mishtika opened this issue Jul 26, 2017 · 6 comments
Closed

Comments

@mishtika
Copy link

I have configured twem proxy with 2 redis servers.
When one of these redis server fails the twem proxy gives a error saying " Connection refused"

My twem conf
beta:
listen: 127.0.0.1:22122
hash: fnv1a_64
hash_tag: "{}"
distribution: ketama
auto_eject_hosts: true
timeout: 400
redis: true
servers:

  • 127.0.0.1:6381:1
  • 127.0.0.1:6379:1

i am trying to get a key from redis. If i kill one instance of redis and then try to run the get command then it gives the following error:
[2017-07-26 16:25:24.548] nc_core.c:237 close s 15 '127.0.0.1:6381' on event FFFFFF eof 0 done 0 rb 0 sb 0: Connection refused

@therealbill
Copy link

Twemproxy is a key distributing proxy. When you have two servers half the keys (roughly) go to one, the other to the other. When one of those is down you don't have access to those keys. If you are looking for master/slave availability you'll need to use Redis' native replication to handle that rather than Twemproxy.

@tuananh
Copy link

tuananh commented Jul 26, 2017

@therealbill shouldn't i get a cache miss for that case instead of connection refused?

@jslusher
Copy link

I think my issue is related to this. I'm proxying memcached, but I have the same problem where if one of the servers goes down the proxy doesn't use the second server: #534

@niuchenglei
Copy link

i think my issue is that, for example:
we got key a,b,c,d and a,b on redis server No.1, c,b on redis server No.2, and we request key a,c with twemproxy, it will request nothing when No.1 has down.

@chrizkiran
Copy link

Even am facing the same issue. I get the below error when one of the Redis instance is down.

[2018-06-09 20:02:17.672] nc_request.c:96 req 91 done on c 14 req_time 0.562 msec type REQ_REDIS_HMGET narg 3 req_len 53 rsp_len 34 key0 'mqtt_user:client1' peer '127.0.0.1:33311' done 1 error 0 [2018-06-09 20:02:17.681] nc_request.c:96 req 93 done on c 13 req_time 0.163 msec type REQ_REDIS_HMGET narg 3 req_len 53 rsp_len 34 key0 'mqtt_user:client1' peer '127.0.0.1:47821' done 1 error 0 [2018-06-09 20:02:20.660] nc_response.c:118 s 15 active 0 is done [2018-06-09 20:02:20.660] nc_core.c:237 close s 15 '127.0.0.1:6379' on event 00FF eof 1 done 1 rb 68 sb 106

I have configured Redis Sentinel . Am I misunderstanding anything with respect to twemproxy??

Any how after some frequent connect requests, I get the response back.

@TysonAndre
Copy link
Collaborator

If i kill one instance of redis and then try to run the get command then it gives the following error:
[2017-07-26 16:25:24.548] nc_core.c:237 close s 15 '127.0.0.1:6381' on event FFFFFF eof 0 done 0 rb 0 sb 0: Connection refused

That syslog is hard to read but expected - the connection to the server closed, and twemproxy attempted to reconnect to that server(s) when you sent another command and got a Connection refused error because nothing listened on that port


Aside: #608 will significantly refactor the way reconnection is implemented in a future 0.6.0 release (heartbeat, failover).

I have configured Redis Sentinel . Am I misunderstanding anything with respect to twemproxy??

Redis sentinel support hasn't been merged in from a fork yet, but is planned as part of #608 for 0.6.0 - even when the sentinel client implementation is merged in you'd have to configure the list of sentinels.


Closing this because the original question was answered

@TysonAndre TysonAndre changed the title Twemproxy " Connection refused " on failure of one redis instance Twemproxy " Connection refused " in logs on failure of one redis instance Jul 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants