ConnectionError [SSL: CERTIFICATE_VERIFY_FAILED] when connecting to AWS ElastiCache over TLS

[ghost]

We're experiencing connection issues with v3.x when trying to use our AWS managed ElastiCache Redis Cluster (version 4.0.10) over TLS (rediss://...). Rolling back to 2.10 fixes the issue immediately.

This is the stacktrace of our application:
ConnectionError: Error 1 connecting to master.**********.cache.amazonaws.com:6379. [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1051).
(8 additional frame(s) were not displayed)
...
File "redis/client.py", line 1182, in exists
return self.execute_command('EXISTS', *names)
File "redis/client.py", line 760, in execute_command
connection.send_command(*args)
File "redis/connection.py", line 619, in send_command
self.send_packed_command(self.pack_command(*args))
File "redis/connection.py", line 594, in send_packed_command
self.connect()
File "redis/connection.py", line 498, in connect
raise ConnectionError(self._error_message(e))

[andymccurdy]

@u2mejc thoughts? This is related to the SSL 'strict' change in #1017

[andymccurdy]

@dkms-its You can use redis-py 3.x and create a client instance with the ssl_cert_reqs=None option. This will behave identical to redis-py 2.x with regard to encrypted connections.

[ghost]

Thanks for the quick reply and workaround, @andymccurdy. However, we're using redis.py not directly but through django-redis, so i guess i have to ask them for a corresponding option.

[u2mejc]

Hi @dkms-its, are you by chance using a cname to connect? This would cause the cert to not match the host name. AWS doesn't support custom ssl certs for Elasticache at this time, so you'll need to use the actually target's name (ie master.**********.cache.amazonaws.com). I'd recommend you submit another request to AWS support to add support requesting the ability to use ACM / IAM certs in elacticache (I have already).

[ramkrt]

Had the same issue when accessing azure redis ( using Azure end point , not a CNAME ) .
specifying ssl_ca_certs='/etc/pki/tls/certs/ca-bundle.crt' when creating the redis client resolved the issue.

[rwsanders]

Im having this issue with an Azure Redis Cache.

Traceback (most recent call last):
File "C:\Python37\lib\site-packages\flask\app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "C:\Python37\lib\site-packages\flask\app.py", line 1799, in dispatch_request
return self.view_functionsrule.endpoint
File "C:\Python37\lib\site-packages\flask_restful_init_.py", line 458, in wrapper
resp = resource(*args, kwargs)
File "C:\Python37\lib\site-packages\flask\views.py", line 88, in view
return self.dispatch_request(*args, kwargs)
File "C:\Python37\lib\site-packages\flask_restful_init_.py", line 573, in dispatch_request
resp = meth(*args, kwargs)
File "C:\Users*\resources.py", line 44, in get
app_gateway = manager.get_app_gateway(args)
File "C:\Users*\app_gateway.py", line 79, in get_app_gateway
subscription_id)
File "C:\Users*\azure_manager.py", line 12, in get_management_client
tenant_creds = creds_mgr.get_tenant_credentials(tenant_id)
File "C:\Users*\credentials.py", line 42, in get_tenant_credentials
keys = self.get_credential_keys(tenant_id)
File "C:\Users*\credentials.py", line 31, in get_credential_keys
val = self.sget(c_key)
File "C:\Users***\cache.py", line 182, in sget
r = self._client.get(name)
File "C:\Python37\lib\site-packages\redis\client.py", line 1207, in get
return self.execute_command('GET', name)
File "C:\Python37\lib\site-packages\redis\client.py", line 760, in execute_command
connection.send_command(*args)
File "C:\Python37\lib\site-packages\redis\connection.py", line 619, in send_command
self.send_packed_command(self.pack_command(*args))
File "C:\Python37\lib\site-packages\redis\connection.py", line 594, in send_packed_command
self.connect()
File "C:\Python37\lib\site-packages\redis\connection.py", line 498, in connect
raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 1 connecting to *********.redis.cache.windows.net:6380. [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1051).

Just updated from 2.10.6 to 3.0.0. The constructor I'm using is the following.
self._client = StrictRedis(
host='*********.redis.cache.windows.net',
port=6380,
password='REDIS-APP-KEY',
db=0,
ssl=True)

[u2mejc]

CERTIFICATE_VERIFY_FAILED

@rwsanders Looks like you didn't load the CA certs, see the comment just above yours. 😉

[rwsanders]

Not sure what this means in terms of using Azure PaaS Redis and Azure PaaS WebApp (where Python is running) where the implementation is bootstrapped with SSL out of the box. I'm not sure if the CA certs are made available to me. Is there something I'm missing?

[rwsanders]

Aha! It looks like I need to create a separate app service certificate using the Azure App Service Certificate service. Thanks for the help.

[Alessandro-QDT]

I am experiencing the same issue as the two above posts. I am sorry to repeat the question here but I can't decipher the solution. Can you please provide some details?
From an Azure Ubuntu 18.04 VM I am trying to access an Azure Redis Cache:

import redis
redis_host='<...>.redis.cache.windows.net'
redis_pwd='<...>'
r = redis.StrictRedis(host=redis_host, port=6380, db=0, password=redis_pwd, ssl=True)
r.set('foo', 'bar')

and I get the error:

redis.exceptions.ConnectionError: Error 1 connecting to <...>.redis.cache.windows.net:6380. [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:847).

I get the error while issuing the set() command and not in creating the redis object.
On the same VM I have the Redis Desktop Manager GUI configured with the same host and password and it works just fine .

[andymccurdy]

@Alessandro-QDT You need to either

1. Specify the location of the certificates. I don't know where they are located on Azure servers.

or

2. Turn SSL strict mode off (and accept the security risks of doing so)

[martin-weber]

I had the same error with Azure Redis Cache on Windows and on MacOS and solved it by adding ssl_cert_reqs=u'none' as mentioned by @andymccurdy.

pool = redis.ConnectionPool(host=..., 
                            port=6380,
                            password=...,
                            db=0, 
                            connection_class=redis.SSLConnection, 
                            ssl_cert_reqs=u'none')
client = redis.Redis(connection_pool=pool)

[andymccurdy]

It seems like redis-py is behaving correctly. Closing this issue as there are ways to load the correct certs with both Azure and AWS.

[mikelibg]

I still think that the problem is that by default the ssl_cert_reqs is set to 'required'. If someone wants to use ssl=true with password only, without certs, one needs to change this parameter instead of setting it to required only when you want to specify specific cert files.

[cruisemaniac]

Is there a known cert bundle for the elasticache service like how the rds service has it? How is the line truly encrypted in transit if we cannot validate the CA?