Unable to connect to etcd cluster started with "auto-tls" flag

[RaeesBhatti]

Hey there, I started a 3 node etcd cluster with –auto-tls and –peer-auto-tls flags. The cluster is up and running but I can't connect to it. I've tried using etcdctl.

$ ETCDCTL_ENDPOINT=https://127.0.0.1:2379 etcdctl member list
Error:  x509: cannot validate certificate for 10.129.16.183 because it doesn't contain any IP SANs

Using curl:

$ curl https://127.0.0.1:2379/
curl: (60) SSL certificate problem: unable to get local issuer certificate
(further information truncated)

From the response of curl, I'm guessing that it requires an explicit CA certificate but where do I find that? Please provide your insight on the problem.
Also, when using auto-tls and peer-auto-tls flags, is the communication between server-server and server-client really tamper proof?

[gyuho]

When you use --auto-tls, you will see logs of

2017-04-03 19:13:31.414098 I | embed: ClientTLS: cert = default.etcd/fixtures/client/cert.pem, key = default.etcd/fixtures/client/key.pem, ca = , trusted-ca = , client-cert-auth = false

And you specify this certs in your etcdctl or curl commands (it's for client TLS connections)

[RaeesBhatti]

That results in an malformed HTTP response error.

$ ETCDCTL_ENDPOINT=https://127.0.0.1:2379 sudo etcdctl --cert-file=/var/lib/etcd/fixtures/client/cert.pem --key-file=/var/lib/etcd/fixtures/client/key.pem member list
Error:  malformed HTTP response "\x15\x03\x01\x00\x02\x02"

[RaeesBhatti]

CURL throws the same error even when I specify client certificate.

$ sudo curl --cert /var/lib/etcd/fixtures/client/cert.pem --key /var/lib/etcd/fixtures/client/key.pem https://127.0.0.1:2379/
curl: (60) SSL certificate problem: unable to get local issuer certificate
(truncated)

[RaeesBhatti]

The earlier malformed HTTP response error was because I was setting the env variable in current user and then using sudo with etcdctl, it tried to parse response as plain HTTP and failed. That was fixed by using sudo at the start of the command. And it resulted in error faced earlier.

$ sudo ETCDCTL_ENDPOINT=https://127.0.0.1:2379 etcdctl --cert-file=/var/lib/etcd/fixtures/client/cert.pem --key-file=/var/lib/etcd/fixtures/client/key.pem member list
Error:  x509: cannot validate certificate for 127.0.0.1 because it doesn't contain any IP SANs

[gyuho]

Ok I can repro... this doesn't seem to work

./bin/etcd --auto-tls --listen-client-urls=https://localhost:2379 --advertise-client-urls=https://127.0.0.1:2379

ETCDCTL_API=3 ./bin/etcdctl \
    --endpoints localhost:2379 \
    --cert default.etcd/fixtures/client/cert.pem \
    --key default.etcd/fixtures/client/key.pem \
    endpoint health

Will look into it. Thanks.

[gyuho]

@RaeesBhatti etcd auto-tls generates CAs and certificates but does not expose the self-signing CA, which is why you are getting x509: ... errors with client request (since there is no signing CA provided).

Error: x509: cannot validate certificate for 127.0.0.1 because it doesn't contain any IP SANs

This is fixed via #7659. And #7674 will provide similar information if you use v3 etcdctl.

You can still use the generated keys with --insecure-skip-tls-verify flag

ETCDCTL_API=3 etcdctl \
    --insecure-skip-tls-verify \
    --endpoints localhost:2379 \
    --cert $HOME/certs/my-etcd-1.pem \
    --key $HOME/certs/my-etcd-1-key.pem \
    put foo bar

However, --insecure-skip-tls-verify should only be used for testing purposes: it will disable all verification. I would not recommend auto-tls for production use. You can generate your own cert and CA with openssl or cfssl.

Thanks.