Local TLS Support for Cloud SQL Auth Proxy

[raulsiles]

Feature Description

There are new scenarios, complementing the ones mentioned in 2017 for issue #102, that might require local TLS support in Cloud SQL Auth Proxy to be able to use TLS (I mention SSL here just for reference) from an app, or from a local SQL client, to the proxy.

A very specific example is when MySQL is used with "caching_sha2_password" (SHA-2) authentication (vs. the standard "mysql_native_password" authentication type). In this case, MySQL server and MySQL clients, such as MySQL Workbench, enforce the usage of TLS.

For example (using Cloud SQL Auth Proxy in localhost:3307), in MySQL Workbench, if the default connection setup for TLS is used ("If available"), due to the SHA-2 authentication type, a TLS connection is required:

If the connection TLS setting is changed to "Require", then an error is generated, as Cloud SQL Auth proxy does not support TLS:

I might understand that TLS might not be a strong requirement from a security perspective when connecting to the proxy via localhost. I'm also aware of the overhead associated to adding TLS support to the proxy, both from a binary/code perspective, and from a configuration point of view (a new proxy key and certificate need to be generated, and/or even a new CA, or capabilities to provide these files via command line options to the proxy).

However, new scenarios like the one described above might promote re-evaluating the TLS support in the proxy.

This kind of scenario might even apply to GCP infrastructures with the sidecar pattern, not just to the local SQL client and proxy scenario described above (please DM if interested).

Alternatives Considered

It seems there is no workaround for the specific MySQL issue described above.

Additional Context

Issue #102

[kurtisvg]

Does the workaround described in this SO post work for MySQL Workbench: https://stackoverflow.com/a/70026049/9414803?

[raulsiles]

No, the "useSSL=0" directive is equivalent to the "If available" menu option I mentioned above. This directive was required in MySQL Workbench 8.0.27 due to a bug in that version that removed the "If available" menu option by mistake. In MySQL Workbench 8.0.28 the option is back, and there is no need to set it up with the directive manually.

The issue here is that, based on the standards and/or security best practices, there are scenarios where MySQL clients will enforce the usage of TLS (SHA-2 authentication is one example), and the Cloud SQL Auth Proxy does not support it.

[kurtisvg]

No, the "useSSL=0" directive is equivalent to the "If available" menu option I mentioned above.

Unless MySQL Workbench is doing something weird, useSSL=0 is typically "DISABLED" which is different from the usually default "PREFERRED" (which is what I would guess "If available" is)

According to the docs for caching_sha2_password:

For each user account, the first client connection for the user after any of the following operations must use a secure connection (made using TCP using TLS credentials, a Unix socket file, or shared memory)

Since the server side proxy is using a unix domain socket, the server isn't able to tell that the connection is over TCP. Instead, there must be some client side check happening. Switching to a unix domain socket on the local proxy side should bypass whatever client check is happening?

[raulsiles]

Yes, potentially using Unix sockets might change the local Cloud SQL Auth Proxy behaviour, but I'm currently testing it under Windows using TCP (Unix sockets are not supported: "unsupported network: unix"). I think the solution should cover all possible scenarios, hence my suggestion to add support for TLS to the proxy.

Potentially, the other GCP scenarios I referred to related to the sidecar pattern, Unix-based, are also affected (I don't know if they are using Unix sockets or TCP).

[enocom]

Related to #84.

[kurtisvg]

I thought I had responded to this issue earlier, but revisiting it I realize I had typed a draft but failed to send. So allow me a belated response on why this isn't likely to get prioritized 🙂

In order to support "local TLS" between the application and the proxy, we have to have 2 things:

1. The proxy has to start being protocol aware (e.i. it needs to be able to speak "postgres" to a protocol instance)
2. Before accepting any connections, the proxy has to know which engine type it's talking too

The first is complicated because it means we as maintainers need to have a good understanding of the protocols for to implement it ourselves. It also means a significant increase in maintenance and testing burden, as we've got to continuously tests and observe changes to the protocol and make sure any edge cases aren't missed, and if Cloud SQL adds support of more database versions in the future, we have significant work to do to match those database versions as well.

The second is problematic because it means we have to make and complete an API call before we can start accepting connections. We actually already have to do this when listening on unix domain sockets (because some postgres drivers assume a specific suffix), and it's a bit of a friction point - not only does it increase the initial start up time, but also means when that API call fails the proxy startup also fails, which means the proxy startup is a little less deterministic than we like.

Finally, there are a bunch of tools and alternatives that do cover this use case really well. Stuff like ProxySQL and PGbouncer already offer protcol-aware proxies and add a bunch of other great features like connection pooling and read/write traffic splitting. We could try to implement similar features, but almost certainly it's not going to be as good as those projects without a ton of work and community adoption. So using one of these proxies with the Auth proxy is likely better than us trying to implement something ourselves.

So in summary the benefits there are pretty limited, and the drawbacks are pretty significant. Unless we see an overwhelming amount of feedback that says otherwise, we're probably going to focus on improving the Auth proxy in other ways (we've got a pretty long list of feature requests) before we spend too much time on this case.

[enocom]

Closing this out as we're unlikely to pick up this work for the reasons stated above.

As usual, though, if there's more information, feel free to re-open and add comments here.