Added support for postgres certificate authentication for native tls #1166

linuxuser586 · 2021-04-10T23:34:02Z

Fixes #1162 Adds support for postgres certificate authentication for native tls with openssl or rustls.

This allows using the uri format postgres://postgres:password@localhost:5432/sqlx?sslmode=verify-full&sslrootcert=./ca.crt&sslcert=./client.crt&sslkey=./client.key or the following environment variables

PGSSLROOTCERT
PGSSLCERT
PGSSLKEY

This can be used for better security by disabling password login. It using the same patters used with psql.

Example

# pg_hba.conf
# omit host to prevent password login
hostssl all all all cert

mehcode · 2021-04-27T19:37:12Z

I'm thinking it might be best to allow a native_tls::TlsConnector or rustls::Config to be be passed to the connection options directly to allow anything.

linuxuser586 · 2021-04-27T20:44:49Z

@mehcode I'm not sure if rusttls supports certificate authentication in the current implementation. I was trying to make the connection string match the way psql works. I had to use some openssl functions to handle the pkcs12 format. One way to get around this would be to do the pkcs12 conversion and specify that. I did that in my first test and it worked so seems possible for rustls. I will have to investigate more to see how to make it works with both tls supported options.

…-fixes Conflicts: sqlx-core/Cargo.toml

linuxuser586 · 2021-07-05T18:36:24Z

@mehcode I finally found time to add support for certificate authentication using rustls. I updated the github workflows to test native tls using openssl, which is required for pkcs12 to identity transformation. I added a separate matrix for rustls without openssl since it does not need to use the pkcs12 format. I made openssl optional with native tls since the openssl library is only needed to support creating the identity from pkcs12 when using certificate authentication.

abonander · 2021-08-16T23:52:36Z

@linuxuser586 sorry for the delay.

We had some recent changes to our Github Actions matrices that need to be rebased into this PR. Do you mind taking care of that?

…-fixes

linuxuser586 · 2021-08-17T09:13:30Z

@abonander Done

abonander

I was giving this a more in-depth review before merging it and came across a number of issues.

abonander · 2021-09-22T20:47:31Z

sqlx-core/src/net/tls/rustls.rs

+    let cur = Cursor::new(pem);
+    let mut reader = BufReader::new(cur);
+    rustls_pemfile::certs(&mut reader)
+        .unwrap()


The docs for rustls_pemfile::certs() is a bit misleading; it does propagate I/O errors but it will also return I/O errors if there's any syntax errors in the data. We probably shouldn't panic here in that case.

abonander · 2021-09-22T20:48:11Z

sqlx-core/src/net/tls/rustls.rs

@@ -77,3 +94,29 @@ impl ServerCertVerifier for NoHostnameTlsVerifier {
        }
    }
 }
+
+fn to_certs(pem: Vec<u8>) -> Vec<rustls::Certificate> {


This name isn't really idiomatic as it doesn't take &self.

I think certs_from_pem() would be better.

abonander · 2021-09-22T20:57:54Z

sqlx-core/src/net/tls/rustls.rs

+        .collect()
+}
+
+fn to_private_key(pem: Vec<u8>) -> Result<rustls::PrivateKey, Error> {


Same here, private_key_from_pem()

abonander · 2021-09-22T20:59:06Z

sqlx-core/src/net/tls/mod.rs

+                    X509::from_pem(&cert_data),
+                ) {
+                    if let Ok(pkcs) =
+                        Pkcs12::builder().build(PASSWORD, PASSWORD, pkey.as_ref(), cert.as_ref())


I'm not a fan of this hack, though I do understand that the Identity API doesn't really give you any other option.

I would like to see this PR merged in native-tls but it's been stale for over a year now so I dunno what we wanna do here: sfackler/rust-native-tls#147

If nothing else I would like to see comments added to this code explaining exactly why it's doing this and what it needs to be better.

I also don't really like how the errors are discarded here.

Honestly I would probably consider requiring the openssl crate for this, even as an optional feature, to be a non-starter. I would prefer to wait for the above PR to be merged, though it's not clear when that's going to happen.

abonander · 2021-09-22T21:18:46Z

sqlx-core/src/net/tls/mod.rs

@@ -117,8 +121,16 @@ async fn configure_tls_connector(
    accept_invalid_certs: bool,
    accept_invalid_hostnames: bool,
    root_cert_path: Option<&CertificateInput>,
+    client_cert_path: Option<&CertificateInput>,


Can we fix these warnings?

If the native-tls feature is enabled but not openssl-native, we should check if these are not None and if so, log a warning that these do nothing without the openssl-native feature.

abonander · 2021-09-22T21:24:59Z

sqlx-core/Cargo.toml


 # support offline/decoupled building (enables serialization of `Describe`)
 offline = ["serde", "either/serde"]

+# openssl
+openssl-native = [ "openssl" ]


I don't really see the reasoning behind giving this a different feature name. Why not just openssl like in the sqlx crate?

If you tried to define an openssl feature but got an error, it's because you don't need to explicitly define features for optional = true dependencies.

abonander · 2021-11-09T19:59:35Z

Keep an eye on sfackler/rust-native-tls#209, that looks promising.

emk · 2021-12-28T18:36:20Z

Hello! Thank you for working on client certificates!

I'm the maintainer of https://github.com/emk/rust-musl-builder, a moderately popular tool for statically-linking Rust apps that use OpenSSL and libpq against libmusl. I'm trying to migrate away from that approach and encourage people to migrate to rustls, which is much easier to cross-compile and statically link.

Here is the code that I'm using for "pure Rust" TLS support, including some (as yet untested) client certificate support: https://github.com/dbcrossbar/dbcrossbar/pull/188/files. This uses rustls and rustls-pemfile to set up a single client certificate in a fashion much like that of libpq.

I would be happy to contribute any part of this code to help with this PR, or to prepare a draft PR based on this approach.

Google's hosted PostgreSQL servers actually require client certs to connect using TLS. So this is a potentially very useful feature.

abonander · 2022-01-21T21:55:22Z

Google's hosted PostgreSQL servers actually require client certs to connect using TLS. So this is a potentially very useful feature.

Which ones? Because we use PostgreSQL servers hosted by Google Cloud and it works fine without client certs. It just requires TLS in general.

larksoftware · 2022-01-28T02:34:07Z

Google's hosted PostgreSQL servers actually require client certs to connect using TLS. So this is a potentially very useful feature.

As a new user of GCP I was under this impression as well. But after trying to configure sqlx with TLS and failing I did some more reading and found there is another method:

https://cloud.google.com/sql/docs/mysql/connect-admin-proxy

Using the Cloud SQL Auth proxy is the recommended method for connecting to a Cloud SQL instance. The Cloud SQL Auth proxy:

Works with both public and private IP endpoints
Validates connections using credentials for a user or service account
Wraps the connection in a SSL/TLS layer that's authorized for a Cloud SQL instance

So using this auth proxy seems to be the recommended solution for connecting. I'm going to give this a try now and hope I can then use sqlx with my GCP postgres db.

linuxuser586 · 2022-01-28T10:14:10Z

@abonander @larksoftware I experienced this very issue and is why I created the https://github.com/linuxuser586/sqlx/tree/postgres-native-tls-fixes fork. Unfortunately, my changes were never merged into the official version and I don't have the time to address the concerns to make it in to the official version; however, the forked version does meet my needs and may meet yours. It is easy to add the git version in rust and so for me, not using the official version is not an issue. I hope this helps you solve your problem.

abonander · 2022-01-28T20:28:11Z

I don't have the time to address the concerns to make it in to the official version

I guess we can close this PR then. My review comments still stand, as I would rather see proper support in native-tls than the round-tripping through OpenSSL here.

Note to anyone using this fork, it doesn't properly bubble up errors if there's something wrong with the certificate, it'll just panic: #1166 (comment)

abonander · 2022-01-28T20:30:23Z

Code quality aside, the solution for rustls was acceptable so if anyone wants to open a PR based on this one with just the rustls changes and address the nits, we'd gladly merge it.

linuxuser586 added 8 commits April 8, 2021 17:12

added support for postgres sslcert and sslkey

2630b9c

fixed broken build after refactor

019364d

fixed incorrect var order

dbce967

Merge branch 'master' into postgres-native-tls-fixes

7e4cf5e

added tests for postgres certificate authentication

1463670

Merge branch 'master' into postgres-native-tls-fixes

acdce64

added missing env vars to PgConnectOptions docs

adfe6d4

added missing parameters for non-postgres

a10cb65

linuxuser586 added 2 commits July 5, 2021 08:46

Merge remote-tracking branch 'origin/master' into postgres-native-tls…

b92fee5

…-fixes Conflicts: sqlx-core/Cargo.toml

added postgres rustls certificate authentiaction support

00e73be

Merge remote-tracking branch 'origin/master' into postgres-native-tls…

691cbd6

…-fixes

linuxuser586 and others added 2 commits August 24, 2021 06:38

Merge branch 'launchbadge:master' into postgres-native-tls-fixes

673c9c2

Merge branch 'master' into postgres-native-tls-fixes

13dc17a

abonander requested changes Sep 22, 2021

View reviewed changes

abonander mentioned this pull request Sep 22, 2021

Implement PKCS8 certificate support for all three backends. sfackler/rust-native-tls#147

Closed

seguidor777 mentioned this pull request Nov 22, 2021

How can I add SSL options for Cockroach DB ? #1544

Closed

abonander closed this Jan 28, 2022

abonander mentioned this pull request Jan 28, 2022

Postgres native tls does not support client certificates. #1162

Open

ThibsG mentioned this pull request May 2, 2022

Add client SSL authentication using key-file for Postgres, MySQL and MariaDB #1850

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added support for postgres certificate authentication for native tls #1166

Added support for postgres certificate authentication for native tls #1166

linuxuser586 commented Apr 10, 2021 •

edited

Loading

mehcode commented Apr 27, 2021

linuxuser586 commented Apr 27, 2021

linuxuser586 commented Jul 5, 2021

abonander commented Aug 16, 2021

linuxuser586 commented Aug 17, 2021

abonander left a comment

abonander Sep 22, 2021 •

edited

Loading

abonander Sep 22, 2021

abonander Sep 22, 2021

abonander Sep 22, 2021

abonander Sep 22, 2021

abonander Sep 22, 2021

abonander Sep 22, 2021

abonander Sep 22, 2021

abonander commented Nov 9, 2021

emk commented Dec 28, 2021

abonander commented Jan 21, 2022

larksoftware commented Jan 28, 2022

linuxuser586 commented Jan 28, 2022

abonander commented Jan 28, 2022

abonander commented Jan 28, 2022

Added support for postgres certificate authentication for native tls #1166

Added support for postgres certificate authentication for native tls #1166

Conversation

linuxuser586 commented Apr 10, 2021 • edited Loading

mehcode commented Apr 27, 2021

linuxuser586 commented Apr 27, 2021

linuxuser586 commented Jul 5, 2021

abonander commented Aug 16, 2021

linuxuser586 commented Aug 17, 2021

abonander left a comment

Choose a reason for hiding this comment

abonander Sep 22, 2021 • edited Loading

Choose a reason for hiding this comment

abonander Sep 22, 2021

Choose a reason for hiding this comment

abonander Sep 22, 2021

Choose a reason for hiding this comment

abonander Sep 22, 2021

Choose a reason for hiding this comment

abonander Sep 22, 2021

Choose a reason for hiding this comment

abonander Sep 22, 2021

Choose a reason for hiding this comment

abonander Sep 22, 2021

Choose a reason for hiding this comment

abonander Sep 22, 2021

Choose a reason for hiding this comment

abonander commented Nov 9, 2021

emk commented Dec 28, 2021

abonander commented Jan 21, 2022

larksoftware commented Jan 28, 2022

linuxuser586 commented Jan 28, 2022

abonander commented Jan 28, 2022

abonander commented Jan 28, 2022

linuxuser586 commented Apr 10, 2021 •

edited

Loading

abonander Sep 22, 2021 •

edited

Loading