diff --git a/site2/docs/security-tls-authentication.md b/site2/docs/security-tls-authentication.md index 295c1620dba457..d82122dd7f19a4 100644 --- a/site2/docs/security-tls-authentication.md +++ b/site2/docs/security-tls-authentication.md @@ -8,8 +8,6 @@ sidebar_label: "Authentication using TLS" TLS authentication is an extension of [TLS transport encryption](security-tls-transport.md). Not only servers have keys and certs that the client uses to verify the identity of servers, clients also have keys and certs that the server uses to verify the identity of clients. You must have TLS transport encryption configured on your cluster before you can use TLS authentication. This guide assumes you already have TLS transport encryption configured. -`Bouncy Castle Provider` provides TLS related cipher suites and algorithms in Pulsar. If you need [FIPS](https://www.bouncycastle.org/fips_faq.html) version of `Bouncy Castle Provider`, please reference [Bouncy Castle page](security-bouncy-castle.md). - ### Create client certificates Client certificates are generated using the certificate authority. Server certificates are also generated with the same certificate authority. @@ -87,11 +85,6 @@ To configure brokers to authenticate clients, add the following parameters to `b authenticationEnabled=true authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls -# operations and publish/consume from all topics -superUserRoles=admin - -# Authentication settings of the broker itself. Used when the broker connects to other brokers, either in same or other clusters -brokerClientTlsEnabled=true brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls brokerClientAuthenticationParameters={"tlsCertFile":"/path/my-ca/admin.cert.pem","tlsKeyFile":"/path/my-ca/admin.key-pk8.pem"} brokerClientTrustCertsFilePath=/path/my-ca/certs/ca.cert.pem @@ -124,15 +117,10 @@ When you use TLS authentication, client connects via TLS transport. You need to [Command-line tools](reference-cli-tools.md) like [`pulsar-admin`](/tools/pulsar-admin/), [`pulsar-perf`](reference-cli-tools.md#pulsar-perf), and [`pulsar-client`](reference-cli-tools.md#pulsar-client) use the `conf/client.conf` config file in a Pulsar installation. -You need to add the following parameters to that file to use TLS authentication with the CLI tools of Pulsar: +To use TLS authentication with the CLI tools of Pulsar, you need to add the following parameters to the `conf/client.conf` file, alongside [the configuration to enable TLS transport](security-tls-transport.md#cli-tools): ```properties -webServiceUrl=https://broker.example.com:8443/ -brokerServiceUrl=pulsar+ssl://broker.example.com:6651/ -useTls=true -tlsAllowInsecureConnection=false -tlsTrustCertsFilePath=/path/to/ca.cert.pem authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls authParams=tlsCertFile:/path/to/my-role.cert.pem,tlsKeyFile:/path/to/my-role.key-pk8.pem @@ -146,7 +134,6 @@ import org.apache.pulsar.client.api.PulsarClient; PulsarClient client = PulsarClient.builder() .serviceUrl("pulsar+ssl://broker.example.com:6651/") - .enableTls(true) .tlsTrustCertsFilePath("/path/to/ca.cert.pem") .authentication("org.apache.pulsar.client.impl.auth.AuthenticationTls", "tlsCertFile:/path/to/my-role.cert.pem,tlsKeyFile:/path/to/my-role.key-pk8.pem") diff --git a/site2/docs/security-tls-keystore.md b/site2/docs/security-tls-keystore.md index d655290a2cbb9f..775415aab4b210 100644 --- a/site2/docs/security-tls-keystore.md +++ b/site2/docs/security-tls-keystore.md @@ -85,6 +85,7 @@ The next step is to sign all certificates in the keystore with the CA we generat ```shell keytool -keystore broker.keystore.jks -alias localhost -certreq -file cert-file +keytool -keystore client.keystore.jks -alias localhost -certreq -file cert-file ``` @@ -103,6 +104,8 @@ Finally, you need to import both the certificate of the CA and the signed certif keytool -keystore broker.keystore.jks -alias CARoot -import -file ca-cert keytool -keystore broker.keystore.jks -alias localhost -import -file cert-signed +keytool -keystore client.keystore.jks -alias CARoot -import -file ca-cert +keytool -keystore client.keystore.jks -alias localhost -import -file cert-signed ``` The definitions of the parameters are the following: @@ -124,7 +127,10 @@ If `tlsRequireTrustedClientCertOnConnect` is `true`, broker will reject the Conn The following TLS configs are needed on the broker side: ```properties +brokerServicePortTls=6651 +webServicePortTls=8081 +tlsRequireTrustedClientCertOnConnect=true tlsEnabledWithKeyStore=true # key store tlsKeyStoreType=JKS @@ -142,6 +148,9 @@ brokerClientTlsEnabledWithKeyStore=true brokerClientTlsTrustStoreType=JKS brokerClientTlsTrustStore=/var/private/tls/client.truststore.jks brokerClientTlsTrustStorePassword=clientpw +brokerClientTlsKeyStoreType=JKS +brokerClientTlsKeyStore=/var/private/tls/client.keystore.jks +brokerClientTlsKeyStorePassword=clientpw ``` @@ -193,7 +202,10 @@ For example: tlsTrustStoreType=JKS tlsTrustStorePath=/var/private/tls/client.truststore.jks tlsTrustStorePassword=clientpw - + tlsKeyStoreType=JKS + tlsKeyStorePath=/var/private/tls/client.keystore.jks + keyStorePassword=clientpw + ``` 1. for java client @@ -204,11 +216,15 @@ For example: PulsarClient client = PulsarClient.builder() .serviceUrl("pulsar+ssl://broker.example.com:6651/") - .enableTls(true) .useKeyStoreTls(true) + .tlsTrustStoreType("JKS") .tlsTrustStorePath("/var/private/tls/client.truststore.jks") .tlsTrustStorePassword("clientpw") - .allowTlsInsecureConnection(false) + .tlsKeyStoreType("JKS") + .tlsKeyStorePath("/var/private/tls/client.keystore.jks") + .tlsKeyStorePassword("clientpw") + .enableTlsHostnameVerification(false) // false by default, in any case + .allowTlsInsecureConnection(false) // false by default, in any case .build(); ``` @@ -218,10 +234,14 @@ For example: ```java PulsarAdmin amdin = PulsarAdmin.builder().serviceHttpUrl("https://broker.example.com:8443") - .useKeyStoreTls(true) + .tlsTrustStoreType("JKS") .tlsTrustStorePath("/var/private/tls/client.truststore.jks") .tlsTrustStorePassword("clientpw") - .allowTlsInsecureConnection(false) + .tlsKeyStoreType("JKS") + .tlsKeyStorePath("/var/private/tls/client.keystore.jks") + .tlsKeyStorePassword("clientpw") + .enableTlsHostnameVerification(false) // false by default, in any case + .allowTlsInsecureConnection(false) // false by default, in any case .build(); ``` @@ -242,12 +262,9 @@ This similar to [TLS authentication with PEM type](security-tls-authentication.m authenticationEnabled=true authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls -# this should be the CN for one of client keystore. -superUserRoles=admin - # Enable KeyStore type tlsEnabledWithKeyStore=true -requireTrustedClientCertOnConnect=true +tlsRequireTrustedClientCertOnConnect=true # key store tlsKeyStoreType=JKS @@ -268,8 +285,6 @@ brokerClientTlsTrustStorePassword=clientpw # internal auth config brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls brokerClientAuthenticationParameters={"keyStoreType":"JKS","keyStorePath":"/var/private/tls/client.keystore.jks","keyStorePassword":"clientpw"} -# currently websocket not support keystore type -webSocketServiceEnabled=false ``` @@ -289,7 +304,7 @@ For example: tlsTrustStorePath=/var/private/tls/client.truststore.jks tlsTrustStorePassword=clientpw authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls - authParams={"keyStoreType":"JKS","keyStorePath":"/path/to/keystorefile","keyStorePassword":"keystorepw"} + authParams={"keyStoreType":"JKS","keyStorePath":"/var/private/tls/client.keystore.jks","keyStorePassword":"clientpw"} ``` @@ -301,11 +316,11 @@ For example: PulsarClient client = PulsarClient.builder() .serviceUrl("pulsar+ssl://broker.example.com:6651/") - .enableTls(true) .useKeyStoreTls(true) .tlsTrustStorePath("/var/private/tls/client.truststore.jks") .tlsTrustStorePassword("clientpw") .allowTlsInsecureConnection(false) + .enableTlsHostnameVerification(false) .authentication( "org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls", "keyStoreType:JKS,keyStorePath:/var/private/tls/client.keystore.jks,keyStorePassword:clientpw") @@ -322,6 +337,7 @@ For example: .tlsTrustStorePath("/var/private/tls/client.truststore.jks") .tlsTrustStorePassword("clientpw") .allowTlsInsecureConnection(false) + .enableTlsHostnameVerification(false) .authentication( "org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls", "keyStoreType:JKS,keyStorePath:/var/private/tls/client.keystore.jks,keyStorePassword:clientpw") diff --git a/site2/docs/security-tls-transport.md b/site2/docs/security-tls-transport.md index f25898c47e0520..881563a31e5953 100644 --- a/site2/docs/security-tls-transport.md +++ b/site2/docs/security-tls-transport.md @@ -28,7 +28,9 @@ For TLS transport encryption, the clients can use the **trust cert** to verify t For TLS authentication, the server uses the **trust cert** to verify that the client has a key pair that the certificate authority signed. The common name of the **client cert** is then used as the client's role token (see [Overview](security-overview.md)). -`Bouncy Castle Provider` provides cipher suites and algorithms in Pulsar. If you need [FIPS](https://www.bouncycastle.org/fips_faq.html) version of `Bouncy Castle Provider`, please reference [Bouncy Castle page](security-bouncy-castle.md). +Pulsar uses [netty-tcnative](https://github.com/netty/netty-tcnative) and [Conscrypt](https://github.com/google/conscrypt) as security providers. There are two certificate formats: +* Java KeyStore(JKS): Pulsar uses Conscrypt by default for both broker service and Web service. +* CAcerts: Pulsar uses netty-tcnative by default, which includes two implementations, OpenSSL (default) and JDK. When OpenSSL is unavailable, JDK is used. ## Create TLS certificates @@ -144,6 +146,46 @@ openssl ca -config openssl.cnf -extensions server_cert \ At this point, you have a cert, `broker.cert.pem`, and a key, `broker.key-pk8.pem`, which you can use along with `ca.cert.pem` to configure TLS transport encryption for your broker and proxy nodes. +### Client certificate + +1. Enter the command below to generate the key. + +```bash + +openssl genrsa -out client.key.pem 2048 + +``` + +The client expects the key to be in [PKCS 8](https://en.wikipedia.org/wiki/PKCS_8) format, so enter the following command to convert it. + +```bash + +openssl pkcs8 -topk8 -inform PEM -outform PEM \ + -in client.key.pem -out client.key-pk8.pem -nocrypt + +``` + +2. Enter the following command to generate the certificate request. + +```bash + +openssl req -config openssl.cnf \ + -key client.key.pem -new -sha256 -out client.csr.pem + +``` + +3. Sign it with the certificate authority by entering the command below. + +```bash + +openssl ca -config openssl.cnf -extensions client_cert \ + -days 1000 -notext -md sha256 \ + -in client.csr.pem -out client.cert.pem + +``` + +At this point, you have a cert `client.cert.pem` and a key `client.key-pk8.pem`, which you can use along with `ca.cert.pem` to configure TLS encryption for your client. + ## Configure broker To configure a Pulsar [broker](reference-terminology.md#broker) to use TLS transport encryption, you need to make some changes to `broker.conf`, which locates in the `conf` directory of your [Pulsar installation](getting-started-standalone.md). @@ -159,6 +201,10 @@ tlsCertificateFilePath=/path/to/broker.cert.pem tlsKeyFilePath=/path/to/broker.key-pk8.pem tlsTrustCertsFilePath=/path/to/ca.cert.pem +brokerClientTlsEnabled=true +brokerClientTrustCertsFilePath=/path/to/ca.cert.pem +brokerClientCertificateFilePath=/path/to/client.cert.pem +brokerClientKeyFilePath=/path/to/client.key-pk8.pem ``` > You can find a full list of parameters available in the `conf/broker.conf` file, @@ -189,8 +235,11 @@ Proxies need to configure TLS in two directions, for clients connecting to the p ```properties +servicePortTls=6651 +webServicePortTls=8081 + # For clients connecting to the proxy -tlsEnabledInProxy=true +tlsRequireTrustedClientCertOnConnect=true tlsCertificateFilePath=/path/to/broker.cert.pem tlsKeyFilePath=/path/to/broker.key-pk8.pem tlsTrustCertsFilePath=/path/to/ca.cert.pem @@ -198,7 +247,8 @@ tlsTrustCertsFilePath=/path/to/ca.cert.pem # For the proxy to connect to brokers tlsEnabledWithBroker=true brokerClientTrustCertsFilePath=/path/to/ca.cert.pem - +brokerClientCertificateFilePath=/path/to/client.cert.pem +brokerClientKeyFilePath=/path/to/client.key-pk8.pem ``` ## Client configuration @@ -227,9 +277,10 @@ You need to add the following parameters to that file to use TLS transport with webServiceUrl=https://broker.example.com:8443/ brokerServiceUrl=pulsar+ssl://broker.example.com:6651/ -useTls=true tlsAllowInsecureConnection=false tlsTrustCertsFilePath=/path/to/ca.cert.pem +tlsKeyFilePath=/path/to/client.key-pk8.pem +tlsCertFile=/path/to/client-cert.pem tlsEnableHostnameVerification=false ``` @@ -242,7 +293,8 @@ import org.apache.pulsar.client.api.PulsarClient; PulsarClient client = PulsarClient.builder() .serviceUrl("pulsar+ssl://broker.example.com:6651/") - .enableTls(true) + .tlsKeyFilePath("/path/to/client.key-pk8.pem") + .tlsCertificateFilePath("/path/to/client.cert.pem") .tlsTrustCertsFilePath("/path/to/ca.cert.pem") .enableTlsHostnameVerification(false) // false by default, in any case .allowTlsInsecureConnection(false) // false by default, in any case