From a532bac914c1df9077a29a0556fbac7ca4d39348 Mon Sep 17 00:00:00 2001 From: withlin Date: Wed, 4 Dec 2024 19:35:40 +0800 Subject: [PATCH] fix: missing markdown Signed-off-by: withlin Signed-off-by: withlin --- docs/configuration/plugins/outputs/kafka.md | 178 ++++++++++---------- 1 file changed, 89 insertions(+), 89 deletions(-) diff --git a/docs/configuration/plugins/outputs/kafka.md b/docs/configuration/plugins/outputs/kafka.md index ad73f79f5..a23b83c09 100644 --- a/docs/configuration/plugins/outputs/kafka.md +++ b/docs/configuration/plugins/outputs/kafka.md @@ -265,7 +265,7 @@ Username when using PLAIN/SCRAM SASL authentication ## RdkafkaOptions -GlobalConfig represents the global configuration properties for librdkafka. +RdkafkaOptions represents the global configuration properties for librdkafka. ### allow.auto.create.topics (bool, optional) {#rdkafkaoptions-allow.auto.create.topics} @@ -352,14 +352,14 @@ Default topic configuration for automatically subscribed topics If enabled librdkafka will initialize the PRNG with srand(current_time.milliseconds) on the first invocation of rd_kafka_new() (required only if rand_r() is not available on your platform). If disabled the application must call srand() prior to calling rd_kafka_new(). -### enable.sasl.oauthbearer.unsecure.jwt (bool, optional) {#rdkafkaoptions-enable.sasl.oauthbearer.unsecure.jwt} +### enable.ssl.certificate.verification (bool, optional) {#rdkafkaoptions-enable.ssl.certificate.verification} -Enable the builtin unsecure JWT OAUTHBEARER token handler if no oauthbearer_refresh_cb has been set. This builtin handler should only be used for development or testing, and not in production. +Enable OpenSSL's builtin broker (server) certificate verification. This verification can be extended by the application by implementing a certificate_verify_cb. -### enable.ssl.certificate.verification (bool, optional) {#rdkafkaoptions-enable.ssl.certificate.verification} +### enable.sasl.oauthbearer.unsecure.jwt (bool, optional) {#rdkafkaoptions-enable.sasl.oauthbearer.unsecure.jwt} -Enable OpenSSL's builtin broker (server) certificate verification. This verification can be extended by the application by implementing a certificate_verify_cb. +Enable the builtin unsecure JWT OAUTHBEARER token handler if no oauthbearer_refresh_cb has been set. This builtin handler should only be used for development or testing, and not in production. ### enabled_events (int, optional) {#rdkafkaoptions-enabled_events} @@ -477,214 +477,214 @@ The initial time to wait before reconnecting to a broker after the connection ha Address resolution callback (set with rd_kafka_conf_set_resolve_cb()) -### sasl.kerberos.keytab (string, optional) {#rdkafkaoptions-sasl.kerberos.keytab} +### ssl.ca.location (string, optional) {#rdkafkaoptions-ssl.ca.location} -Path to Kerberos keytab file. This configuration property is only used as a variable in sasl.kerberos.kinit.cmd as ... -t "%{sasl.kerberos.keytab}". +File or directory path to CA certificate(s) for verifying the broker's key. Defaults: On Windows the system's CA certificates are automatically looked up in the Windows Root certificate store. On Mac OSX this configuration defaults to `probe`. It is recommended to install openssl using Homebrew, to provide CA certificates. On Linux install the distribution's ca-certificates package. If OpenSSL is statically linked or `ssl.ca.location` is set to `probe` a list of standard paths will be probed and the first one found will be used as the default CA certificate location path. If OpenSSL is dynamically linked the OpenSSL library's default path will be used (see `OPENSSLDIR` in `openssl version -a`). -### sasl.kerberos.kinit.cmd (string, optional) {#rdkafkaoptions-sasl.kerberos.kinit.cmd} +### ssl.ca.pem (string, optional) {#rdkafkaoptions-ssl.ca.pem} -Shell command to refresh or acquire the client's Kerberos ticket. This command is executed on client creation and every sasl.kerberos.min.time.before.relogin (0=disable). +CA certificate string (PEM format) for verifying the broker's key. -### sasl.kerberos.min.time.before.relogin (int, optional) {#rdkafkaoptions-sasl.kerberos.min.time.before.relogin} +### ssl.certificate.location (string, optional) {#rdkafkaoptions-ssl.certificate.location} -Minimum time in milliseconds between key refresh attempts. Disable automatic key refresh by setting this property to 0. +Path to client's public key (PEM) used for authentication. -### sasl.kerberos.principal (string, optional) {#rdkafkaoptions-sasl.kerberos.principal} +### ssl.certificate.pem (string, optional) {#rdkafkaoptions-ssl.certificate.pem} -This client's Kerberos principal name. (Not supported on Windows, will use the logon user's principal). +Client's public key string (PEM format) used for authentication. -### sasl.kerberos.service.name (string, optional) {#rdkafkaoptions-sasl.kerberos.service.name} +### ssl.cipher.suites (string, optional) {#rdkafkaoptions-ssl.cipher.suites} -Kerberos principal name that Kafka runs as, not including /hostname@REALM. +A cipher suite is a named combination of authentication, encryption, MAC and key exchange algorithm used to negotiate the security settings for a network connection using TLS or SSL network protocol. See manual page for `ciphers(1)` and `SSL_CTX_set_cipher_list(3). -### sasl.mechanisms (string, optional) {#rdkafkaoptions-sasl.mechanisms} +### ssl.crl.location (string, optional) {#rdkafkaoptions-ssl.crl.location} -SASL mechanism to use for authentication. Supported: GSSAPI, PLAIN, SCRAM-SHA-256, SCRAM-SHA-512, OAUTHBEARER. NOTE: Despite the name only one mechanism must be configured. +Path to CRL for verifying broker's certificate validity. -### sasl.oauthbearer.client.id (string, optional) {#rdkafkaoptions-sasl.oauthbearer.client.id} +### ssl.curves.list (string, optional) {#rdkafkaoptions-ssl.curves.list} -Public identifier for the application. Must be unique across all clients that the authorization server handles. Only used when sasl.oauthbearer.method is set to "oidc". +The supported-curves extension in the TLS ClientHello message specifies the curves (standard/named, or 'explicit' GF(2^k) or GF(p)) the client is willing to have the server use. See manual page for `SSL_CTX_set1_curves_list(3)`. OpenSSL >= 1.0.2 required. -### sasl.oauthbearer.client.secret (string, optional) {#rdkafkaoptions-sasl.oauthbearer.client.secret} +### ssl.endpoint.identification.algorithm (string, optional) {#rdkafkaoptions-ssl.endpoint.identification.algorithm} -Client secret only known to the application and the authorization server. This should be a sufficiently random string that is not guessable. Only used when sasl.oauthbearer.method is set to "oidc". +Endpoint identification algorithm to validate broker hostname using broker certificate. https - Server (broker) hostname verification as specified in RFC2818. none - No endpoint verification. OpenSSL >= 1.0.2 required. -### sasl.oauthbearer.config (string, optional) {#rdkafkaoptions-sasl.oauthbearer.config} +### ssl.engine.id (string, optional) {#rdkafkaoptions-ssl.engine.id} -SASL/OAUTHBEARER configuration. The format is implementation-dependent and must be parsed accordingly. The default unsecured token implementation (see https://tools.ietf.org/html/rfc7515#appendix-A.5) recognizes space-separated name=value pairs with valid names including principalClaimName, principal, scopeClaimName, scope, and lifeSeconds. The default value for principalClaimName is "sub", the default value for scopeClaimName is "scope", and the default value for lifeSeconds is 3600. The scope value is CSV format with the default value being no/empty scope. For example: principalClaimName=azp principal=admin scopeClaimName=roles scope=role1,role2 lifeSeconds=600. In addition, SASL extensions can be communicated to the broker via extension_NAME=value. For example: principal=admin extension_traceId=123. +OpenSSL engine id is the name used for loading engine. -### sasl.oauthbearer.extensions (string, optional) {#rdkafkaoptions-sasl.oauthbearer.extensions} +### ssl.engine.location (string, optional) {#rdkafkaoptions-ssl.engine.location} -Allow additional information to be provided to the broker. Comma-separated list of key=value pairs. E.g., "supportFeatureX=true,organizationId=sales-emea".Only used when sasl.oauthbearer.method is set to "oidc". +**DEPRECATED** Path to OpenSSL engine library. OpenSSL >= 1.1.x required. DEPRECATED: OpenSSL engine support is deprecated and should be replaced by OpenSSL 3 providers. -### sasl.oauthbearer.method (string, optional) {#rdkafkaoptions-sasl.oauthbearer.method} +### ssl.key.location (string, optional) {#rdkafkaoptions-ssl.key.location} -Set to "default" or "oidc" to control which login method to be used. If set to "oidc", the following properties must also be specified: sasl.oauthbearer.client.id, sasl.oauthbearer.client.secret, and sasl.oauthbearer.token.endpoint.url. +Path to client's private key (PEM) used for authentication. -### sasl.oauthbearer.scope (string, optional) {#rdkafkaoptions-sasl.oauthbearer.scope} +### ssl.key.password (string, optional) {#rdkafkaoptions-ssl.key.password} -Client use this to specify the scope of the access request to the broker. Only used when sasl.oauthbearer.method is set to "oidc". +Private key passphrase (for use with `ssl.key.location` and `set_ssl_cert()`). -### sasl.oauthbearer.token.endpoint.url (string, optional) {#rdkafkaoptions-sasl.oauthbearer.token.endpoint.url} +### ssl.key.pem (string, optional) {#rdkafkaoptions-ssl.key.pem} -OAuth/OIDC issuer token endpoint HTTP(S) URI used to retrieve token. Only used when sasl.oauthbearer.method is set to "oidc". +Client's private key string (PEM format) used for authentication. -### sasl.password (string, optional) {#rdkafkaoptions-sasl.password} +### ssl.keystore.location (string, optional) {#rdkafkaoptions-ssl.keystore.location} -SASL password for use with the PLAIN and SASL-SCRAM-.. mechanism. +Path to client's keystore (PKCS#12) used for authentication. -### sasl.username (string, optional) {#rdkafkaoptions-sasl.username} +### ssl.keystore.password (string, optional) {#rdkafkaoptions-ssl.keystore.password} -SASL username for use with the PLAIN and SASL-SCRAM-.. mechanisms. +Client's keystore (PKCS#12) password. -### security.protocol (string, optional) {#rdkafkaoptions-security.protocol} +### ssl.providers (string, optional) {#rdkafkaoptions-ssl.providers} -Protocol used to communicate with brokers. +Comma-separated list of OpenSSL 3.0.x implementation providers. E.g., "default,legacy". -### socket.blocking.max.ms (int, optional) {#rdkafkaoptions-socket.blocking.max.ms} +### ssl.sigalgs.list (string, optional) {#rdkafkaoptions-ssl.sigalgs.list} -DEPRECATED No longer used. +The client uses the TLS ClientHello signature_algorithms extension to indicate to the server which signature/hash algorithm pairs may be used in digital signatures. See manual page for `SSL_CTX_set1_sigalgs_list(3)`. OpenSSL >= 1.0.2 required. -### socket_cb (string, optional) {#rdkafkaoptions-socket_cb} +### sasl.kerberos.keytab (string, optional) {#rdkafkaoptions-sasl.kerberos.keytab} -Socket creation callback to provide race-free CLOEXEC +Path to Kerberos keytab file. This configuration property is only used as a variable in sasl.kerberos.kinit.cmd as ... -t "%{sasl.kerberos.keytab}". -### socket.connection.setup.timeout.ms (int, optional) {#rdkafkaoptions-socket.connection.setup.timeout.ms} +### sasl.kerberos.kinit.cmd (string, optional) {#rdkafkaoptions-sasl.kerberos.kinit.cmd} -Maximum time allowed for broker connection setup (TCP connection setup as well SSL and SASL handshake). If the connection to the broker is not fully functional after this the connection will be closed and retried. +Shell command to refresh or acquire the client's Kerberos ticket. This command is executed on client creation and every sasl.kerberos.min.time.before.relogin (0=disable). -### socket.keepalive.enable (bool, optional) {#rdkafkaoptions-socket.keepalive.enable} +### sasl.kerberos.min.time.before.relogin (int, optional) {#rdkafkaoptions-sasl.kerberos.min.time.before.relogin} -Enable TCP keep-alives (SO_KEEPALIVE) on broker sockets +Minimum time in milliseconds between key refresh attempts. Disable automatic key refresh by setting this property to 0. -### socket.max.fails (int, optional) {#rdkafkaoptions-socket.max.fails} +### sasl.kerberos.principal (string, optional) {#rdkafkaoptions-sasl.kerberos.principal} -Disconnect from broker when this number of send failures (e.g., timed out requests) is reached. Disable with 0. WARNING: It is highly recommended to leave this setting at its default value of 1 to avoid the client and broker to become desynchronized in case of request timeouts. NOTE: The connection is automatically re-established. +This client's Kerberos principal name. (Not supported on Windows, will use the logon user's principal). -### socket.nagle.disable (bool, optional) {#rdkafkaoptions-socket.nagle.disable} +### sasl.kerberos.service.name (string, optional) {#rdkafkaoptions-sasl.kerberos.service.name} -Disable the Nagle algorithm (TCP_NODELAY) on broker sockets. +Kerberos principal name that Kafka runs as, not including /hostname@REALM. -### socket.receive.buffer.bytes (int, optional) {#rdkafkaoptions-socket.receive.buffer.bytes} +### sasl.mechanisms (string, optional) {#rdkafkaoptions-sasl.mechanisms} -Broker socket receive buffer size. System default is used if 0. +SASL mechanism to use for authentication. Supported: GSSAPI, PLAIN, SCRAM-SHA-256, SCRAM-SHA-512, OAUTHBEARER. NOTE: Despite the name only one mechanism must be configured. -### socket.send.buffer.bytes (int, optional) {#rdkafkaoptions-socket.send.buffer.bytes} +### sasl.oauthbearer.client.id (string, optional) {#rdkafkaoptions-sasl.oauthbearer.client.id} -Broker socket send buffer size. System default is used if 0. +Public identifier for the application. Must be unique across all clients that the authorization server handles. Only used when sasl.oauthbearer.method is set to "oidc". -### socket.timeout.ms (int, optional) {#rdkafkaoptions-socket.timeout.ms} +### sasl.oauthbearer.client.secret (string, optional) {#rdkafkaoptions-sasl.oauthbearer.client.secret} -Default timeout for network requests. Producer: ProduceRequests will use the lesser value of `socket.timeout.ms` and remaining `message.timeout.ms` for the first message in the batch. Consumer: FetchRequests will use `fetch.wait.max.ms` + `socket.timeout.ms`. Admin: Admin requests will use `socket.timeout.ms` or explicitly set `rd_kafka_AdminOptions_set_operation_timeout()` value. +Client secret only known to the application and the authorization server. This should be a sufficiently random string that is not guessable. Only used when sasl.oauthbearer.method is set to "oidc". -### ssl.ca.location (string, optional) {#rdkafkaoptions-ssl.ca.location} +### sasl.oauthbearer.config (string, optional) {#rdkafkaoptions-sasl.oauthbearer.config} -File or directory path to CA certificate(s) for verifying the broker's key. Defaults: On Windows the system's CA certificates are automatically looked up in the Windows Root certificate store. On Mac OSX this configuration defaults to `probe`. It is recommended to install openssl using Homebrew, to provide CA certificates. On Linux install the distribution's ca-certificates package. If OpenSSL is statically linked or `ssl.ca.location` is set to `probe` a list of standard paths will be probed and the first one found will be used as the default CA certificate location path. If OpenSSL is dynamically linked the OpenSSL library's default path will be used (see `OPENSSLDIR` in `openssl version -a`). +SASL/OAUTHBEARER configuration. The format is implementation-dependent and must be parsed accordingly. The default unsecured token implementation (see https://tools.ietf.org/html/rfc7515#appendix-A.5) recognizes space-separated name=value pairs with valid names including principalClaimName, principal, scopeClaimName, scope, and lifeSeconds. The default value for principalClaimName is "sub", the default value for scopeClaimName is "scope", and the default value for lifeSeconds is 3600. The scope value is CSV format with the default value being no/empty scope. For example: principalClaimName=azp principal=admin scopeClaimName=roles scope=role1,role2 lifeSeconds=600. In addition, SASL extensions can be communicated to the broker via extension_NAME=value. For example: principal=admin extension_traceId=123. -### ssl.ca.pem (string, optional) {#rdkafkaoptions-ssl.ca.pem} +### sasl.oauthbearer.extensions (string, optional) {#rdkafkaoptions-sasl.oauthbearer.extensions} -CA certificate string (PEM format) for verifying the broker's key. +Allow additional information to be provided to the broker. Comma-separated list of key=value pairs. E.g., "supportFeatureX=true,organizationId=sales-emea".Only used when sasl.oauthbearer.method is set to "oidc". -### ssl.certificate.location (string, optional) {#rdkafkaoptions-ssl.certificate.location} +### sasl.oauthbearer.method (string, optional) {#rdkafkaoptions-sasl.oauthbearer.method} -Path to client's public key (PEM) used for authentication. +Set to "default" or "oidc" to control which login method to be used. If set to "oidc", the following properties must also be specified: sasl.oauthbearer.client.id, sasl.oauthbearer.client.secret, and sasl.oauthbearer.token.endpoint.url. -### ssl.certificate.pem (string, optional) {#rdkafkaoptions-ssl.certificate.pem} +### sasl.oauthbearer.scope (string, optional) {#rdkafkaoptions-sasl.oauthbearer.scope} -Client's public key string (PEM format) used for authentication. +Client use this to specify the scope of the access request to the broker. Only used when sasl.oauthbearer.method is set to "oidc". -### ssl.cipher.suites (string, optional) {#rdkafkaoptions-ssl.cipher.suites} +### sasl.oauthbearer.token.endpoint.url (string, optional) {#rdkafkaoptions-sasl.oauthbearer.token.endpoint.url} -A cipher suite is a named combination of authentication, encryption, MAC and key exchange algorithm used to negotiate the security settings for a network connection using TLS or SSL network protocol. See manual page for `ciphers(1)` and `SSL_CTX_set_cipher_list(3). +OAuth/OIDC issuer token endpoint HTTP(S) URI used to retrieve token. Only used when sasl.oauthbearer.method is set to "oidc". -### ssl.crl.location (string, optional) {#rdkafkaoptions-ssl.crl.location} +### sasl.password (string, optional) {#rdkafkaoptions-sasl.password} -Path to CRL for verifying broker's certificate validity. +SASL password for use with the PLAIN and SASL-SCRAM-.. mechanism. -### ssl.curves.list (string, optional) {#rdkafkaoptions-ssl.curves.list} +### sasl.username (string, optional) {#rdkafkaoptions-sasl.username} -The supported-curves extension in the TLS ClientHello message specifies the curves (standard/named, or 'explicit' GF(2^k) or GF(p)) the client is willing to have the server use. See manual page for `SSL_CTX_set1_curves_list(3)`. OpenSSL >= 1.0.2 required. +SASL username for use with the PLAIN and SASL-SCRAM-.. mechanisms. -### ssl.endpoint.identification.algorithm (string, optional) {#rdkafkaoptions-ssl.endpoint.identification.algorithm} +### security.protocol (string, optional) {#rdkafkaoptions-security.protocol} -Endpoint identification algorithm to validate broker hostname using broker certificate. https - Server (broker) hostname verification as specified in RFC2818. none - No endpoint verification. OpenSSL >= 1.0.2 required. +Protocol used to communicate with brokers. -### ssl.engine.id (string, optional) {#rdkafkaoptions-ssl.engine.id} +### socket.blocking.max.ms (int, optional) {#rdkafkaoptions-socket.blocking.max.ms} -OpenSSL engine id is the name used for loading engine. +DEPRECATED No longer used. -### ssl.engine.location (string, optional) {#rdkafkaoptions-ssl.engine.location} +### socket_cb (string, optional) {#rdkafkaoptions-socket_cb} -**DEPRECATED** Path to OpenSSL engine library. OpenSSL >= 1.1.x required. DEPRECATED: OpenSSL engine support is deprecated and should be replaced by OpenSSL 3 providers. +Socket creation callback to provide race-free CLOEXEC -### ssl.key.location (string, optional) {#rdkafkaoptions-ssl.key.location} +### socket.connection.setup.timeout.ms (int, optional) {#rdkafkaoptions-socket.connection.setup.timeout.ms} -Path to client's private key (PEM) used for authentication. +Maximum time allowed for broker connection setup (TCP connection setup as well SSL and SASL handshake). If the connection to the broker is not fully functional after this the connection will be closed and retried. -### ssl.key.password (string, optional) {#rdkafkaoptions-ssl.key.password} +### socket.keepalive.enable (bool, optional) {#rdkafkaoptions-socket.keepalive.enable} -Private key passphrase (for use with `ssl.key.location` and `set_ssl_cert()`). +Enable TCP keep-alives (SO_KEEPALIVE) on broker sockets -### ssl.key.pem (string, optional) {#rdkafkaoptions-ssl.key.pem} +### socket.max.fails (int, optional) {#rdkafkaoptions-socket.max.fails} -Client's private key string (PEM format) used for authentication. +Disconnect from broker when this number of send failures (e.g., timed out requests) is reached. Disable with 0. WARNING: It is highly recommended to leave this setting at its default value of 1 to avoid the client and broker to become desynchronized in case of request timeouts. NOTE: The connection is automatically re-established. -### ssl.keystore.location (string, optional) {#rdkafkaoptions-ssl.keystore.location} +### socket.nagle.disable (bool, optional) {#rdkafkaoptions-socket.nagle.disable} -Path to client's keystore (PKCS#12) used for authentication. +Disable the Nagle algorithm (TCP_NODELAY) on broker sockets. -### ssl.keystore.password (string, optional) {#rdkafkaoptions-ssl.keystore.password} +### socket.receive.buffer.bytes (int, optional) {#rdkafkaoptions-socket.receive.buffer.bytes} -Client's keystore (PKCS#12) password. +Broker socket receive buffer size. System default is used if 0. -### ssl.providers (string, optional) {#rdkafkaoptions-ssl.providers} +### socket.send.buffer.bytes (int, optional) {#rdkafkaoptions-socket.send.buffer.bytes} -Comma-separated list of OpenSSL 3.0.x implementation providers. E.g., "default,legacy". +Broker socket send buffer size. System default is used if 0. -### ssl.sigalgs.list (string, optional) {#rdkafkaoptions-ssl.sigalgs.list} +### socket.timeout.ms (int, optional) {#rdkafkaoptions-socket.timeout.ms} -The client uses the TLS ClientHello signature_algorithms extension to indicate to the server which signature/hash algorithm pairs may be used in digital signatures. See manual page for `SSL_CTX_set1_sigalgs_list(3)`. OpenSSL >= 1.0.2 required. +Default timeout for network requests. Producer: ProduceRequests will use the lesser value of `socket.timeout.ms` and remaining `message.timeout.ms` for the first message in the batch. Consumer: FetchRequests will use `fetch.wait.max.ms` + `socket.timeout.ms`. Admin: Admin requests will use `socket.timeout.ms` or explicitly set `rd_kafka_AdminOptions_set_operation_timeout()` value. ### statistics.interval.ms (int, optional) {#rdkafkaoptions-statistics.interval.ms}