You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Google Cloud Logging API requires strings to be UTF-8 encoded[1]. Log entries with invalid UTF-8 characters fail to be ingested when partial_success is enabled, otherwise the whole WriteLogEntries request is dropped.
The out_google_cloud plugin in fluentd provides a feature that converts invalid UTF-8 characters to replacement strings[2].
I propose to add the same feature available in out_stackdriver plugin.
Is your feature request related to a problem? Please describe.
Google Cloud Logging API requires strings to be UTF-8 encoded[1]. Log entries with invalid UTF-8 characters fail to be ingested when
partial_success
is enabled, otherwise the whole WriteLogEntries request is dropped.The out_google_cloud plugin in fluentd provides a feature that converts invalid UTF-8 characters to replacement strings[2].
I propose to add the same feature available in out_stackdriver plugin.
[1]https://cloud.google.com/logging/docs/reference/v2/rpc/google.logging.v2#google.logging.v2.LoggingServiceV2.WriteLogEntries
[2]https://github.com/GoogleCloudPlatform/fluent-plugin-google-cloud/blob/master/lib/fluent/plugin/out_google_cloud.rb#L1873C9-L1873C24
Describe the solution you'd like
To add two configuration parameters:
coerce_to_utf8
andnon_utf8_replacement_string
. The expected behaviour is:coerce_to_utf8
defaults to false,non_utf8_replacement_string
defaults to a space character ' '.coerce_to_utf8
is enabled, convert invalid UTF-8 characters to configured replacement string for all string fields in LogEntrycoerce_to_utf8
is disabled, skip the conversionDescribe alternatives you've considered
Additional context
The text was updated successfully, but these errors were encountered: