Default to Key, Value based Access Logging to STDOUT

[arkodg]

Description:

Default to the JSON format for Access Logs written to STDOUT
https://www.envoyproxy.io/docs/envoy/latest/configuration/observability/access_log/usage#format-dictionaries

this reduces the burden on the end user to parse the text based access logs enabled by default today which dont have the field name printed beside it
https://www.envoyproxy.io/docs/envoy/latest/configuration/observability/access_log/usage#default-format-string
which was enabled with #704

[arkodg]

thoughts @envoyproxy/gateway-maintainers ?

[tmsnan]

In our gateway, we use the format type to directly output the json type to replace the json_format type, because when we actually use the json_format type, the performance is very poor. Here is an example of using

                  - name: envoy.access_loggers.file
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
                      path: /dev/stdout
                      format: "{\"start_time\":\"%START_TIME%\",\"method\":\"%REQ(:METHOD)%\",\"x-envoy-origin-path\":\"%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%\",\"protocol\":\"%PROTOCOL%\",\"response_code\":\"%RESPONSE_CODE%\",\"response_flags\":\"%RESPONSE_FLAGS%\",\"response_code_details\":\"%RESPONSE_CODE_DETAILS%\",\"connection_termination_details\":\"%CONNECTION_TERMINATION_DETAILS%\",\"upstream_transport_failure_reason\":\"%UPSTREAM_TRANSPORT_FAILURE_REASON%\",\"bytes_received\":\"%BYTES_RECEIVED%\",\"bytes_sent\":\"%BYTES_SENT%\",\"duration\":\"%DURATION%\",\"x-envoy-upstream-service-time\":\"%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%\",\"x-forwarded-for\":\"%REQ(X-FORWARDED-FOR)%\",\"user-agent\":\"%REQ(USER-AGENT)%\",\"x-request-id\":\"%REQ(X-REQUEST-ID)%\",\":authority\":\"%REQ(:AUTHORITY)%\",\"upstream_host\":\"%UPSTREAM_HOST%\",\"upstream_cluster\":\"%UPSTREAM_CLUSTER%\",\"upstream_local_address\":\"%UPSTREAM_LOCAL_ADDRESS%\",\"downstream_local_address\":\"%DOWNSTREAM_LOCAL_ADDRESS%\",\"downstream_remote_address\":\"%DOWNSTREAM_REMOTE_ADDRESS%\",\"requested_server_name\":\"%REQUESTED_SERVER_NAME%\",\"route_name\":\"%ROUTE_NAME%\"}\n"

[arkodg]

haha @tmsnan, nice workaround :) , can you share any relevant upstream link that highlights the poor performance

[tmsnan]

@arkodg ，field data was not saved at the time. But this scene is easy to compare. If necessary, I will try to compare it locally. The reason is that the json format output needs to be serialized, while the text only needs to be simply spliced ​​according to the format

[haq204]

I would prefer keeping the default to text due to the serialization cost @tmsnan mentioned. I don't think I've ever seen json format set as the default for logs in other places and probably for good reason. logs can be voluminous so the serialization can add a lot of extra CPU cycles/memory util and I'm not sure that's a cost I want to enable by default to users.

[arkodg]

related upstream issue envoyproxy/envoy#10415

[arkodg]

chatted with @cpakulski who confirmed the extra cost for serialization when using json format, so I'm a
+1 to @tmsnan's approach of using text format, but representing the text as a json string as shown here.
which can be used to update the defaults set here

gateway/internal/xds/translator/accesslog.go

Line 28 in 0775da1

// EnvoyTextLogFormat is the default log format for Envoy.

wdyt @AliceProxy @zirain @Xunzhuo ?

[zirain]

I will be -1 to use it as default.
AFAIK, fluent-bit/istio use a plain text as default, prefer to same thing.

[arkodg]

im personally tired of staring at

[2016-04-15T20:17:00.310Z] "POST /api/v1/locations HTTP/2" 204 - 154 0 226 100 "10.0.35.28"
"nsq2http" "cc21d9b0-cf5c-432b-8c7e-98aeb7988cd2" "locations" "tcp://10.0.2.1:80"

and figuring out which field maps to which attribute.
Offline tooling for parsing these logs and creating the attribute mappings exist, and so does the ability to change the format, but sometimes its faster to get to the root cause by directly looking at the pod logs, which might be useful for most imo

[zirain]

That's why EG offer the ability to customize

[tmsnan]

I think changing to json string type by default is more friendly to novices.

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days.

[arkodg]

had a chat with @zirain who is not completely opposed to this :), but worries about perf .
I'd prefer if we tried this out sooner than later and can always revert if perf drops significantly.

Hoping we can get @tmsnan's suggestion in #1446 (comment)
@tmsnan can you help with this one :)

[arkodg]

assigning this to myself