Allow for logfile naming like in Nomad

[Esity]

Feature Description

Currently consul logs files like this consul-{timestamp}.log
This is a pain to figure out the most recent log file. Sure we collect files and ship them out but sometimes it's useful to look inside /var/log

Nomad Example

ls -alh /var/log/nomad/
-rw-r-----.  1 root  root   51K Dec  6 01:50 nomad-1670313323123267918.log
-rw-r-----.  1 root  root   12K Dec  7 01:50 nomad-1670399732186160816.log
-rw-r-----.  1 root  root   16K Dec  7 09:55 nomad.log

But consul looks like this

ls -alh /var/log/consul/
drwxr-xr-x.  2 consul consul 4.0K Dec  7 08:34 .
drwxr-xr-x. 24 root   root   8.0K Dec  7 08:34 ..
-rw-r-----.  1 consul consul  39K Dec  1 11:35 consul-1669914326514455232.log
-rw-r-----.  1 consul consul 230K Dec  2 11:35 consul-1669916117380991503.log

Nomad Config Code
https://github.com/hashicorp/nomad/blob/1c9b4e398dd2b954dc0414b7c1568f04a3143355/command/agent/config.go#L66
https://github.com/hashicorp/nomad/blob/1c9b4e398dd2b954dc0414b7c1568f04a3143355/command/agent/config.go#L1326-L1328

Consul Config Code

consul/agent/config/config.go

Line 193 in 36f05bc

LogFile *string `mapstructure:"log_file" json:"log_file,omitempty"`

Use Case(s)

The use here is to make this easier to tail the most recent log file

[Esity]

CC @wjpatton

[jkirschner-hashicorp]

Linking to a related issue: #10976

[Globidev]

It seems to me that without that feature, one cannot properly integrate datadog log collection.

The link on their docs shows that the datadog agent configuration requires specifying a file path: https://docs.datadoghq.com/integrations/consul/?tab=host#log-collection

With the current naming scheme, I fail to see how we can always point to an up to date source of logs

[david-yu]

@Esity Just to clarify, nomad.log contains the most recent log entries prior to rotation? Are all the log files mutually exclusive or does the latest share entries with a timestamped log file?

[Esity]

Recent/new logs would go to consul.log
Once they are rotated based on the log settings in the hcl, then they go to consul-timestamp.log

Let me know if that makes sense

[david-yu]

@Esity There is a PR now up that will implement this could you review if this works for you? #18617

[gaelik]

@Esity There is a PR now up that will implement this could you review if this works for you? #18617

It is exactly what Matt and I were looking for, thank you for making our life easier!