util/log: simplify the crdb-v1/v2 format code based on options

Release note (cli change): The `crdb-v1` log format now recognizes the format options `show-counter` and `colors`. The existing formats `crdb-v1-tty`, `crdb-v1-count`, `crdb-v1-tty-count` have been redefined to become aliases for `crdb-v1` with different defaults for the two new options. Release note (cli change): The `crdb-v2` log format now recognizes the format option `colors`. The existing formats `crdb-v2-tty` has been redefined to become aliases for `crdb-v2` with a different default for the new option.
cockroachdb · Jun 13, 2023 · fae003c · fae003c
1 parent 88da4b5
commit fae003c
Show file tree

Hide file tree

Showing 7 changed files with 237 additions and 253 deletions.
diff --git a/docs/generated/logformats.md b/docs/generated/logformats.md
@@ -26,23 +26,25 @@ The supported log output formats are documented below.
 
 ## Format `crdb-v1`
 
-This is the legacy file format used from CockroachDB v1.0.
 
-Each log entry is emitted using a common prefix, described below,
-followed by:
+This is a legacy file format used from CockroachDB v1.0.
+
+Each log entry is emitted using a common prefix, described below, followed by:
 
 - The logging context tags enclosed between `[` and `]`, if any. It is possible
   for this to be omitted if there were no context tags.
+- Optionally, a counter column, if the option 'show-counter' is enabled. See below for details.
 - the text of the log entry.
 
-Beware that the text of the log entry can span multiple lines. The following caveats apply:
-
+Beware that the text of the log entry can span multiple lines.
+The following caveats apply:
 
 - The text of the log entry can start with text enclosed between `[` and `]`.
   If there were no logging tags to start with, it is not possible to distinguish between
   logging context tag information and a `[...]` string in the main text of the
-  log entry. This means that this format is ambiguous. For an unambiguous alternative,
-  consider `crdb-v1-count`.
+  log entry. This means that this format is ambiguous.
+
+  To remove this ambiguity, you can use the option 'show-counter'.
 
 - The text of the log entry can embed arbitrary application-level strings,
   including strings that represent log entries. In particular, an accident
@@ -75,7 +77,10 @@ if known, and other metadata about the running process.
 
 Each line of output starts with the following prefix:
 
-     Lyymmdd hh:mm:ss.uuuuuu goid [chan@]file:line marker
+     Lyymmdd hh:mm:ss.uuuuuu goid [chan@]file:line marker tags counter
+
+Reminder, the tags may be omitted; and the counter is only printed if the option
+'show-counter' is specified.
 
 | Field           | Description                                                                                                                          |
 |-----------------|--------------------------------------------------------------------------------------------------------------------------------------|
@@ -89,6 +94,8 @@ Each line of output starts with the following prefix:
 | file            | The file name where the entry originated.                                                                                            |
 | line            | The line number where the entry originated.                                                                                          |
 | marker          | Redactability marker ` + redactableIndicator + ` (see below for details).                                                  |
+| tags            | The logging tags, enclosed between `[` and `]`. May be absent.                                                   |
+| counter         | The entry counter. Only included if 'show-counter' is enabled.                                                                       |
 
 The redactability marker can be empty; in this case, its position in the common prefix is
 a double ASCII space character which can be used to reliably identify this situation.
@@ -99,88 +106,41 @@ fields that are considered sensitive. These markers are automatically recognized
 by [`cockroach debug zip`](cockroach-debug-zip.html) and [`cockroach debug merge-logs`](cockroach-debug-merge-logs.html) when log redaction is requested.
 
 
-## Format `crdb-v1-count`
-
-This is an alternative, backward-compatible legacy file format used from CockroachDB v2.0.
-
-Each log entry is emitted using a common prefix, described below,
-followed by the text of the log entry.
-
-Beware that the text of the log entry can span multiple lines. The following caveats apply:
-
-
-- The text of the log entry can embed arbitrary application-level strings,
-  including strings that represent log entries. In particular, an accident
-  of implementation can cause the common entry prefix (described below)
-  to also appear on a line of its own, as part of the payload of a previous
-  log entry. There is no automated way to recognize when this occurs.
-  Care must be taken by a human observer to recognize these situations.
-
-- The log entry parser provided by CockroachDB to read log files is faulty
-  and is unable to recognize the aforementioned pitfall; nor can it read
-  entries larger than 64KiB successfully. Generally, use of this internal
-  log entry parser is discouraged for entries written with this format.
-
-See the newer format `crdb-v2` for an alternative
-without these limitations.
-
-### Header lines
-
-At the beginning of each file, a header is printed using a similar format as
-regular log entries. This header reports when the file was created,
-which parameters were used to start the server, the server identifiers
-if known, and other metadata about the running process.
-
-- This header appears to be logged at severity `INFO` (with an `I` prefix
-  at the start of the line) even though it does not really have a severity.
-- The header is printed unconditionally even when a filter is configured to
-  omit entries at the `INFO` level.
+Additional options recognized via `format-options`:
 
-### Common log entry prefix
+| Option | Description |
+|--------|-------------|
+| `show-counter` | Whether to include the counter column in the line header. Without it, the format may be ambiguous due to the optionality of tags. |
+| `colors` | The color profile to use. Possible values: none, auto, ansi, 256color. Default is auto. |
 
-Each line of output starts with the following prefix:
 
-     Lyymmdd hh:mm:ss.uuuuuu goid [chan@]file:line marker tags counter
 
-| Field           | Description                                                                                                                          |
-|-----------------|--------------------------------------------------------------------------------------------------------------------------------------|
-| L               | A single character, representing the [log level](logging.html#logging-levels-severities) (e.g., `I` for `INFO`). |
-| yy              | The year (zero padded; i.e., 2016 is `16`).                                                                                |
-| mm              | The month (zero padded; i.e., May is `05`).                                                                                |
-| dd              | The day (zero padded).                                                                                                               |
-| hh:mm:ss.uuuuuu | Time in hours, minutes and fractional seconds. Timezone is UTC.                                                                      |
-| goid            | The goroutine id (omitted if zero for use by tests).                                                                                 |
-| chan            | The channel number (omitted if zero for backward compatibility).                                                                     |
-| file            | The file name where the entry originated.                                                                                            |
-| line            | The line number where the entry originated.                                                                                          |
-| marker          | Redactability marker ` + redactableIndicator + ` (see below for details).                                                  |
-| tags    | The logging tags, enclosed between `[` and `]`. May be absent. |
-| counter | The entry counter. Always present.                                                 |
+## Format `crdb-v1-count`
 
-The redactability marker can be empty; in this case, its position in the common prefix is
-a double ASCII space character which can be used to reliably identify this situation.
+This format name is an alias for 'crdb-v1' with
+the following format option defaults:
 
-If the marker ` + redactableIndicator + ` is present, the remainder of the log entry
-contains delimiters (‹...›) around
-fields that are considered sensitive. These markers are automatically recognized
-by [`cockroach debug zip`](cockroach-debug-zip.html) and [`cockroach debug merge-logs`](cockroach-debug-merge-logs.html) when log redaction is requested.
+- `show-counter: true`
+- `colors: none`
 
 
 ## Format `crdb-v1-tty`
 
-Same textual format as `crdb-v1`.
+This format name is an alias for 'crdb-v1' with
+the following format option defaults:
+
+- `show-counter: false`
+- `colors: auto`
 
-In addition, if the output stream happens to be a VT-compatible terminal,
-and the flag `no-color` was *not* set in the configuration, the entries
-are decorated using ANSI color codes.
 
 ## Format `crdb-v1-tty-count`
 
-Same textual format as `crdb-v1-count`.
+This format name is an alias for 'crdb-v1' with
+the following format option defaults:
+
+- `show-counter: true`
+- `colors: auto`
 
-In addition, if the output stream happens to be a VT-compatible terminal,
-and the flag `no-color` was *not* set in the configuration, the entries
-are decorated using ANSI color codes.
 
 ## Format `crdb-v2`
 
@@ -306,13 +266,21 @@ the new format, they are prefixed by the `=` continuation
 indicator.
 
 
+Additional options recognized via `format-options`:
+
+| Option | Description |
+|--------|-------------|
+| `colors` | The color profile to use. Possible values: none, auto, ansi, 256color. Default is auto. |
+
+
+
 ## Format `crdb-v2-tty`
 
-Same textual format as `crdb-v2`.
+This format name is an alias for 'crdb-v2' with
+the following format option defaults:
+
+- `colors: auto`
 
-In addition, if the output stream happens to be a VT-compatible terminal,
-and the flag `no-color` was *not* set in the configuration, the entries
-are decorated using ANSI color codes.
 
 ## Format `json`
 

diff --git a/docs/generated/logsinks.md b/docs/generated/logsinks.md
@@ -280,7 +280,7 @@ Type-specific configuration options:
 | Field | Description |
 |--|--|
 | `channels` | the list of logging channels that use this sink. See the [channel selection configuration](#channel-format) section for details.  |
-| `no-color` | forces the omission of VT color codes in the output even when stderr is a terminal. |
+| `no-color` | forces the omission of VT color codes in the output even when stderr is a terminal. This option is deprecated; its effects are equivalent to 'format-options: {colors: none}'. |
 
 
 Configuration options shared across all sink types:

diff --git a/pkg/util/log/flags.go b/pkg/util/log/flags.go
@@ -248,6 +248,17 @@ func ApplyConfig(config logconfig.Config) (logShutdownFn func(), err error) {
 	if err := logging.stderrSinkInfoTemplate.applyConfig(config.Sinks.Stderr.CommonSinkConfig); err != nil {
 		return nil, err
 	}
+	if config.Sinks.Stderr.NoColor {
+		// This branch exists for backward compatibility with CockroachDB
+		// v23.1 and previous versions. The same effect can be obtained
+		// using 'format-options: {colors: none}'.
+		switch t := logging.stderrSinkInfoTemplate.formatter.(type) {
+		case *formatCrdbV1:
+			t.colorProfile = nil
+		case *formatCrdbV2:
+			t.colorProfile = nil
+		}
+	}
 	logging.stderrSinkInfoTemplate.applyFilters(config.Sinks.Stderr.Channels)
 
 	// Create the per-channel loggers.