pino plugin please

[naseemkullah]

Describe the solution you'd like
an equivalent to the bunyan or winston plugins, but for https://github.com/pinojs/pino

Describe alternatives you've considered
manually configuring pino: https://github.com/pinojs/pino/blob/master/docs/help.md#stackdriver

[mcollina]

I'm happy to assist/review.

[naseemkullah]

Apart from the level to severity mapping, an important consideration to work with error reporting, the stack trace for errors should be in a specific field, according to an exchange with the Error Reporting team these are message, exception or stack_trace ... @mcollina please correct me if I am wrong but I believe we could configure a pino error serializer slightly different than the standard err serializer to output the error stack on the field expected by Google Cloud Error Reporting (exception or stack_trace are preferred, whereas as message should be saved for https://github.com/pinojs/pino/blob/master/docs/api.md#message-string, where we could put a stack trace in technically, but probably better to leverage the serializer).

The field isn't customizable. Error Reporting will also accept "stack_trace" or "exception" as aliases for "message" (which leaves the "message" field available for a more human readable text).

[naseemkullah]

@mcollina please correct me if I am wrong but I believe we could configure a pino error serializer slightly different than the standard err serializer to output the error stack on the field expected by Google Cloud Error Reporting (exception or stack_trace are preferred,

Conversely, Google Cloud Error Reporting may wish to add err.stack to the aliases of stack trace fields, that way we can take full advantage of pino's error serializer as is.

[mcollina]

@naseemkullah I think so, yes! See https://www.npmjs.com/package/@elastic/ecs-pino-format as an example for a custom config for a specific cloud provider.

[naseemkullah]

@simonz130 In case this helps, this is our current config, which most if not all could be offloaded to the plugin. the fact that we add the @type prop to every severity>=ERROR log should probably be an option

import {context, getSpan} from '@opentelemetry/api';
import * as pino from 'pino';

// Map Pino levels to Google Cloud Logging severity levels
// https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#LogSeverity
const levelToSeverity = {
  trace: 'DEBUG',
  debug: 'DEBUG',
  info: 'INFO',
  warn: 'WARNING',
  error: 'ERROR',
  fatal: 'CRITICAL',
};

export function initLogger(opts: {
  serviceName: string;
  level?: pino.Level;
  format?: 'text' | 'json';
  pinoLoggerOptions?: pino.LoggerOptions;
}) {
  const {serviceName, level, format, pinoLoggerOptions} = opts;

  const prettyPrint =
    (format || process.env.LOG_FORMAT?.toLowerCase()) === 'text';

  return pino({
    ...{
      // https://cloud.google.com/error-reporting/docs/formatting-error-messages#json_representation
      base: {serviceContext: {service: serviceName}},
      formatters: {
        level(label) {
          const pinoLevel = label as pino.Level;
          // `@type` property tells Error Reporting to track even if there is no `stack_trace`
          const typeProp =
            label === 'error' || label === 'fatal'
              ? {
                  '@type':
                    'type.googleapis.com/google.devtools.clouderrorreporting.v1beta1.ReportedErrorEvent',
                }
              : {};
          return {
            level: pinoLevel,
            severity: levelToSeverity[pinoLevel],
            ...typeProp,
          };
        },
        log(object: object & {err?: Error}) {
          const stackTrace = object.err?.stack;
          const stackProp = stackTrace ? {stack_trace: stackTrace} : {};
          return {
            ...object,
            ...stackProp,
          };
        },
      },
      level: level || process.env.LOG_LEVEL?.toLowerCase() || 'info',
      messageKey: 'message',
      mixin: () => getSpan(context.active())?.context() ?? {},
      prettyPrint,
      timestamp: prettyPrint
        ? pino.stdTimeFunctions.isoTime
        : () => `,"eventTime":${Date.now() / 1000.0}`,
    },
    ...pinoLoggerOptions,
  });
}

[teebot]

@naseemkullah thanks for opening this issue and providing the required config for ErrorReporting to work with pino!

[gnarea]

FYI: I too wanted my app to push logs to Stackdriver and errors to Error Reporting, but my app is cloud-agnostic, so I couldn't have references to GCP in it, which led me to create this cloud-agnostic integration for Pino: https://github.com/relaycorp/pino-cloud-js

So far only GCP is supported but PRs to support further cloud providers are welcomed.

[Mistic92]

Hi, how is work going on this plugin? We are using Cloud Run and struggling with using "normal" Pino with structured logs. Would be great if there was just a plugin

[gnarea]

@Mistic92, did you try the library I mentioned in the previous comment?

[Mistic92]

I missed it when browsing on mobile. Looks good but I don't see way to make it use httpRequest field for structured logging. Is it only for fix severity issue?

[guillenotfound]

@Mistic92 I created https://github.com/guillenotfound/pino-gcp-logger and so far is working for logging & error reporting 👍

[Mistic92]

@guillenotfound do you have plans for structured logging, trace and httprequest fields?

[yoitsro]

Hey! Any updates on this? 🙏 Is there anything we can do to push this forward?

[guillenotfound]

@guillenotfound do you have plans for structured logging, trace and httprequest fields?

Sorry for the late reply, point me to the docs and I'll investigate 👍

[jordanebelanger]

This is what I use on CloudRun, it does the job for me.

function pinoLevelToStackdriverSeverity(level) {
  switch (level) {
    case 10 /* PINO_LEVELS.trace */:
    case 20 /* PINO_LEVELS.debug */:
      return "DEBUG" /* StackdriverSeverity.debug */;
    case 30 /* PINO_LEVELS.info */:
      return "INFO" /* StackdriverSeverity.info */;
    case 40 /* PINO_LEVELS.warn */:
      return "WARNING" /* StackdriverSeverity.warning */;
    case 50 /* PINO_LEVELS.error */:
      return "ERROR" /* StackdriverSeverity.error */;
    case 60 /* PINO_LEVELS.fatal */:
      return "CRITICAL" /* StackdriverSeverity.critical */;
    default:
      return "DEFAULT" /* StackdriverSeverity.default */;
  }
}
function truncateToDecimalPlace(value, decimalPlace) {
  return Math.trunc(value * Math.pow(10, decimalPlace)) / Math.pow(10, decimalPlace);
}
/** Logs Stackdriver structured JSON to stdout */
const stackdriverLoggerOptions = {
  messageKey: "message",
  timestamp: false,
  base: undefined,
  formatters: {
    level(_label, number) {
      return {
        severity: pinoLevelToStackdriverSeverity(number),
      };
    },
    log(object) {
      // Setuping the Stackdriver httpRequest property on the log entry
      if (object.req != null) {
        object.httpRequest = {
          ...(object.httpRequest ?? {},
          {
            requestMethod: object.req.method,
            requestUrl: object.req.url,
            userAgent: object.req.headers["user-agent"],
            remoteIp: object.req.ips?.[0] ?? object.req.ip,
            protocol: object.req.protocol,
          }),
        };
      }
      if (object.res != null) {
        object.httpRequest = {
          ...(object.httpRequest ?? {},
          {
            requestMethod: object.res.request.method,
            requestUrl: object.res.request.url,
            status: object.res.statusCode,
            userAgent: object.res.request.headers["user-agent"],
            latency: `${truncateToDecimalPlace(object.res.getResponseTime() / 1000, 9)}s`,
            remoteIp: object.res.request.ips?.[0] ?? object.res.request.ip,
            protocol: object.res.request.protocol,
          }),
        };
      }
      return object;
    },
  },
  serializers: {
    // Nullifying the standard Fastify Request/Response serializer for better stackdriver support
    req() {
      return undefined;
    },
    res() {
      return undefined;
    },
    responseTime: function () {
      return undefined;
    },
  },
};

export default stackdriverLoggerOptions;

[bkw]

Heads Up: This seems to have been solved by @google-cloud/pino-logging-gcp-config, see https://googlecloudplatform.github.io/cloud-solutions/pino-logging-gcp-config/ for details.