Hapi Instrumentation does not Capture Request Body

[astorm]

Unsure if this is an enhancement request or a bug report, but it looks like our HAPI instrumentation does not collect the request bodies as captured by the built-in HAPI payload handling. HAPI populates request.payload.name with these values, but we read our request bodies in from the native node IncomeingMessage/Request object (i.e. req.body)

You can observe this behavior by running the following small sample program

const Hapi = require('@hapi/hapi')

const init = async () => {
  const server = Hapi.server({
    port: 3000,
    host: '0.0.0.0'
  })

  server.route({
    method: 'POST',
    path: '/test',
    handler: (request, h) => {
      // request.response.header({foo:"bar"})
      console.log(request.payload)
      const response = h.response('Hello World!')
      for (const [header, value] of Object.entries({ 'X-foo': 'bar' })) {
        response.header(header, value)
      }
      return response
    }
  })
  await server.start()
  console.log(server.info)
  console.log('Server running on %s', server.info.uri)
}

and posting some form variables.

$ curl -i http://localhost:3000 -d foo=bar

HAPI captures the request body, but if you examine the generated transactions, no request body is captured.

Ideally our HAPI instrumentation should match our express instrumentation's functionality and capture these values.

[mshustov]

we read our request bodies in from the native node IncomeingMessage/Request object (i.e. req.body)

Hapi doesn't seem to extend IncomingRequest object with a parsed payload object. The request body is parsed in a route lifecycle
and assigned to the Hapi request object https://github.com/hapijs/hapi/blob/master/lib/route.js#L430-L434
https://github.com/hapijs/hapi/blob/master/lib/route.js#L289-L291
Can we store request payload in Hapi instrumentation logic to access it later when finalizing a transaction?

apm-agent-nodejs/lib/instrumentation/transaction.js

Line 206 in 170e411

payload.context.request = parsers.getContextFromRequest(this.req, conf, 'transactions')

[trentm]

A slight digression on request body capture. When I started looking at this I noticed that the Hapi Request object provides a 'peek' event that enables on to "tap" into the raw chunks of request data. (There is a 'peek' event documented on the Hapi Response object, https://hapi.dev/api/#-responseevents, so I'm not sure if this request.events.on('peek', ...) counts as undocumented, an oversight or a bug in the Hapi reference docs.)

  function onPreAuth (request, reply) {
...
    request.events.on('peek', function (chunk, encoding) {
      console.warn('XXX request "peek" event', chunk, encoding)
    })
...

Theoretically this could be used to capture the raw HTTP request body. However, this isn't the general way we capture the body for other frameworks (e.g. Express).

Instead the other existing instrumentation lets the framework/app parse the request body, and the APM instrumentation looks for the parsed body set to req.json || req.body || req.payload and uses that, if available. To store it on the APM transaction context, that object is then re-serialized as JSON. There are pros and cons here, but this is what we are doing elsewhere now. An important "Pro" of using the parsed-body is that we can apply sanitizeFieldNames to body fields in some cases for privacy reasons.