fix: better payload for proxy logs and operations #2631
Conversation
| @@ -213,7 +238,18 @@ class ProxyController { | |||
| logCtx: LogContext; | |||
| }) { | |||
| const safeHeaders = proxyService.stripSensitiveHeaders(config.headers, config); | |||
| await logCtx.info(`${config.method.toUpperCase()} ${url} was successful`, { headers: safeHeaders }); | |||
| await logCtx.http(`${config.method.toUpperCase()} ${url} was successful`, { | |||
| meta: null, | |||
There was a problem hiding this comment.
meta could potentially be optional. Not a blocker for now though
There was a problem hiding this comment.
You can definitely removed it from the required field
| }, | ||
| response: { | ||
| code: res.statusCode, | ||
| headers: getHeaders(res.getHeaders()) |
There was a problem hiding this comment.
we are gonna log way more content with the headers. Is it gonna be ok with ES?
There was a problem hiding this comment.
I need to increase the disk size anyway. It will increase the storage a bit but most logs are empty (!= operation)
| response: { | ||
| code: error.response?.status || 500, | ||
| headers: error.response?.headers as Record<string, string> | ||
| } |
There was a problem hiding this comment.
note we can also modify the client to build the object automatically (like the way I did with error) just to reuse the logic
There was a problem hiding this comment.
Not sure it make sense to tie logs to Axios.
| const getHeaders = (hs: IncomingHttpHeaders | OutgoingHttpHeaders): Record<string, string> => { | ||
| const headers: Record<string, string> = {}; | ||
| for (const [key, value] of Object.entries(hs)) { | ||
| if (typeof value === 'string') { | ||
| headers[key] = value; | ||
| } else if (Array.isArray(value)) { | ||
| headers[key] = value.join(', '); | ||
| } | ||
| } | ||
| return headers; | ||
| }; | ||
| const reqHeaders = getHeaders(req.headers); | ||
| reqHeaders['authorization'] = 'REDACTED'; |
There was a problem hiding this comment.
You can't use proxy.service->stripSensitiveHeaders ?
There was a problem hiding this comment.
it requires a proxy config but things can fail before having a valid config
I had the following error once after switching branches Did you get the same thing? |
TBonnin
force-pushed
the
tbonnin/proxy-logs-improvements
branch
from
046253a
to
a50da8d
Compare
we are experiencing version_conflict_engine_exception error, setting the retry_on_conflict parameter to automatically retry when version conflict
TBonnin
force-pushed
the
tbonnin/proxy-logs-improvements
branch
from
a50da8d
to
b3c6e53
Compare
Showing request and response info in operations and logs payload for proxy call.
Operation payload shows info about the request to nango api
Log payload shows info about the request to the 3rd party api
Successful operation:
Successful log:
Failed operation:
Failed log:
Issue ticket number and link
https://linear.app/nango/issue/NAN-1526/improve-proxy-logs