Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logging format, not really json. #12

Open
andrew-pickin-epi opened this issue Jan 24, 2022 · 3 comments
Open

Logging format, not really json. #12

andrew-pickin-epi opened this issue Jan 24, 2022 · 3 comments

Comments

@andrew-pickin-epi
Copy link
Contributor

Consider:

{"timestamp":"2022-01-24T17:27:56.819+0000","level":"INFO","thread":"http-apr-8080-exec-3","logger":"com.epimorphics.simpleAPI.webapi.LogRequestFilter","message":"Request  [23192] : /sr-manager/system/metrics","context":"default"}

and

{"timestamp":"2022-01-24T17:27:56.820+0000","level":"INFO","thread":"http-apr-8080-exec-3","logger":"com.epimorphics.simpleAPI.webapi.LogRequestFilter","message":"Response [23192] : 200 (1 ms)","context":"default"}

Timestamp isn't of requested format.
The Response [n] means that it is difficult to filter log the log entries concerned with metrics. It's not obvious which call the status/duration belongs to.

Request a single event of a form similar to:

{
"ts":"2022-01-24T17:27:56.820Z",
"level":"INFO",
"thread":"http-apr-8080-exec-3",
"logger":"com.epimorphics.simpleAPI.webapi.LogRequestFilter",
"pid:" "23192",
"method": "GET",
"path": "/sr-manager/system/metrics",
"status": 200,
"duration": 1,
"context":"default"
}
@der
Copy link
Member

der commented Jan 24, 2022

Hmm. If we waited until a response has been successfully completed before we logged anything then we would miss requests that get lost (can happen with SR and important to be able to detect). Even for requests that succeed they can take considerable time and it can be important during that time to be tell from the logs that the request has been received.

I can see that including the request path in the response message so the response message is more self contained would make then easier to handle in a kibana world (though it's not that hard to filter on [23192]).

I guess we can look at attempting to make response messages more self contained and reformat then. Pretty substantial bit of work though, especially since it means redeveloping some of the upstream components that are currently frozen.

@andrew-pickin-epi
Copy link
Contributor Author

But I don't want to filter on [23192].
I want to exclude all the metrics paths. I can't do this with a single click because all of the pids (if they are pids).

der added a commit that referenced this issue Apr 28, 2022
- timestamp field now ts in compliant format
- request/response logs now contain an explicit path to enable filtering
- includes request_id field
- uses incoming x-request-id field if present else generates transaction count

Addresses: #12 and part of  epimorphics/hmlr-ansible-deployment#76
@der
Copy link
Member

der commented Apr 28, 2022

  • timestamp field now ts in compliant format
  • request/response logs now contain an explicit path to enable filtering of e.g. metric requests
  • includes request_id field
  • uses incoming x-request-id header, if present, else generates transaction count

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants