Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Node plugin #305

Merged
merged 13 commits into from
Aug 20, 2021
Merged

Add Node plugin #305

merged 13 commits into from
Aug 20, 2021

Conversation

ericwholt
Copy link
Contributor

@ericwholt ericwholt commented Aug 19, 2021

This will add new plugin to parse Node.js logs. Logs produced using Winston for logging. Includes kubernetes as log source option.

{
  "timestamp": "2021-08-18T20:54:18.272872-04:00",
  "severity": 30,
  "severity_text": "200",
  "labels": {
    "file_name": "node.log",
    "log_type": "node",
    "plugin_id": "node"
  },
  "record": {
    "headers": {
      "accept-encoding": "gzip",
      "host": "express:3000",
      "user-agent": "Go-http-client/1.1"
    },
    "level": "info",
    "message": "HTTP GET /members",
    "method": "GET",
    "originalUrl": "/members",
    "protocol": "http",
    "protocol_version": "1.1",
    "query": {},
    "responseTime": 2,
    "status": 200,
    "url": "/members"
  }
}

@ericwholt ericwholt requested a review from jsirianni August 19, 2021 00:00
Copy link
Member

@jsirianni jsirianni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, a few comments. I still need to test on kubernetes, which I will do before approving.

CI is failing as well but not due to anything in this PR (as far as I can tell). I will be merging a fix to master and then rebase into this branch.

plugins/node.yaml Outdated Show resolved Hide resolved
plugins/node.yaml Outdated Show resolved Hide resolved
@jsirianni
Copy link
Member

CI is fixed

  1. Let me know when you handle my comments
  2. Ill test on k8s
  3. Approve and merge assuming k8s works (I suspect it does).

@jsirianni
Copy link
Member

@ericwholt Lets rename from node --> nodejs

@jsirianni
Copy link
Member

k8s log

{
  "timestamp": "2021-08-20T01:17:05.057422949Z",
  "severity": 30,
  "severity_text": "200",
  "labels": {
    "k8s-ns/kubernetes.io/metadata.name": "default",
    "k8s-pod/app": "express",
    "k8s-pod/pod-template-hash": "57966494d",
    "log_type": "nodejs",
    "plugin_id": "kubernetes_container",
    "stream": "stdout"
  },
  "resource": {
    "k8s.cluster.name": "",
    "k8s.container.id": "a63bf128d19a2da0d40ce73e701fb3e9e098d3f624631f63d0698bc585743fbb",
    "k8s.container.name": "express",
    "k8s.deployment.name": "express",
    "k8s.namespace.name": "default",
    "k8s.namespace.uid": "f0d2a872-ea9c-4ca6-a92e-b31571bc76d7",
    "k8s.node.name": "",
    "k8s.pod.name": "express-57966494d-pjq8j",
    "k8s.pod.uid": "3b4df96c-9805-480e-a44c-c574739cffa5",
    "k8s.replicaset.name": "express-57966494d"
  },
  "record": {
    "headers": {
      "accept-encoding": "gzip",
      "host": "express:3000",
      "user-agent": "Go-http-client/1.1"
    },
    "level": "info",
    "message": "HTTP GET /members",
    "method": "GET",
    "originalUrl": "/members",
    "protocol": "http",
    "protocol_version": "1.1",
    "query": {},
    "responseTime": 1,
    "status": 200,
    "url": "/members"
  }
}

@jsirianni jsirianni self-requested a review August 20, 2021 01:17
Copy link
Member

@jsirianni jsirianni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 🦖

@jsirianni jsirianni self-requested a review August 20, 2021 01:48
Copy link
Member

@jsirianni jsirianni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to handle situations that include arbitary json, such as

{
  "message": "Received 0 database results",
  "level": "info"
}

and

{
  "message": "Received 0 database results",
}

If level exists, parse it, else just parse as json and send to output.

@jsirianni jsirianni self-requested a review August 20, 2021 02:36
@jsirianni jsirianni merged commit 0d90ea8 into master Aug 20, 2021
@jsirianni jsirianni deleted the add-node-plugin branch August 20, 2021 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants