Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fleet] Integrate the <LogStream /> component in the Agent details page. #77189

Closed
ph opened this issue Sep 10, 2020 · 10 comments · Fixed by #83356
Closed

[Fleet] Integrate the <LogStream /> component in the Agent details page. #77189

ph opened this issue Sep 10, 2020 · 10 comments · Fixed by #83356
Assignees
Labels
Team:Fleet Team label for Observability Data Collection Fleet team v7.11.0

Comments

@ph
Copy link
Contributor

ph commented Sep 10, 2020

It is now possible to embed the logs component in other places in the UI, I think it would be great to be able to see the log of the Agent in the details page.

91872879-acf21b80-ec78-11ea-9744-bf5c2e2346c0

@jen-huang @hbharding WDYT?

Questions:

@ruflin Should we display only agent logs or we should include filebeat or metricbeat logs in the agent page? I think ideally we would just have the agent, but I think for now there is values in having the filebeat, metricbeat or endpoint logs.

@ph ph added the Team:Fleet Team label for Observability Data Collection Fleet team label Sep 10, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/ingest-management (Team:Ingest Management)

@ph
Copy link
Contributor Author

ph commented Sep 10, 2020

@blakerouse Are we sending the agent uuid in the logs of the agent?

@jen-huang
Copy link
Contributor

Just to clarify, this refers to Infra's LogStream component, right? https://github.com/elastic/kibana/blob/master/x-pack/plugins/infra/public/components/log_stream/README.md

👍 from me, all for reusing existing components.

@ph
Copy link
Contributor Author

ph commented Sep 17, 2020

Looking at the doc above, can it display logs from multiple datastream? Initially we can show agent log but it would be nice to see metricbeat/filebeat logs too. cc @ruflin

@ruflin
Copy link
Contributor

ruflin commented Sep 18, 2020

There is a query part and as all our indices use constant keywords for the dataset, we can add there multiple, not only for agent.

@jen-huang jen-huang self-assigned this Sep 24, 2020
@jen-huang
Copy link
Contributor

I'm looking into this and a few questions came up:

  1. This does not replace the existing Activity Logs table, right? Rather it would just be another tab/table to view raw agent logs? The current activity logs table show agent event saved objects, which are created when the agent checks in with Fleet and does not seem to be 1:1 with the raw agent logs.

  2. I took a look at the data that is ingested into logs-elastic.agent-* data stream to see which identifier field I can use to query for the agent that the user is looking at. But I couldn't find an overlap between agent IDs in the logs versus the agent saved object.

@ph @michalpristas Could you help me answer why the field agent.id in agent logs, is not the same as the local_metadata.elastic.agent.id that is sent to Fleet upon enrollment/check in?

Here is a redacted logs document:

{
  "_index": ".ds-logs-elastic.agent-default-000001",
  "_id": "SIn1wXQBg8juVJpHI2NE",
  "_version": 1,
  "_score": null,
  "_source": {
    "@timestamp": "2020-09-24T21:11:41.884Z",
    "host": {
      "id": "88769239-A9C7-5D8C-8907-ADA0E530270F",
      "ip": [...],
      "mac": [...],
      "hostname": "jens-mbp.lan",
      "architecture": "x86_64",
      "os": {
        "platform": "darwin",
        "version": "10.14.6",
        "family": "darwin",
        "name": "Mac OS X",
        "kernel": "18.7.0",
        "build": "18G2022"
      },
      "name": "jens-mbp.lan"
    },
    "log": {
      "offset": 21902,
      "file": {
        "path": "/Users/jen/Projects/elastic/elastic-agent-8.0.0-SNAPSHOT-darwin-x86_64 5/data/logs/elastic-agent-json.log"
      }
    },
    "log.level": "error",
    "ecs.version": "1.5.0",
    "input": {
      "type": "log"
    },
    "data_stream": {
      "namespace": "default",
      "type": "logs",
      "dataset": "elastic.agent"
    },
    "event": {
      "dataset": "elastic.agent"
    },
    "ecs": {
      "version": "1.6.0"
    },
    "log.origin": {
      "file.name": "application/fleet_gateway.go",
      "file.line": 176
    },
    "message": "Could not communicate with Checking API will retry, error: fail to checkin to fleet: Post \"http://localhost:5603/jen/api/ingest_manager/fleet/agents/387d125d-6931-4fc2-a3f6-9077ea27f48e/checkin?\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)",
    "agent": {
      "type": "filebeat",
      "version": "8.0.0",
      "ephemeral_id": "1b300a3c-9c68-4665-b560-aa53f1023fb0",
      "id": "7194bfb0-1281-47e2-8cc0-21d7499b4b15",
      "name": "jens-mbp.lan"
    },
    "container": {
      "id": "elastic-agent-8.0.0-SNAPSHOT-darwin-x86_64 5"
    }
  },
  "fields": {
    "@timestamp": [
      "2020-09-24T21:11:41.884Z"
    ]
  },
  "sort": [
    1600981901884
  ]
}

And here is the agent SO document:

{
  "_index" : ".kibana_2",
  "_id" : "fleet-agents:387d125d-6931-4fc2-a3f6-9077ea27f48e",
  "_score" : 2.9519293,
  "_source" : {
    "fleet-agents" : {
      "active" : true,
      "policy_id" : "d9413650-fea5-11ea-aac0-17011452a646",
      "type" : "PERMANENT",
      "enrolled_at" : "2020-09-24T21:03:20.480Z",
      "user_provided_metadata" : { },
      "local_metadata" : {
        "elastic" : {
          "agent" : {
            "id" : "387d125d-6931-4fc2-a3f6-9077ea27f48e",
            "version" : "8.0.0"
          }
        },
        "host" : {
          "architecture" : "x86_64",
          "hostname" : "jens-mbp.lan",
          "name" : "jens-mbp.lan",
          "id" : "88769239-A9C7-5D8C-8907-ADA0E530270F",
          "ip" : [...],
          "mac" : [...]
        },
        "os" : {
          "family" : "darwin",
          "kernel" : "18.7.0",
          "platform" : "darwin",
          "version" : "10.14.6",
          "name" : "Mac OS X",
          "full" : "Mac OS X(10.14.6)"
        }
      },
      "access_api_key_id" : "QontwXQBg8juVJpHfDAa",
      "default_api_key" : "4Pr0W/9nfQggLJ/Q7Y39k4Ho8SX9eE3JS+2b9XwElxR8mfTvfxdYgwRkdsZbCHgHj9ZdzEjtkPxPGSFCJL7FrlbmLDQ+uQQkHammRIXxVF/B8NdQMmLd3BiuQPlLq/RIBJ0So6Hcmqs09IDNwfcCqG5xSMNayiZIxYanpPglfu0e0O3nfrgCDd0=",
      "default_api_key_id" : "SontwXQBg8juVJpHlTA2",
      "last_checkin" : "2020-09-24T21:35:32.135Z",
      "policy_revision" : 1,
      "packages" : [
        "system"
      ]
    },
    "type" : "fleet-agents",
    "references" : [ ],
    "migrationVersion" : {
      "fleet-agents" : "7.10.0"
    },
    "updated_at" : "2020-09-24T21:35:32.136Z"
  }
}

@ruflin
Copy link
Contributor

ruflin commented Sep 28, 2020

I think there is an other issue here which I also stumbled over and it is that elastic.agent.id is not reported as part of the events currently and I think it should. It is up to each process to ship agent.id or not, but elastic.agent.id should be in all events and log entries we ship to figure out from which agent the data is coming.

@jen-huang
Copy link
Contributor

The missing elastic.agent.id will be fixed by elastic/beats#21543.

@jen-huang
Copy link
Contributor

Work on this is currently blocked by elastic/beats#21864.

@jen-huang
Copy link
Contributor

Tested today with 7.10 BC3 agent build and expected fields are now present 🎉 No longer blocked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Fleet Team label for Observability Data Collection Fleet team v7.11.0
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants