Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Elastic Agent] Add elastic agent ID and version to events from filebeat and metricbeat. #21543

Merged
merged 3 commits into from
Oct 6, 2020
Merged

[Elastic Agent] Add elastic agent ID and version to events from filebeat and metricbeat. #21543

merged 3 commits into from
Oct 6, 2020

Conversation

blakerouse
Copy link
Contributor

What does this PR do?

Adds the elastic.agent.id, elastic.agent.version, and elastic.agent.snapshot to all events published from filebeat and metricbeat inputs. This includes when filebeat/metricbeat is monitoring the running Elastic Agent itself.

Why is it important?

So it's clear which Elastic Agent sent the events. Elastic Agent has never included this information with each event.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

@blakerouse blakerouse self-assigned this Oct 5, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/ingest-management (Team:Ingest Management)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Oct 5, 2020
@elasticmachine
Copy link
Collaborator

elasticmachine commented Oct 5, 2020 •
edited by jenkins-beats-ci bot
Loading

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #21543 updated]

  • Start Time: 2020-10-05T19:07:20.111+0000

  • Duration: 39 min 5 sec

Test stats 🧪

Test Results
Failed 0
Passed 1386
Skipped 4
Total 1390

@jen-huang
Copy link

This looks like it could be a fix for an issue I ran into when trying to match up agent logs to agent ID: elastic/kibana#77189 (comment). @blakerouse do you think this PR is relevant to that?

@ph ph requested a review from michalpristas October 5, 2020 19:20
@blakerouse
Copy link
Contributor Author

@jen-huang Yes this is the fix you need.

ph
ph approved these changes Oct 5, 2020
View reviewed changes
Copy link
Contributor

@ph ph left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, @blakerouse are we missing heartbeat?

@jen-huang jen-huang mentioned this pull request Oct 5, 2020
Closed
@blakerouse
Copy link
Contributor Author

@ph Yes I did not add heartbeat, I didn't know if Uptime needed the same events and if the processors would work the same.

If we can confirm that heartbeat needs it I can add it in.

michalpristas
michalpristas approved these changes Oct 6, 2020
View reviewed changes
Copy link
Contributor

@michalpristas michalpristas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

x-pack/elastic-agent/pkg/agent/application/info/agent_info.go
@@ -44,3 +46,13 @@ func ForceNewAgentInfo() (*AgentInfo, error) {
func (i *AgentInfo) AgentID() string {
return i.agentID
}

// Version returns the version for this Agent.
func (*AgentInfo) Version() string {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we use release.Version and release.Snapshot throughout the code, so maybe we can unify the usage of these where it's possible in a followup

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I would like to. I only had to add this here for testing purposes as I needed a way to mock the version for the unit tests.

@blakerouse blakerouse merged commit f5d13aa into elastic:master Oct 6, 2020
@blakerouse blakerouse deleted the agent-info-events branch October 6, 2020 13:02
@blakerouse blakerouse mentioned this pull request Oct 6, 2020
Merged
4 tasks
blakerouse added a commit to blakerouse/beats that referenced this pull request Oct 6, 2020
@blakerouse
[Elastic Agent] Add elastic agent ID and version to events from fileb…
117bf32 
…eat and metricbeat. (elastic#21543)

* Add elastic agent ID and version to events from filebeat and metricbeat.

* Add changelog and fix inputs.

(cherry picked from commit f5d13aa)
blakerouse added a commit that referenced this pull request Oct 6, 2020
@blakerouse
[Elastic Agent] Add elastic agent ID and version to events from fileb…
538cc11 
…eat and metricbeat. (#21543) (#21566)

* Add elastic agent ID and version to events from filebeat and metricbeat.

* Add changelog and fix inputs.

(cherry picked from commit f5d13aa)
v1v added a commit to v1v/beats that referenced this pull request Oct 6, 2020
@v1v
Merge remote-tracking branch 'upstream/master' into feature/windows-2016
fcd6db3 
* upstream/master:
  [CI] Setup git config globally (elastic#21562)
  docs: update generate_fields_docs.py (elastic#21359)
  Add support for additional fields from V2 ALB logs (elastic#21540)
  Move Prometheus query & remote_write to GA (elastic#21507)
  feat: add a new step to run the e2e tests for certain parts of Beats (elastic#21100)
  [Elastic Agent] Add elastic agent ID and version to events from filebeat and metricbeat. (elastic#21543)
  Release cloudfoundry input and processor as GA (elastic#21525)
  [Packetbeat] New SIP protocol (elastic#21221)
  [Filebeat][New Module] Add support for Microsoft MTP / 365 Defender (elastic#21446)
  [Beats][pytest] Asserting if filebeat logs include errors (elastic#20999)
  junipersrx-module initial release (elastic#20017)
  Add a persistent cache for cloudfoundry metadata based on badger (elastic#20775)
  Add missing changelog entry for cisco umbrella (elastic#21550)
  [Elastic Agent] Add upgrade CLI to initiate upgrade of Agent locally (elastic#21425)
  Enable filestream input (elastic#21533)
  Add filestream input reader (elastic#21481)
  [CI] fix 'no matches found within 10000' (elastic#21466)
  Fix billing.go aws.GetStartTimeEndTime (elastic#21531)
@ph
Copy link
Contributor

ph commented Oct 6, 2020 •
edited
Loading

@blakerouse I think we should also have the same for heartbeat. This can come after 7.10. cc @andrewvc

@blakerouse
Copy link
Contributor Author

@ph I don't believe that heartbeat is also using the new index strategy either. So we have a few things that need to be updated in the spec file for Heartbeat.

@ph
Copy link
Contributor

ph commented Oct 6, 2020

@blakerouse I think heartbeat should be disabled in 7.10, @andrewvc WDYT?

@ruflin
Copy link
Contributor

ruflin commented Oct 8, 2020

Each process run by Elastic Agent should be "forced" to add these fields. Do we have a good place to document this?

Is this field also shipped by endpoint?

Mid term, these fields should not be added anymore through processors but by the processes (MB, FB, *) itself.

@jen-huang jen-huang mentioned this pull request Oct 15, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@michalpristas michalpristas michalpristas approved these changes

@ph ph ph approved these changes

Assignees

@blakerouse blakerouse

Labels
v7.10.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@blakerouse @elasticmachine @jen-huang @ph @ruflin @michalpristas