Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Nomad #16853

Open
3 of 7 tasks
jsoriano opened this issue Mar 5, 2020 · 9 comments
Open
3 of 7 tasks

Add support for Nomad #16853

jsoriano opened this issue Mar 5, 2020 · 9 comments
Labels
enhancement meta Team:Integrations Label for the Integrations team Team:Platforms Label for the Integrations - Platforms team

Comments

@jsoriano
Copy link
Member

jsoriano commented Mar 5, 2020

Nomad is an orchestrator for heterogeneous workloads maintained by Hashicorp.

This issue keeps track of efforts to support Nomad in Beats.

Nomad support could follow a similar approach to the available support for Kubernetes, for that the following components should be implemented:

  • nomad autodiscover provider.
  • add_nomad_metadata processor.
  • Try to use the events API for watching https://www.nomadproject.io/api-docs/events (it is polling now).
  • nomad metricbeat module.
  • nomad filebeat module for nomad agents.
  • Filebeat input if current ones are not enough.
  • Documentation and reference job files to deploy beats in nomad.

The code for these features should be placed under x-pack (licensed as Basic).

At the moment of creating this issue there are some open community PRs that provide some of the required components, we have to review and continue with them:

There are also some known issues regarding Nomad support (though they can be general issues):

// cc: @jorgelbg and @jorgemarey as authors of current open PRs

@jsoriano jsoriano added enhancement meta [zube]: Inbox Team:Platforms Label for the Integrations - Platforms team labels Mar 5, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-platforms (Team:Platforms)

@jsoriano
Copy link
Member Author

Hey @jorgelbg and @jorgemarey,

I have been taking a look to your pull requests and the truth is that both look great, I would love to see Nomad support in Beats. Thanks both for your efforts!

Both PRs contain an add_nomad_metadata processor, and they both work pretty well, but #14954 is more complete, it has support for indexers and matchers, what I think will help to support more Beats and nomad runtimes in the future, and it also includes an autodiscover provider. So I think that it'd be better to go on with this PR only.
In any case, @jorgemarey, if you have some time, it'd be great if you could also take a look to #14954, I am sure you can give some ideas there! For example I like that your processor in #14500 automatically gets the node ID of the local agent.

Thanks again!

@jsoriano
Copy link
Member Author

jsoriano commented Jan 7, 2021

Initial support merged, what includes an autodiscover provider and an add_nomad_metadata processor. It will be released in principle as experimental feature in 7.12.

@Cobular
Copy link

Cobular commented Feb 5, 2021

Thank you! Really looking forward to this being fully released.

@botelastic
Copy link

botelastic bot commented Jan 27, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@botelastic botelastic bot added the Stalled label Jan 27, 2022
@jsoriano jsoriano removed the Stalled label Jan 27, 2022
@botelastic
Copy link

botelastic bot commented Jan 27, 2023

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

@botelastic botelastic bot added the Stalled label Jan 27, 2023
@joaquinjsb
Copy link

👍🏼

@botelastic botelastic bot removed the Stalled label Jul 14, 2023
@botelastic
Copy link

botelastic bot commented Jul 13, 2024

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

@botelastic botelastic bot added the Stalled label Jul 13, 2024
@matya
Copy link

matya commented Jul 13, 2024

The hope dies last 😅

@botelastic botelastic bot removed the Stalled label Jul 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement meta Team:Integrations Label for the Integrations team Team:Platforms Label for the Integrations - Platforms team
Projects
None yet
Development

No branches or pull requests

6 participants