Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Asset Manager] services endpoint #160294

Merged

Conversation

klacabane
Copy link
Contributor

@klacabane klacabane commented Jun 22, 2023

Summary

Closes #159641

Implements /assets/services endpoint that returns service assets found in the configured source (signals or assets indices). Consumer can provide a parent query to filter the returned services. While the assets mode supports any kind of parent/depth thanks to its common interface, the signals mode only supports host parent for the moment.

Testing

  1. pull this branch and point it at an oblt-cli created cluster that uses cross-cluster search to read from the edge cluster
  2. add the following[1] to your kibana.yml file
  3. hit /api/asset-manager/assets/services?from=<from>&to=<to>&(parent=<host>)?. services should be returned. Add/remove parent query string to filter services only running on specific host.
  4. update lockedSource: assets to read from assets generated by implicit collection
  5. repeat 3.

[1]

xpack.assetManager:
  alphaEnabled: true
  sourceIndices:
    metrics: remote_cluster:metricbeat*,remote_cluster:metrics-*
    logs: remote_cluster:filebeat*,remote_cluster:logs-*
    traces: remote_cluster:traces-*
    serviceMetrics: remote_cluster:metrics-apm*
    serviceLogs: remote_cluster:logs-apm*
  lockedSource: signals
  implicitCollection:
    enabled: true
    interval: 30s

@apmmachine
Copy link
Contributor

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@klacabane klacabane changed the base branch from main to feat/obs-asset-manager-demo June 22, 2023 15:57
@klacabane klacabane added Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services Team:Observed Asset Management Label used for engineers working on various parts of observed asset management Feature:Asset Manager labels Jun 23, 2023
@klacabane klacabane marked this pull request as ready for review June 28, 2023 10:11
@klacabane klacabane requested review from a team as code owners June 28, 2023 10:11
@elasticmachine
Copy link
Contributor

Pinging @elastic/infra-monitoring-ui (Team:Infra Monitoring UI)

@@ -74,9 +74,6 @@ disabled:
# Scalability testing config that we run in its own pipeline
- x-pack/test/scalability/config.ts

# Asset Manager configs, in tech preview, will move to enabled after more stability introduced
- x-pack/test/api_integration/apis/asset_manager/config.ts
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see it based on the description: why is this test removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is removed from the disabled block and added to enabled

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my bad :)

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@neptunian
Copy link
Contributor

neptunian commented Jul 26, 2023

I think we should expose a public client to get the services as part of this PR. What do you think?

@klacabane
Copy link
Contributor Author

Good call for the client, but I'd be inclined to implement it as a separate change since it will also need to expose the /hosts endpoint

@neptunian
Copy link
Contributor

@elasticmachine merge upstream

klacabane added a commit that referenced this pull request Aug 17, 2023
## Summary
Closes #161887

Merges most of the functionality from `feat/obs-asset-manager-demo`
branch. We remove implicit collection code while including:
- asset collectors from signals (also include pods and containers but we
don't use then directly)
- source configuration code (assets or signals)
- `assetAccessor` logic that determines which indices to query

The change also enables ftr test suite. We'll also merge the services
endpoint #160294 when approved.

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Jason Rhodes <[email protected]>
@klacabane klacabane merged commit 644bf81 into elastic:feat/obs-asset-manager-demo Aug 17, 2023
klacabane added a commit to klacabane/kibana that referenced this pull request Aug 17, 2023
Closes elastic#159641

Implements `/assets/services` endpoint that returns service assets found
in the configured source (signals or assets indices). Consumer can
provide a `parent` query to filter the returned services. While the
_assets_ mode supports any kind of parent/depth thanks to its common
interface, the _signals_ mode only supports host parent for the moment.

1. pull this branch and point it at an oblt-cli created cluster that
uses cross-cluster search to read from the edge cluster
2. add the following[1] to your kibana.yml file
3. hit
`/api/asset-manager/assets/services?from=<from>&to=<to>&(parent=<host>)?`.
services should be returned. Add/remove parent query string to filter
services only running on specific host.
4. update `lockedSource: assets` to read from assets generated by
implicit collection
5. repeat 3.

[1]
```
xpack.assetManager:
  alphaEnabled: true
  sourceIndices:
    metrics: remote_cluster:metricbeat*,remote_cluster:metrics-*
    logs: remote_cluster:filebeat*,remote_cluster:logs-*
    traces: remote_cluster:traces-*
    serviceMetrics: remote_cluster:metrics-apm*
    serviceLogs: remote_cluster:logs-apm*
  lockedSource: signals
```

---------

Co-authored-by: Kibana Machine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Asset Manager Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services Team:Observed Asset Management Label used for engineers working on various parts of observed asset management
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Asset Manager] Add getServicesForHost method
7 participants