[hashicorp_vault] undefined metric fields detected by tests

[andrewkroh]

These fields are covered by dynamic templates in the package, but the system test still complains.

integrations/packages/hashicorp_vault/data_stream/metrics/manifest.yml

Lines 55 to 68 in 7d201d7

	dynamic_templates:
	- all_values:
	path_match: "*.value"
	mapping:
	type: double
	- all_counters:
	path_match: "*.counter"
	mapping:
	type: double
	- all_rates:
	path_match: "*.rate"
	mapping:
	type: double
	- all_histograms:

one or more errors found in documents stored in metrics-hashicorp_vault.metrics-ep data stream: [0] field "hashicorp_vault.metrics.vault_barrier_list.value" is undefined
[1] field "hashicorp_vault.metrics.vault_rollback_attempt_auth_token_.value" is undefined
[2] field "hashicorp_vault.metrics.vault_rollback_attempt_auth_token__count.counter" is undefined
[3] field "hashicorp_vault.metrics.vault_rollback_attempt_auth_token__count.rate" is undefined
[4] field "hashicorp_vault.metrics.vault_rollback_attempt_auth_token__sum.counter" is undefined
[5] field "hashicorp_vault.metrics.vault_rollback_attempt_auth_token__sum.rate" is undefined
[6] field "hashicorp_vault.metrics.vault_rollback_attempt_cubbyhole_.value" is undefined
[7] field "hashicorp_vault.metrics.vault_rollback_attempt_cubbyhole__count.counter" is undefined
[8] field "hashicorp_vault.metrics.vault_rollback_attempt_cubbyhole__count.rate" is undefined
[9] field "hashicorp_vault.metrics.vault_rollback_attempt_cubbyhole__sum.counter" is undefined
[10] field "hashicorp_vault.metrics.vault_rollback_attempt_cubbyhole__sum.rate" is undefined
[11] field "hashicorp_vault.metrics.vault_rollback_attempt_identity_.value" is undefined
[12] field "hashicorp_vault.metrics.vault_rollback_attempt_identity__count.counter" is undefined
[13] field "hashicorp_vault.metrics.vault_rollback_attempt_identity__count.rate" is undefined
[14] field "hashicorp_vault.metrics.vault_rollback_attempt_identity__sum.counter" is undefined
[15] field "hashicorp_vault.metrics.vault_rollback_attempt_identity__sum.rate" is undefined
[16] field "hashicorp_vault.metrics.vault_rollback_attempt_secret_.value" is undefined
[17] field "hashicorp_vault.metrics.vault_rollback_attempt_secret__count.counter" is undefined
[18] field "hashicorp_vault.metrics.vault_rollback_attempt_secret__count.rate" is undefined
[19] field "hashicorp_vault.metrics.vault_rollback_attempt_secret__sum.counter" is undefined
[20] field "hashicorp_vault.metrics.vault_rollback_attempt_secret__sum.rate" is undefined
[21] field "hashicorp_vault.metrics.vault_rollback_attempt_sys_.value" is undefined
[22] field "hashicorp_vault.metrics.vault_rollback_attempt_sys__count.counter" is undefined
[23] field "hashicorp_vault.metrics.vault_rollback_attempt_sys__count.rate" is undefined
[24] field "hashicorp_vault.metrics.vault_rollback_attempt_sys__sum.counter" is undefined
[25] field "hashicorp_vault.metrics.vault_rollback_attempt_sys__sum.rate" is undefined
[26] field "hashicorp_vault.metrics.vault_route_rollback_auth_token_.value" is undefined
[27] field "hashicorp_vault.metrics.vault_route_rollback_auth_token__count.counter" is undefined
[28] field "hashicorp_vault.metrics.vault_route_rollback_auth_token__count.rate" is undefined
[29] field "hashicorp_vault.metrics.vault_route_rollback_auth_token__sum.counter" is undefined
[30] field "hashicorp_vault.metrics.vault_route_rollback_auth_token__sum.rate" is undefined
[31] field "hashicorp_vault.metrics.vault_route_rollback_cubbyhole_.value" is undefined
[32] field "hashicorp_vault.metrics.vault_route_rollback_cubbyhole__count.counter" is undefined
[33] field "hashicorp_vault.metrics.vault_route_rollback_cubbyhole__count.rate" is undefined
[34] field "hashicorp_vault.metrics.vault_route_rollback_cubbyhole__sum.counter" is undefined
[35] field "hashicorp_vault.metrics.vault_route_rollback_cubbyhole__sum.rate" is undefined
[36] field "hashicorp_vault.metrics.vault_route_rollback_identity_.value" is undefined
[37] field "hashicorp_vault.metrics.vault_route_rollback_identity__count.counter" is undefined
[38] field "hashicorp_vault.metrics.vault_route_rollback_identity__count.rate" is undefined
[39] field "hashicorp_vault.metrics.vault_route_rollback_identity__sum.counter" is undefined
[40] field "hashicorp_vault.metrics.vault_route_rollback_identity__sum.rate" is undefined
[41] field "hashicorp_vault.metrics.vault_route_rollback_secret_.value" is undefined
[42] field "hashicorp_vault.metrics.vault_route_rollback_secret__count.counter" is undefined
[43] field "hashicorp_vault.metrics.vault_route_rollback_secret__count.rate" is undefined
[44] field "hashicorp_vault.metrics.vault_route_rollback_secret__sum.counter" is undefined
[45] field "hashicorp_vault.metrics.vault_route_rollback_secret__sum.rate" is undefined
[46] field "hashicorp_vault.metrics.vault_route_rollback_sys_.value" is undefined
[47] field "hashicorp_vault.metrics.vault_route_rollback_sys__count.counter" is undefined
[48] field "hashicorp_vault.metrics.vault_route_rollback_sys__count.rate" is undefined
[49] field "hashicorp_vault.metrics.vault_route_rollback_sys__sum.counter" is undefined
[50] field "hashicorp_vault.metrics.vault_route_rollback_sys__sum.rate" is undefined

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[andrewkroh]

@elastic/ecosystem Is there any mechanism for system tests to ignore fields defined by path_match rules? Or is there a different way of defining these dynamic fields that I should be using?

My goal is to not need to define each individual metric field in order to keep the index template small. The fields follow a predicable naming pattern such that the data type can be derived from the name.

[mtojek]

I think that dynamic fields are the only option right now. Maybe you can also try something like this: hashicorp_vault.metrics.*.

BTW I'm wondering if we need all these fields collected. cc @ruflin it looks like this is another candidate for stripping extra records.

[ruflin]

@andrewkroh Side note, wouldn't your path match be more precise if it is path_match: "hashicorp_vault.metrics*.counter" for example? At least it is my understand that you only want to match fields under hashicorp_vault.*

On the testing side, I think we should support to exclude fields from checking like @mtojek describes above. In an ideal world, testing could directly understand the dynamic templates specified and make sense of it. But I'm worried things get out of sync very quickly. Will the propose hashicorp_vault.metrics.* workaround work for you in this case @andrewkroh to ignore all fields in this "object"?

[mtojek]

Hey, just a friendly reminder that the issue is still not fixed. Daily job from today.

[andrewkroh]

I think that dynamic fields are the only option right now. Maybe you can also try something like this: hashicorp_vault.metrics.*.

That is a pattern I have observed in several integrations. And I see there is code in elastic-package to match the * wildcard against any single key. So creating a mapping for hashicorp_vault.metrics.*.* does make the elastic-package validation pass. And that's what I've done in #2994.

But there's a big problem with the generated Elasticsearch mappings. Elasticsearch does not support wildcards in field names so such a mapping would only match a literal hashicorp_vault.metrics.*.* key (e.g. {"hashicorp_vault":{"metrics":{"*":{"*":1.0}}}}).

So I think any integration that uses a name containing a * is probably not getting what they expected in their templates.

[andrewkroh]

Side note, wouldn't your path match be more precise if it is path_match: "hashicorp_vault.metrics*.counter" for example?

@ruflin I have incorporated this improvement into #2994.

[jsoriano]

@andrewkroh yes, the fields with wildcards should be creating dynamic mappings, and not normal fields with * as names, but this is missing in Fleet (see elastic/kibana#129344).

[ruflin]

But there's a big problem with the generated Elasticsearch mappings. Elasticsearch does not support wildcards in field names so such a mapping would only match a literal hashicorp_vault.metrics.. key (e.g. {"hashicorp_vault":{"metrics":{"":{"":1.0}}}}).

@andrewkroh Should we pick this up in a separate issue?

[andrewkroh]

@ruflin We're working through adding dynamic mapping support in elastic/kibana#129344 and elastic/package-spec#314 (to clarify the spec).