Add support for multi-level self-nestings

[ebeahan]

Add support to selectively allow a self-nesting to be reused inside another self-nesting.

For example, I want to nest the process.* fields within another nesting of process, process.target.*.

This type of reuse will need to be defined explicitly in the field definitions:

- name: process
  reusable:
    expected:
      - at: process
        as: target
      - at: process.target
        as: parent

Closes #1458

[kgeller]

LGTM!

[djptek]

Do we really want this? I took a better look at the RFC 0016 & I understand the context, however, I don't agree nesting is the right way to go, principally because that may create technical debt downstream for how to unpack this, e.g- O11y. Adding comments to that RFC, sorry for coming in late on this one

[djptek]

There is an alternative to multi-level nesting - split to separate docs and flatten one level e.g. something like:

PUT test/_doc/001
{ "my_id": "001" }

PUT test/_doc/002
{ "my_id": "002", "my_parent_ids": "001" }

PUT test/_doc/003
{ "my_id": "003", "my_parent_ids": ["001", "002"]}

that would require multiple queries to assemble the full chain of parents but that is easier to code and safer than stacking them internally: consider this use case - a process that calls itself perpetually (that happens) may overflow the stack (possibly deliberately, in order to harvest data), either in ECS or worse, downstream :/

[ebeahan]

I framed this PR in the context of #1297, but let's discuss the changes here independently as a feature of tooling and the schema DSL, and if we want this ability to be available for use at the user's discretion.

Some additional background points:

• Foreign reuses already allow a similar pattern by way of the order attribute. For example, group is reused under user, and then user is reused to create user.target.group.*.
• There's been past discussion about the endpoint leveraging this type of pattern in the endpoint's schema.
• Right now, this functionality is implied by the tool's docs and DSL, but it doesn't work as expected. I opt to leave the decision whether to self-nest up to the user but have the functionality available if they choose to use it.

I think we move forward with adding the functionality. If we adopt the pattern into ECS, we'll already have support in our tooling, but if we don't, the tooling will support it if a user chooses to do so.

[djptek]

Thanks for the additional clarifications @ebeahan

There's one comment on #1297 that I think is Key

we will need to make sure that we articulate what each reuse means

So provided each nesting is a manual design decision with peer review and not a consequence of a recursive algorithm, I'm good to go with this 👍

[djptek]

provided each nesting is a manual design decision with peer review and not a consequence of a recursive algorithm, LGTM 👍