[Fleet] Simplify skipArchive behaviour for input packages

[hop-dev]

Summary

Closes #137751

Input packages always have to get their package info from the archive, as they contain fields in their manifest that the package registry API does not return. These fields are needed to create the package policy so we need to get info from the archive even when using the GET package info API (as opposed to just at install time)

Previously, I was ensuring that input packages get their information from the archive by using (unintuitively) the skipArchive flag, which meant we went to the registry to get the packageInfo, then checked if it was an input package, and if so we go and get the info from the archive.

This PR simplifies that behaviour by always attempting to get the info from the registry first, regardless of skipArchive, then deciding whether to skip the archive.

I have also added integration tests for input packages:

• Create valid pkg policy
• Create invalid pkg policy
• Get pkg info

Does this conflict with #142353 "rely on package content during installation"?

This changes the behaviour for get package info, we still rely on the archive for installation. However, we should move to not using the registry at all for the get info API to unblock #131609, but this will be more work, and maybe need some work on the EPR side, I have created an issue for this #143198

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[nchaulet]

@hop-dev @criamico does this two PR are not conflicting #142353?

Why we would like (or need) to get the package info from the registry and not ES if we have the info?

[criamico]

@hop-dev @criamico does this two PR are not conflicting #142353?

Why we would like (or need) to get the package info from the registry and not ES if we have the info?

Yes I just saw your comment and was leaving a similar one. I think we need to review the whole flow and understand what we need to modify.

[hop-dev]

Are we moving to getting all information from the archive every time, or just at install time? If it's every time then I think skipArchive has to be removed as its a shortcut to get the information from the registry and build the assets paths from that instead of the archive.

[nchaulet]

I think the @criamico was for the install time, but it will probably be a good improvement to not call the registry when building the agent policy too, (it will remove a risk of failure here)

[criamico]

@nchaulet @hop-dev
#142353 is limited to the install time but reading the original ticket I thought that the aim is to decouple the registry from the spec and, on the long run, use the registry only for discovering packages.
If this understanding is correct, removing skipArchive it's a further step in this direction. Let me know, however, if I should revert that change from my PR.

[hop-dev]

@criamico I have been back and forth on the scope of this change! And I am basically back where I started. Please read the PR description for a more in depth explanation! 👍

[criamico]

Thanks for investigating further into this, it's not an easy flow.

I left a minor comment, overall LGTM 👍

[criamico]

nit: isn't this the same as skipArchive && registryInfo.type !== 'input' ?

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 4162afd

Metrics [docs]

✅ unchanged

History

• 💔 Build #80037 failed 7ceb230bb25ee5febb7b1170fd95cf683af023c7
• 💔 Build #80024 failed ff231d8e11b0fa6f14f2f42d81ab9398c811adcd
• 💚 Build #79924 succeeded b743d0a
• 💔 Build #79793 failed bd4fa8e3f8293059b21ae9c1fbd76754a550063c
• 💔 Build #79444 failed 329930a45ae6c21eb8e8ff01c5b3bca06f4ef223
• 💔 Build #79164 failed f52ab9b0758b6395f36631922ed74ba671d059be

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream