[telemetry] Adds cloud provider metadata.

[lukeelmers]

Summary

This PR revives some old logic from the monitoring plugin to detect the current cloud environment Kibana is running in (AWS, GCP, or Azure), and provide some basic stats for telemetry purposes.

Changes

• Relocates the cloud detection logic from monitoring to kibana_usage_collection
• Creates a cloud_provider usage collector that fetches the metrics (see sample data below)
• Converts everything to TS
• I purposefully avoided rewriting all of the detector logic in this PR as the old code was still working fine after some initial testing, however there is plenty of cleanup we could still do in followup PRs. Open to any suggestions folks have here.

Sample Data

{
  name: 'aws',
  vm_type: 't2.micro',
  region: 'us-west-2',
  zone: 'us-west-2a',
}

Testing

Testing is a pain because the only way to verify the collector works is by actually deploying Kibana to each of the cloud providers. Since a proper functional test which deploys to each provider isn't possible with the FTR, we've agreed that for now we'll stick with unit tests that have those APIs mocked.

Instead, I'm manually testing this works by deploying this branch to each cloud provider using these Terraform configs. If you'd like to test as well, you can do so following the instructions in the repo, just be sure to configure your repo to be https://github.com/lukeelmers/kibana and your branch to be feat/cloud-telemetry.

To confirm everything's working, I inspect the sample telemetry payload which you can get from scrolling to the bottom of Stack Management > Advanced Settings. The cloud_provider should be provided in the payload.

[lukeelmers]

It looks like Github is showing some of the files I converted to TS as new, but really I just copied the contents of the old js files & only converted to TS with minor formatting updates. The main new thing introduced in this PR is the collector itself.

There's still plenty of cleanup we could do on the old code, just a question of what's worth taking the time to do now. I resisted the urge to overhaul everything, but open to any feedback folks may have 🙂

[lukeelmers]

Just tested the latest again on GCP, Azure, AWS -- all looks good:

          "cloud_provider": {
            "name": "aws",
            "vm_type": "i3.xlarge",
            "region": "us-west-2",
            "zone": "us-west-2b"
          },

          "cloud_provider": {
            "name": "azure",
            "vm_type": "Standard_D4ds_v4",
            "region": "westus2"
          },

          "cloud_provider": {
            "name": "gcp",
            "vm_type": "e2-standard-4",
            "region": "us-west1",
            "zone": "us-west1-a"
          },

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[lukeelmers]

@elasticmachine merge upstream

[afharo]

@elasticmachine merge upstream

[lukeelmers]

@pgayvallet @afharo This should be ready for a final review. I'm trying to strike a balance here between getting in the functionality we want, while avoiding blowing the PR up too much by refactoring the old code.

I addressed the comments that didn't require too much refactoring, but have a few enhancements I'd prefer to do as follow-ups:

1. getting rid of the constructor args which are only there for testing purposes (request, fs), and mocking those in the tests
2. getting rid of the use of the request lib in favor of something like node-fetch which we use more widely in kibana (request looks like it's mostly used in tests atm, not many other places)
3. adding some retry logic in the event of a failure
4. update GCP logic to still return values even if a request for one of the fields fails, instead of using an all-or-nothing approach

Let me know if you feel strongly that we should include any of these in the first iteration, otherwise I will create a follow-up task for us to prioritize.

I think overall (3) is my biggest concern here, however AFAICT we did the same thing (only making 1 attempt) in our previous cloud telemetry implementation. So worst case scenario this has "feature parity" with what we used to report.

[afharo]

@elasticmachine merge upstream

[afharo]

The compromises seem fair to me! LGTM :)

Thank you @lukeelmers!

[lukeelmers]

Thanks @afharo! Follow up issue created here: #96711

I'm looking at the remaining test failure so we can get this back to green.

[pgayvallet]

Let me know if you feel strongly that we should include any of these in the first iteration, otherwise I will create a follow-up task for us to prioritize.

Opening a follow-up SGTM.

[lukeelmers]

@elasticmachine merge upstream

[lukeelmers]

Just did one last test deployment to GCP, Azure, AWS... looks like we are good to go 🚀

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 8993bd1
• Storybooks Preview
• Documentation Changes

Metrics [docs]

✅ unchanged

History

• 💚 Build #118560 succeeded e9f66a9
• 💔 Build #118532 failed b7d0848
• 💔 Build #118433 failed 6e26141
• 💚 Build #118178 succeeded 6d71207
• 💔 Build #117703 failed 0ea6041

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @lukeelmers

[kibanamachine]

💚 Backport successful

Status	Branch	Result
✅	7.x

This backport PR will be merged automatically after passing CI.

[alexfrancoeur]

@lukeelmers @afharo do we need to follow up with a mapping issue in the infra repo?

[afharo]

For now, we have to :)

cc @elastic/infra-telemetry