[kibana] support api_key

[sachin-frayne]

Summary

Adds support for api_key authentication in the kibana module

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

• create an api key and grab the beats format[1]
• configure kibana module to use that api_key:

metricbeat.modules:
  - module: kibana
    xpack.enabled: true
    hosts: ["http://localhost:9200"]
    api_key: <key in beats format>
    period: 10s

• kibana metricsets are successfully collecting documents

[1]

Related issues

• Step one to adding api_key to Kibana module: https://github.com/elastic/enhancements/issues/22286

Use cases

Screenshots

Logs

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @sachin-frayne? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[mergify]

backport-v8.x has been added to help with the transition to the new branch 8.x.

[mergify]

backport-8.x has been added to help with the transition to the new branch 8.x.

[VihasMakwana]

@sachin-frayne can you take care of CI failures. LGTM otherwise.

[VihasMakwana]

I've rebased main.

[sachin-frayne]

I've rebased main.

Thank you

[sachin-frayne]

Hello @VihasMakwana.

I have fixed the CI, I have also changed the setting in: .github/workflows/golangci-lint.yml removing:

skip-go-installation: true

I don't think this was causing any failures, but in debugging I found that the setting is removed/being removed: https://github.com/golangci/golangci-lint-action

Can you please review one more time?

[sachin-frayne]

Hello @VihasMakwana

When you get a chance can you please review this?

[pierrehilbert]

@alexsapran / @pazone could you please review here?

[alexsapran]

The task is to review this line.
https://github.com/elastic/beats/pull/40360/files#diff-d9a956120ac84289d2650137f64bd8f0893331de05e44cc41899dd984c9e0d48

right @pierrehilbert?

[alexsapran]

This was deprecated in

v4.0.0+ requires an explicit actions/setup-go installation step before using this action: uses: actions/setup-go@v5. The skip-go-installation option has been removed.

But we are running version v1.55.2, so I am not sure removing this achieves what you want.
Why we wan to remove this line?

[sachin-frayne]

I will revert this change now.

[pierrehilbert]

Yes this is where your review is needed, the rest of the PR has been reviewed by @belimawr

[sachin-frayne]

I have added skip-go-installation: true back into .github/workflows/golangci-lint.yml.

I removed it while trying to fix some CI issues since it will be deprecated anyways, but since current version is still using it, I have no reason to change it.

[sachin-frayne]

@pierrehilbert and @belimawr

Since I have reverted the change with the skip-go-installation can this move onto the next stage, I believe I am waiting for a reviewer approval.

Thank you.

[sachin-frayne]

Hey @belimawr, :)

Thank you for your review and the suggestions, you are correct the header should only be set if the apikey is not an empty string, I have implemented an if statement around the function.

Please review and let me know if you are happy with the changes?

[sachin-frayne]

Hello @VihasMakwana:

I am not sure, but I believe I am still waiting for your approval.

[pierrehilbert]

@sachin-frayne as @VihasMakwana and @belimawr are in the same team, one of them you should be enough.
Looks like you need someone from @elastic/stack-monitoring

[consulthys]

@sachin-frayne is @elastic/stack-monitoring :-)

[pierrehilbert]

@sachin-frayne is @elastic/stack-monitoring :-)

I still think you need someone from your team to approve the change according to the codeowners.