Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Beats to the current Sarama fork #41655

Open
wants to merge 24 commits into
base: main
Choose a base branch
from

Conversation

khushijain21
Copy link
Contributor

@khushijain21 khushijain21 commented Nov 18, 2024

This PR updates Beats to the current version of Elastic's Sarama fork. It upgrades sarama version to 1.43.3.

The import path has been changed from github.com/Shopify/sarama to github.com/elastic/sarama due to change in ownership.

@khushijain21 khushijain21 requested review from a team as code owners November 18, 2024 06:42
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Nov 18, 2024
Copy link
Contributor

mergify bot commented Nov 18, 2024

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b kafka upstream/kafka
git merge upstream/main
git push upstream kafka

Copy link
Contributor

mergify bot commented Nov 18, 2024

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @khushijain21? 🙏.
For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit

Copy link
Contributor

mergify bot commented Nov 18, 2024

backport-8.x has been added to help with the transition to the new branch 8.x.
If you don't need it please use backport-skip label and remove the backport-8.x label.

@mergify mergify bot added the backport-8.x Automated backport to the 8.x branch with mergify label Nov 18, 2024
@khushijain21 khushijain21 added Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team and removed backport-8.x Automated backport to the 8.x branch with mergify labels Nov 18, 2024
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Nov 18, 2024
@mergify mergify bot added the backport-8.x Automated backport to the 8.x branch with mergify label Nov 18, 2024
@shmsr
Copy link
Member

shmsr commented Nov 18, 2024

qq: Why don't we directly point to the fork i.e., github.com/elastic/sarama@beats-fork? Asking this because we have faced some problem with this before.

See the PR description: #39840 (comment). As it was breaking for the apm team, upon investigation, I found the problem and that's why we moved to using the fork directly. If required, I can also send the related Slack conversation in Slack.

So basically what'd happen is anyone using beats as a dependency, the replace directive won't work for them.

@khushijain21
Copy link
Contributor Author

I think we can. @faec could you please confirm? Any particular reason we don't directly use the fork

@faec
Copy link
Contributor

faec commented Nov 18, 2024

could you please confirm? Any particular reason we don't directly use the fork

I don't see a reason we can't, but the reason we don't is probably that we would prefer to revert to upstream someday. Though there admittedly has not been much movement on that, so I don't think that's an overriding concern if it simplifies dependency handling overall.

go.mod Outdated
@@ -18,7 +18,7 @@ require (
github.com/Microsoft/go-winio v0.6.2
github.com/PaesslerAG/gval v1.2.2
github.com/PaesslerAG/jsonpath v0.1.1
github.com/Shopify/sarama v1.27.0
github.com/Shopify/sarama v1.27.0 // indirect
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the indirect dependency coming from here? (Do we pick up both versions in the actual build?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The output of go mod why github.com/Shopify/sarama says:

# github.com/Shopify/sarama
github.com/elastic/beats/v7/metricbeat/module/kafka/consumergroup
github.com/elastic/beats/v7/metricbeat/module/kafka/consumergroup.test
github.com/bsm/sarama-cluster
github.com/Shopify/sarama

The sarama-cluster package is only used in a single integration test.

@mauri870
Copy link
Member

As a side note, you can automate changes in import paths using gofmt:

gofmt -w -r '"github.com/Shopify/sarama" -> "github.com/IBM/sarama"' .

@pierrehilbert pierrehilbert added the Team:Obs-InfraObs Label for the Observability Infrastructure Monitoring team label Nov 21, 2024
@pierrehilbert
Copy link
Collaborator

@lalit-satapathy could we have someone from your team reviewing here please?

Copy link
Contributor

mergify bot commented Nov 22, 2024

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b kafka upstream/kafka
git merge upstream/main
git push upstream kafka

@@ -57,7 +57,16 @@ var (
"2.4": sarama.V2_4_0_0,
"2.5": sarama.V2_5_0_0,
"2.6": sarama.V2_6_0_0,
"2.7": sarama.V2_7_0_0,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-8.x Automated backport to the 8.x branch with mergify Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team Team:Obs-InfraObs Label for the Observability Infrastructure Monitoring team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants