Benthos 4.9.1 go build issues - Kafka Input / Azure Output

[ChrisSamo632]

Including github.com/benthosdev/benthos/v4 v4.9.1 in a go.mod file and running do get -u && go mod tidy && go build fails with:

github.com/benthosdev/benthos/v4/internal/impl/kafka

/go/pkg/mod/github.com/benthosdev/benthos/[email protected]/internal/impl/kafka/input_sarama_kafka_parts.go:191:71: not enough arguments in call to offsetPutReq.AddBlock
have (string, int32, int64, int64, string)
want (string, int32, int64, int32, int64, string)

and

github.com/benthosdev/benthos/v4/internal/impl/azure

/go/pkg/mod/github.com/benthosdev/benthos/[email protected]/internal/impl/azure/output_table_storage.go:257:25: table.Create undefined (type *aztables.Client has no field or method Create)

Using go: 1.19.

Works fine with github.com/benthosdev/benthos/v4 v4.8.0

[Jeffail]

@ChrisSamo632 both packages have introduced breaking API changes so you need to ensure your own go.mod pins the same versions rather than just using go get -u to pull the very latest:

• sarama v1.30.1: https://github.com/benthosdev/benthos/blob/main/go.mod#L22
• aztables v0.6.0: https://github.com/benthosdev/benthos/blob/main/go.mod#L10

In the case of azure it's understandable as their packages are sub-v1, but in the case of sarama they broke API compatibility without incrementing their major version which is why we have this unfortunate headache. I've opened an issue regarding sarama here: IBM/sarama#2358, until that's resolved we won't be able to support the latest "stable" package version.

I'm going to pin this issue (thanks for raising it) as it's extremely likely others will hit this 😢

[mfamador]

The #1502 PR will upgrade aztables lib to its latest version.

[ChrisSamo632]

Thanks for the fast reply. The suggested pin for sarama does seem to get over the Kafka Input problem, but downgrading the aztables leaves me with:

github.com/Azure/azure-sdk-for-go/sdk/data/aztables/internal
../../../../../../go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/data/[email protected]/internal/models.go:456:30: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)

My go.mod is still set to go 1.19. I've tried some of the other aztables versions between 0.6.0 and 1.0.1 but get differing errors, including the Creata issue in the original description (from aztables 0.8.0 onwards).

[ChrisSamo632]

Ah, OK... so I think my go.mod file had become polluted with some old/incorrect dependencies - Removing all the //implicit requirements and just keeping the few I actually need, then running go mod tidy has pulled in the correct dependencies and re-generated go.sum without issue - go build works again now and I'm using the correct sarama and aztables dependency versions.