Problem with Avro and schema registry

[ripla]

We've been starting to test our Kafka based system with xk6-kafka. At first we just used 1 VU and everything worked perfectly. But as we moved to increase to 10, 100 or 1000 VUs we started seeing schema-related errors from the schema registry. I can repro the problem with a small-ish test case.

docker compose up -d
curl -s -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" --data-binary "@./myschema.json" http://localhost:8085/subjects/mytopic-value/versions
docker run --network="mynetwork" --rm -i mostafamoradian/xk6-kafka:latest run - <burstMessages.js

Even with just 10 VU's, we get the same error at some point. The schema registry logs shows no errors.

time="2022-08-05T10:44:10Z" level=warning msg="Failed to create or get schema, manually encoding the data" error="Failed to get schema from schema registry, OriginalError: %!w(*url.Error=&{Get http://schemaregistry:8085/subjects/mytopic-value/versions/latest 0xc0009694a0})"

and

time="2022-08-05T11:58:32Z" level=error msg="Failed to create codec for encoding Avro, OriginalError: %!w(*errors.errorString=&{cannot unmarshal schema JSON: unexpected end of JSON input})" error="Failed to create codec for encoding Avro, OriginalError: %!w(*errors.errorString=&{cannot unmarshal schema JSON: unexpected end of JSON input})"

Originally we had the schemas in the script, but it didn't matter. The error just has POST instead of GET.

Any ideas what to try next, or is a bug the tool?

[ripla]

Gist of files for reproducing: https://gist.github.com/ripla/fe7db19c835ec7a62d9a69a265229ba4

[oscar067]

Hi,

try the branch from this pull request: #147.

I had similar errors and that patch fix them #146

[mostafa]

Hey @ripla,

Although I have no intention of load-testing the schema registry, it happens while accessing it. I exported the Schema Registry class in #149, which makes it easier to call it in the init context. Based on the contribution by @oscar067, I created #151, which will help with this.