Export Schema Registry client and the serdes functionality

[mostafa]

In this PR I tried to address the following issues. While doing so, I created a SchemaRegistry object and updated the serdes functionality to export the whole Schema Registry client and the serialize and deserialize functions to the JS code. Contrary to my initial testing, the new changes make the code faster and more performant. And the following is the result. The updated examples can be found in the scripts directory.

import { SchemaRegistry } from "k6/x/kafka";

const schemaRegistry = SchemaRegistry({ url: "...", basicAuth: {...}, tls: {...} });

schemaRegistry.getSchema({ version: 0, subject: "..." }); // 0 means latest schema
schemaRegistry.createSchema({ version: 0, subject: "...", schema: "...", schemaType: "AVRO" });
schemaRegistry.getSubjectName({ element: "...", topic: "...", schema: "...", subjectNameStrategy: "..." });
// You can pass a string, byte array, and JSON objects as data. Regardless of the data, the schemaType must also be passed.
// For schemaless data, like string, only the data and the schemaType (SCHEMA_TYPE_STRING) should be passed.
schemaRegistry.serialize({ schema: "...", data: "...", schemaType: "AVRO" });
schemaRegistry.deserialize({ schema: "...", data: "...", schemaType: "AVRO" });

• Should serdes functionality be exported to JS? #50
• Retrieve a specific version of the Schema from Schema Registery #53
• Retrieve schema by a known ID or subject, without hard-coding it in the script #54
• Should Schema Registry client be exported to JS? #55

And this is the result of running the famous scripts/test_json.js for a minute and 50 VUs, which is impressive.

Terminal output of the test

$ ./k6 run -d 60s --vus 50 scripts/test_json.js

          /\      |‾‾| /‾‾/   /‾‾/
     /\  /  \     |  |/  /   /  /
    /  \/    \    |     (   /   ‾‾\
   /          \   |  |\  \ |  (‾)  |
  / __________ \  |__| \__\ \_____/ .io

  execution: local
     script: scripts/test_json.js
     output: -

  scenarios: (100.00%) 1 scenario, 50 max VUs, 1m30s max duration (incl. graceful stop):
           * default: 50 looping VUs for 1m0s (gracefulStop: 30s)


running (1m04.4s), 00/50 VUs, 20170 complete and 0 interrupted iterations
default ✓ [======================================] 50 VUs  1m0s

     ✓ 10 messages are received
     ✓ Topic equals to xk6_kafka_json_topic
     ✓ Key contains key/value and is JSON
     ✓ Value contains key/value and is JSON
     ✓ Header equals {'mykey': 'myvalue'}
     ✓ Time is past
     ✓ Partition is zero
     ✓ Offset is gte zero
     ✓ High watermark is gte zero

     █ teardown

     checks.........................: 100.00% ✓ 181530       ✗ 0
     data_received..................: 0 B     0 B/s
     data_sent......................: 0 B     0 B/s
     iteration_duration.............: avg=153.45ms min=6.01ms med=26.8ms  max=8.14s   p(90)=156.3ms p(95)=206.4ms
     iterations.....................: 20170   313.068545/s
     kafka.reader.dial.count........: 50      0.776075/s
     kafka.reader.dial.seconds......: avg=171.22µs min=0s     med=0s      max=1.09s   p(90)=0s      p(95)=0s
   ✓ kafka.reader.error.count.......: 0       0/s
     kafka.reader.fetch_bytes.max...: 1000000 min=1000000    max=1000000
     kafka.reader.fetch_bytes.min...: 1       min=1          max=1
     kafka.reader.fetch_wait.max....: 200ms   min=200ms      max=200ms
     kafka.reader.fetch.bytes.......: 58 MB   897 kB/s
     kafka.reader.fetch.size........: 147167  2284.25179/s
     kafka.reader.fetches.count.....: 107     1.6608/s
     kafka.reader.lag...............: 1519055 min=0          max=2436190
     kafka.reader.message.bytes.....: 40 MB   615 kB/s
     kafka.reader.message.count.....: 201749  3131.446006/s
     kafka.reader.offset............: 4130    min=11         max=5130
     kafka.reader.queue.capacity....: 1       min=1          max=1
     kafka.reader.queue.length......: 1       min=0          max=1
     kafka.reader.read.seconds......: avg=96.5ms   min=0s     med=0s      max=59.37s  p(90)=0s      p(95)=0s
     kafka.reader.rebalance.count...: 0       0/s
     kafka.reader.timeouts.count....: 57      0.884725/s
     kafka.reader.wait.seconds......: avg=102.71µs min=0s     med=0s      max=85.71ms p(90)=0s      p(95)=0s
     kafka.writer.acks.required.....: 0       min=0          max=0
     kafka.writer.async.............: 0.00%   ✓ 0            ✗ 2017000
     kafka.writer.attempts.max......: 0       min=0          max=0
     kafka.writer.batch.bytes.......: 441 MB  6.8 MB/s
     kafka.writer.batch.max.........: 1       min=1          max=1
     kafka.writer.batch.size........: 2017000 31306.854525/s
     kafka.writer.batch.timeout.....: 0s      min=0s         max=0s
   ✓ kafka.writer.error.count.......: 0       0/s
     kafka.writer.message.bytes.....: 883 MB  14 MB/s
     kafka.writer.message.count.....: 4034000 62613.709051/s
     kafka.writer.read.timeout......: 0s      min=0s         max=0s
     kafka.writer.retries.count.....: 0       0/s
     kafka.writer.wait.seconds......: avg=0s       min=0s     med=0s      max=0s      p(90)=0s      p(95)=0s
     kafka.writer.write.count.......: 4034000 62613.709051/s
     kafka.writer.write.seconds.....: avg=523.21µs min=4.84µs med=14.48µs max=4.05s   p(90)=33.85µs p(95)=42.68µs
     kafka.writer.write.timeout.....: 0s      min=0s         max=0s
     vus............................: 7       min=7          max=50
     vus_max........................: 50      min=50         max=50

Tasks:

• Update API docs
• Add more tests to increase coverage
• Add Writer class test
• Add Reader class test

[mostafa]

@fdahunsibread @enamrik @kusumkappdirect
Hey! Since you showed interest in this feature on the linked issues, I'd be happy to have your opinion on this or a review if you have time.

[sonarcloud]

SonarCloud Quality Gate failed.   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
29 Code Smells

No Coverage information
5.9% Duplication