Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Issue]: Trouble casting some logicalTypes into Number while using Value Schema File Serializer Config #313

Closed
2 tasks done
davidgarciago opened this issue Nov 29, 2022 · 1 comment
Assignees

Comments

@davidgarciago
Copy link
Contributor

davidgarciago commented Nov 29, 2022

What happened?

Value Schema File Serializer Config is having trouble to map logical types int_date and long_timestamp-millis to java.lang.Number when using net.coru.kloadgenserializer.GenericAvroRecordSerializer and net.coru.kloadgenserializer.GenericAvroRecordSerializer.

image

image

avro-schemas.zip
ThreadGroup-Barcodes_Dufry_v1.jmx.zip

KloadGen Version

KloadGen 5.1.3

Relevant log output

2022-11-29 12:29:30,226 INFO o.a.k.c.u.AppInfoParser: Kafka commitId: 37edeed0777bacb3
2022-11-29 12:29:30,226 INFO o.a.k.c.u.AppInfoParser: Kafka startTimeMs: 1669721370226
2022-11-29 12:29:30,239 INFO o.a.k.c.Metadata: [Producer clientId=] Resetting the last seen epoch of partition raw.retailj.ksqldedup.item.v1-0 to 0 since the associated topicId changed from null to cQGTjF5rQoivVrdi9f0ksg
2022-11-29 12:29:30,239 INFO o.a.k.c.Metadata: [Producer clientId=] Cluster ID: wUkh2grTSK-WlIjxb8f4sw
2022-11-29 12:29:30,240 ERROR o.a.j.t.JMeterThread: Error while processing sampler: 'Java Request'.
org.apache.kafka.common.errors.SerializationException: Can't convert value of class org.apache.avro.generic.GenericData$Record to class net.coru.kloadgen.serializer.GenericAvroRecordSerializer specified in value.serializer
	at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:948) ~[kloadgen-5.1.2.jar:?]
	at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:905) ~[kloadgen-5.1.2.jar:?]
	at net.coru.kloadgen.sampler.KafkaProducerSampler.runTest(KafkaProducerSampler.java:141) ~[kloadgen-5.1.2.jar:?]
	at org.apache.jmeter.protocol.java.sampler.JavaSampler.sample(JavaSampler.java:197) ~[ApacheJMeter_java.jar:5.5]
	at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:651) ~[ApacheJMeter_core.jar:5.5]
	at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:570) ~[ApacheJMeter_core.jar:5.5]
	at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:501) ~[ApacheJMeter_core.jar:5.5]
	at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:268) ~[ApacheJMeter_core.jar:5.5]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
Caused by: java.lang.ClassCastException: class java.time.Instant cannot be cast to class java.lang.Number (java.time.Instant and java.lang.Number are in module java.base of loader 'bootstrap') in field DATE_TIME_CREATED
	at org.apache.avro.generic.GenericDatumWriter.addClassCastMsg(GenericDatumWriter.java:191) ~[kloadgen-5.1.2.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:229) ~[kloadgen-5.1.2.jar:?]
	at org.apache.avro.specific.SpecificDatumWriter.writeField(SpecificDatumWriter.java:110) ~[kloadgen-5.1.2.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:210) ~[kloadgen-5.1.2.jar:?]
	at org.apache.avro.specific.SpecificDatumWriter.writeRecord(SpecificDatumWriter.java:84) ~[kloadgen-5.1.2.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:131) ~[kloadgen-5.1.2.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:83) ~[kloadgen-5.1.2.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73) ~[kloadgen-5.1.2.jar:?]
	at net.coru.kloadgen.serializer.GenericAvroRecordSerializer.serialize(GenericAvroRecordSerializer.java:33) ~[kloadgen-5.1.2.jar:?]
	at net.coru.kloadgen.serializer.GenericAvroRecordSerializer.serialize(GenericAvroRecordSerializer.java:44) ~[kloadgen-5.1.2.jar:?]
	at net.coru.kloadgen.serializer.GenericAvroRecordSerializer.serialize(GenericAvroRecordSerializer.java:20) ~[kloadgen-5.1.2.jar:?]
	at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:945) ~[kloadgen-5.1.2.jar:?]
	... 8 more
Caused by: java.lang.ClassCastException: class java.time.Instant cannot be cast to class java.lang.Number (java.time.Instant and java.lang.Number are in module java.base of loader 'bootstrap')
	at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:160) ~[kloadgen-5.1.2.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:81) ~[kloadgen-5.1.2.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:221) ~[kloadgen-5.1.2.jar:?]
	at org.apache.avro.specific.SpecificDatumWriter.writeField(SpecificDatumWriter.java:110) ~[kloadgen-5.1.2.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:210) ~[kloadgen-5.1.2.jar:?]
	at org.apache.avro.specific.SpecificDatumWriter.writeRecord(SpecificDatumWriter.java:84) ~[kloadgen-5.1.2.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:131) ~[kloadgen-5.1.2.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:83) ~[kloadgen-5.1.2.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73) ~[kloadgen-5.1.2.jar:?]
	at net.coru.kloadgen.serializer.GenericAvroRecordSerializer.serialize(GenericAvroRecordSerializer.java:33) ~[kloadgen-5.1.2.jar:?]
	at net.coru.kloadgen.serializer.GenericAvroRecordSerializer.serialize(GenericAvroRecordSerializer.java:44) ~[kloadgen-5.1.2.jar:?]
	at net.coru.kloadgen.serializer.GenericAvroRecordSerializer.serialize(GenericAvroRecordSerializer.java:20) ~[kloadgen-5.1.2.jar:?]
	at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:945) ~[kloadgen-5.1.2.jar:?]
	... 8 more
2022-11-29 12:29:30,241 INFO o.a.j.t.JMeterThread: Thread is done: Kloadgen 1.5.3 1-1
2022-11-29 12:29:30,241 INFO o.a.j.t.JMeterThread: Thread finished: Kloadgen 1.5.3 1-1
2022-11-29 12:29:30,241 INFO o.a.j.e.StandardJMeterEngine: Notifying test listeners of end of test
2022-11-29 12:29:30,241 INFO o.a.k.c.p.KafkaProducer: [Producer clientId=] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms.
2022-11-29 12:29:30,243 INFO o.a.k.c.m.Metrics: Metrics scheduler closed

Have you added your JMX Testplan or a replica?

  • I have added a JMX testplan

Have you added your Schema or a replica?

  • I have added the Schema
@davidgarciago davidgarciago self-assigned this Nov 29, 2022
@github-actions
Copy link

Thank you for collaborating with the project by giving us feedback!!' You will help us a lot if you can attach an example AVRO and an example Test Plan in order to be able to reproduce the error, if that is the case. Cheers

davidgarciago added a commit that referenced this issue Dec 2, 2022
davidgarciago added a commit that referenced this issue Dec 5, 2022
@davidgarciago davidgarciago changed the title [Issue]: long_timestamp-millis having trouble to cast to java.lang.Number using Value Schema File Serializer Config [Issue]: Trouble casting some logicalTypes into Number while using Value Schema File Serializer Config Dec 14, 2022
davidgarciago added a commit that referenced this issue Dec 15, 2022
davidgarciago added a commit that referenced this issue Dec 15, 2022
davidgarciago added a commit that referenced this issue Dec 15, 2022
davidgarciago added a commit that referenced this issue Dec 19, 2022
)

* 303 docker compose for securing kafka first attempt

* 303 changed docker compose SSL to PLAINTEXT

* #303 kafka Auth Configuration

* #303 added user for producer to connect to kafka

* #313 solved serializers casting problems using SpecificDatumWritter

* #313 removed unused asignment to DatumWriter

* #313 updated pom.xml

* #313 datum writer changed to final

* #313 added new test for serializers with logicaltypes and removed unused specific datum writer in serializers

* #313 Solving checkstyle problems

* #313 moving avro logical types conversion config to avro serializers

* #313 checkstyle errors fix

* #313 changed incorrect pom version

Co-authored-by: Jose Enrique García Maciñeiras <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant