Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create Avro sink schema at time of stream/table creation #4219

Closed
vcrfxia opened this issue Jan 4, 2020 · 1 comment · Fixed by #4717
Closed

Create Avro sink schema at time of stream/table creation #4219

vcrfxia opened this issue Jan 4, 2020 · 1 comment · Fixed by #4717

Comments

@vcrfxia
Copy link
Contributor

vcrfxia commented Jan 4, 2020

Is your feature request related to a problem? Please describe.

Currently, when a user creates an Avro sink stream/table, KSQL does not register the Avro schema for the new topic to Schema Registry until the first record is processed. (This is because KSQL relies on the KafkaAvroSerializer to automatically register the schema to Schema Registry, and the serializer is only invoked when a record is processed.) However, this leads to unintuitive behavior such as in #3883:

  • user issues a query that creates a sink stream with Avro format
  • user wishes to define a KSQL table on the sink topic (from the stream above), but the request fails due to Avro schema for message values on topic <topic name> does not exist in the Schema Registry
  • a record is sent to through the Streams app defined by the query
  • user reissues the create table statement, and this time it succeeds

Describe the solution you'd like

KSQL should register the Avro schema with Schema Registry at the time the sink stream/table is defined, rather than waiting until the first record is processed.

Describe alternatives you've considered

N/A

Additional context

N/A

@vcrfxia
Copy link
Contributor Author

vcrfxia commented Jan 4, 2020

Copied over the 0.7.0 milestone from #3883

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants