Schema Registry features meta-issue

[BenPope]

Schema Registry is a work in progress, below is a list of features that we are aware of, and whether they have been implemented.

Endpoints

• schemas:
  • GET /schemas
  • GET /schemas/types
  • GET /schemas/ids/{id}
  • GET /schemas/ids/{id}/subjects Retrieve subjects by schema id #12989
  • GET /schemas/ids/{id}/versions schema_registry: Introduce GET /schemas/ids/{id}/versions #2065
• subjects:
  • GET /subjects
  • GET /subjects/{subject}/versions
  • POST /subjects/{subject}/versions
  • GET /subjects/{subject}/versions/{version}
  • GET /subjects/{subject}/versions/latest
  • DELETE /subjects/{subject}
  • DELETE /subjects/{subject}/versions/{version}
  • DELETE /subjects/{subject}/versions/latest
  • POST /subjects/{subject} schema_registry: Introduce POST /subjects/{subject} #2049
  • GET /subjects/{subject}/versions/{version}/schema
  • GET /subjects/{subject}/versions/{version}/referencedBy (v21.11.3) schema_registry: Support GET /subjects/{subject}/versions/{version}/referencedBy #3299
• compatibility:
  • POST /compatibility/subjects/{subject}/versions/[version}
  • POST /compatibility/subjects/{subject}/versions
  • Support ?verbose=true
• config:
  • GET /config
  • GET /config/{subject}
  • PUT /config
  • PUT /config/{subject}
• mode:
  • modes:
    1. READWRITE
    2. READONLY [CORE-60] Schema Registry: Support /mode #17952
    3. IMPORT
  • GET /mode
  • GET /mode/{subject} [CORE-60] Schema Registry: Support /mode #17952
  • PUT /mode [CORE-60] Schema Registry: Support /mode #17952
  • PUT /mode/{subject} [CORE-60] Schema Registry: Support /mode #17952
  • DELETE /mode/{subject} [CORE-60] Schema Registry: Support /mode #17952
• references (on import)
  • AVRO (v22.1.1) Avro schema references on import #4146
  • JSON
  • PROTOBUF (v21.11.1)
• exporters
  • GET /exporters
  • GET /contexts
  • POST /exporters
  • PUT /exporters/{name}
  • PUT /exporters/{name}/config
  • GET /exporters/{name}
  • GET /exporters/{name}/status
  • GET /exporters/{name}/config
  • PUT /exporters/{name}/pause
  • PUT /exporters/{name}/reset
  • PUT /exporters/{name}/resume
  • DELETE /exporters/{name}

Schema Types

• AVRO
• JSON
• PROTOBUF (v21.11.1)

Endpoints on SwaggerHub

JIRA Link: CORE-692

[raphaelauv]

Exposing the "timestamp" of the publication of the schema in the endpoints would be a nice feature since this info is available in the Kafka topic

-> confluentinc/schema-registry#1899

[emaxerrno]

Exposing the "timestamp" of the publication of the schema in the endpoints would be a nice feature since this info is available in the Kafka topic

-> confluentinc/schema-registry#1899

This is a nice idea. We should.

[dbwrangler]

When do you expect the 'referencedby' endpoint to be available and I'm guessing your protobuf implementation is already reliant on working with schema references?

[BenPope]

When do you expect the 'referencedby' endpoint to be available and I'm guessing your protobuf implementation is already reliant on working with schema references?

@dbwrangler References are hooked up internally for Protobuf (see the blog post), so that should all work. For Avro, the references will be stored internally, but imports aren't actually working yet.

It's not yet planned, but since you asked nicely, I'll knock out referencedBy for you one evening, hopefully it'll be in the next release.

[JayGhiya]

Does schema registry work with confluent serialization library which does registration /serialization/deserialization. Using KafkaAvroSerializer and schema registry url through AbstractKafkaSchemaSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG?

[BenPope]

Does schema registry work with confluent serialization library which does registration /serialization/deserialization. Using KafkaAvroSerializer and schema registry url through AbstractKafkaSchemaSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG?

Yes, that's the intention. Currently it doesn't support references for AVRO.

[JayGhiya]

Thought so tried did not work. What Is the eta on this? So right now we can make it work by doing a byte array serialise and deserialise. Correct ?

…

On 16-Jan-2022, at 7:34 PM, Ben Pope ***@***.***> wrote: Does schema registry work with confluent serialization library which does registration /serialization/deserialization. Using KafkaAvroSerializer and schema registry url through AbstractKafkaSchemaSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG? Yes, that's the intention. Currently it doesn't support references for AVRO. — Reply to this email directly, view it on GitHub <#1878 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF5IO3KVMRLDEPX4XBGIUV3UWLF7HANCNFSM5A34R5PA>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.

[BenPope]

When do you expect the 'referencedby' endpoint to be available and I'm guessing your protobuf implementation is already reliant on working with schema references?

@dbwrangler - v21.11.3 has been released with the endpoint.

[JayGhiya]

Guys so is there any eta on getting json schema support?

[jakthom]

+1, would <3 jsonschema support.

[dangoslen]

Are there plans to have a UI for this for fully managed Redpanda?

[vuldin]

Are there plans to have a UI for this for fully managed Redpanda?

There are definitely plans, but I don't know what the timeline is. Another related feature is schema registry support within the rpk CLI. I'll post once I find more details on the plan and timeline for schema registry UI.

[raphaelauv]

@dangoslen -> https://redpanda.com/blog/redpanda-acquires-cloudhut-kowl-data-streaming-kafka-ui/

[robgtaylor]

Are there plans to introduce a feature like Confluent's broker-side schema validation ? This returns an error to a producer when they write a message to a topic with a schema if the message doesn't validate against the schema. It prevents non-compliant messages being written to topics:

https://docs.confluent.io/platform/current/schema-registry/schema-validation.html