[Bug] Upload BYTES schema but got NULL schema type

[codelipenghui]

Search before asking

• I searched in the issues and found nothing similar.

Version

Latest master

Minimal reproduce step

root@stg-billing-broker-0:/pulsar# cat schema.txt
{
"type": "BYTES",
"schema": "",
"properties": {}
}

root@stg-billing-broker-0:/pulsar# ./bin/pulsar-admin schemas upload persistent://public/default/asher-test -f schema.txt

root@stg-billing-broker-0:/pulsar# ./bin/pulsar-admin schemas get persistent://public/default/asher-test
{
"version": 4,
"schemaInfo": {
"name": "asher-test",
"schema": "",
"type": "NONE",
"properties": {}
}
}

What did you expect to see?

To keep consistent with the client side.
I think we should prevent the BYTES schema uploading
It's need to have discussion under the mailing list since it will change the existing behavior

What did you see instead?

No

Anything else?

No

Are you willing to submit a PR?

• I'm willing to submit a PR!

[labuladong]

Assign to me please

[codelipenghui]

Thanks @labuladong

[labuladong]

The root cause is BYTES is a negative enum:

pulsar/pulsar-client-api/src/main/java/org/apache/pulsar/common/schema/SchemaType.java

Lines 146 to 155 in 90f6758

	//
	// Schemas that don't have schema info. the value should be negative.
	//
	/**
	* A bytes array.
	*/
	BYTES(-1),

Then convertFromDomainType convert BYTES to NONE

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/schema/SchemaRegistryServiceImpl.java

Lines 592 to 598 in 90f6758

	static SchemaRegistryFormat.SchemaInfo.SchemaType convertFromDomainType(SchemaType type) {
	if (type.getValue() < 0) {
	return SchemaRegistryFormat.SchemaInfo.SchemaType.NONE;
	} else {
	return SchemaRegistryFormat.SchemaInfo.SchemaType.valueOf(type.getValue() + 1);
	}
	}

[labuladong]

I think there are two solutions:

1. Forbid users to upload the BYTES schema. This is the easiest way, don't need to change much code. (recommended)

I tried Java client Producer<byte[]> producer = client.newProducer(Schema.BYTES), this producer won't auto-create a BYTES schema. If we allow users to upload a BYTES schema, that may cause inconsistency.

2. Add a BYTES type schema, that needs to add BYTES type to proto file:

pulsar/pulsar-broker/src/main/proto/SchemaRegistryFormat.proto

Lines 25 to 48 in 90f6758

	message SchemaInfo {
	enum SchemaType {
	NONE = 1;
	STRING = 2;
	JSON = 3;
	PROTOBUF = 4;
	AVRO = 5;
	BOOLEAN = 6;
	INT8 = 7;
	INT16 = 8;
	INT32 = 9;
	INT64 = 10;
	FLOAT = 11;
	DOUBLE = 12;
	DATE = 13;
	TIME = 14;
	TIMESTAMP = 15;
	KEYVALUE = 16;
	INSTANT = 17;
	LOCALDATE = 18;
	LOCALTIME = 19;
	LOCALDATETIME = 20;
	PROTOBUFNATIVE = 21;
	}

And client.newProducer(Schema.BYTES) should also create BYTES type schema in schema factory.

[labuladong]

mail list:
https://lists.apache.org/thread/672zmptfblwjmrf9z8336mk12r7csngf