Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql/types: types.EnumMetadata implements encoding.TextMarshaler interface #72640

Conversation

chengxiong-ruan
Copy link
Contributor

@chengxiong-ruan chengxiong-ruan commented Nov 11, 2021

Fixes #63379

types.EnumMetadata needs to implement encoding.TextMarshaler
interface so that goto/protobuf won't panic when text marshaling
protobuf struct has child field of type types.EnumMetadata.
See the issue for more details why it would fail without this
fix.

Release note: None

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@chengxiong-ruan chengxiong-ruan force-pushed the enummetadata_implements_textmarshaler_interface branch from 395ae5c to 309f765 Compare November 11, 2021 14:14
@chengxiong-ruan
Copy link
Contributor Author


pkg/sql/types/types.go, line 231 at r1 (raw file):

	for i, bs := range e.PhysicalRepresentations {
		if i > 0 {
			bf.WriteString(",")

I'm totally open to remove this kind of logic to insert separator, we would just have an extra comma at the end...

Copy link
Contributor

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The more I think about it, the less I think I care about actually populating these. The biggest reason is that the type metadata in the cases we're dealing with should not be populated. The point of the types.T having the embedded internal type and then the type metadata is to support in-memory operations and to make sure we never, ever serialize the user-defined metadata to disk. The reason for all of this is that the user-defined metadata can change and needs to be "hydrated" into the type descriptor. This is a topic we can unpack at some future point.

I think what I'd prefer is that we just skip the user defined type metadata altogether. To do this, I'd do the following and be done with it.

// MarshalText is implemented because ...
func (e *T) MarshalText() (text []byte, err error) {
	var buf bytes.Buffer
        if err := proto.MarshalText(&buf, e.InternalType); err != nil {
            return nil, err
        }
        return buf.Bytes(), nil
}

It'd then be good to write some tests of marshaling protos which use types and may or may not have populated user-defined type data to string.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @chengxiong-ruan and @postamar)


pkg/sql/types/types.go, line 228 at r1 (raw file):

	// Convert PhysicalRepresentations to byte slice representation of string like
	// "PhysicalRepresentations<[0x12,0x34],[0x56]>".
	bf.WriteString("PhysicalRepresentations<")

If we're going to go for a prototext format, may as well go all the way. However,

@chengxiong-ruan chengxiong-ruan force-pushed the enummetadata_implements_textmarshaler_interface branch from 309f765 to 2b0c053 Compare November 11, 2021 16:33
Copy link
Contributor Author

@chengxiong-ruan chengxiong-ruan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good call out and thanks for the context! 👍

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @chengxiong-ruan and @postamar)

@chengxiong-ruan
Copy link
Contributor Author


pkg/sql/types/types.go, line 228 at r1 (raw file):

Previously, ajwerner wrote…

If we're going to go for a prototext format, may as well go all the way. However,

this part is removed :)

@chengxiong-ruan chengxiong-ruan force-pushed the enummetadata_implements_textmarshaler_interface branch from 2b0c053 to 79f8936 Compare November 11, 2021 16:45
…face

Fixes cockroachdb#63379

Having `types.T` to implement the `encoding.TextMarshaler` interface
to avoid panics when gogo/protobuf tries to text marshal any protobuf
struct which has direct/indirect `types.T` child. The panics was caused
by non-protobuf types in `types.EnumMetadata`. This fix implements
the `MarshalText` method to ignore user defined metadata. Only InternalType
is dumped.

Release note: None
@chengxiong-ruan chengxiong-ruan force-pushed the enummetadata_implements_textmarshaler_interface branch from 79f8936 to 4271af4 Compare November 11, 2021 18:03
@chengxiong-ruan chengxiong-ruan requested a review from a team as a code owner November 11, 2021 18:03
@chengxiong-ruan
Copy link
Contributor Author


pkg/sql/opt/exec/execbuilder/testdata/show_trace_nonmetamorphic, line 42 at r3 (raw file):

----
batch flow coordinator  CPut /NamespaceTable/30/1/53/29/"kv"/4/1 -> 54
batch flow coordinator  CPut /Table/3/1/54/2/1 -> table:<name:"kv" id:54 version:1 modification_time:<> parent_id:53 unexposed_parent_schema_id:29 columns:<name:"k" id:1 type:<family: IntFamily width: 64 precision: 0 locale: "" visible_type: 0 oid: 20 time_precision_is_set: false > nullable:false hidden:false inaccessible:false generated_as_identity_type:NOT_IDENTITY_COLUMN virtual:false pg_attribute_num:0 alter_column_type_in_progress:false system_column_kind:NONE > columns:<name:"v" id:2 type:<family: IntFamily width: 64 precision: 0 locale: "" visible_type: 0 oid: 20 time_precision_is_set: false > nullable:true hidden:false inaccessible:false generated_as_identity_type:NOT_IDENTITY_COLUMN virtual:false pg_attribute_num:0 alter_column_type_in_progress:false system_column_kind:NONE > next_column_id:3 families:<name:"primary" id:0 column_names:"k" column_names:"v" column_ids:1 column_ids:2 default_column_id:2 > next_family_id:1 primary_index:<name:"kv_pkey" id:1 unique:true version:4 key_column_names:"k" key_column_directions:ASC store_column_names:"v" key_column_ids:1 store_column_ids:2 foreign_key:<table:0 index:0 name:"" validity:Validated shared_prefix_len:0 on_delete:NO_ACTION on_update:NO_ACTION match:SIMPLE > interleave:<> partitioning:<num_columns:0 num_implicit_columns:0 > type:FORWARD created_explicitly:false encoding_type:1 sharded:<is_sharded:false name:"" shard_buckets:0 > disabled:false geo_config:<> predicate:"" > next_index_id:2 privileges:<users:<user_proto:"admin" privileges:2 > users:<user_proto:"public" privileges:0 > users:<user_proto:"root" privileges:2 > owner_proto:"root" version:2 > next_mutation_id:1 format_version:3 state:PUBLIC offline_reason:"" view_query:"" is_materialized_view:false new_schema_change_job_id:0 drop_time:0 replacement_of:<id:0 time:<> > audit_mode:DISABLED drop_job_id:0 create_query:"" create_as_of_time:<> temporary:false partition_all_by:false >

these logic test case were updated with --rewrite

Copy link
Contributor

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 2 of 3 files at r2, 3 of 3 files at r3, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @postamar)

@chengxiong-ruan
Copy link
Contributor Author

bors r+

@craig craig bot merged commit 4300949 into cockroachdb:master Nov 11, 2021
@craig
Copy link
Contributor

craig bot commented Nov 11, 2021

Build succeeded:

@chengxiong-ruan chengxiong-ruan deleted the enummetadata_implements_textmarshaler_interface branch December 2, 2021 02:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sql: calling String() on some ColumnDescriptors (and possibly other types) can panic
3 participants