Primitive Keys: Support CAST in join criteria #4130
Comments
big-andy-coates
added a commit
to big-andy-coates/ksql
that referenced
this issue
Dec 13, 2019
Adds support for using primitive types in joins. BREAKING CHANGE: Some existing joins may now fail and the type of `ROWKEY` in the result schema of joins may have changed. When `ROWKEY` was always a `STRING` it was possible to join an `INTEGER` column with a `BIGINT` column. This is no longer the case. A `JOIN` requires the join columns to be of the same type. (See confluentinc#4130 which tracks adding support for being able to `CAST` join criteria). Where joining on two `INT` columns would previously have resulted in a schema containing `ROWKEY STRING KEY`, it would not result in `ROWKEY INT KEY`.
2 tasks
big-andy-coates
added a commit
that referenced
this issue
Dec 17, 2019
* chore: primitive key support in JOINs Adds support for using primitive types in joins. BREAKING CHANGE: Some existing joins may now fail and the type of `ROWKEY` in the result schema of joins may have changed. When `ROWKEY` was always a `STRING` it was possible to join an `INTEGER` column with a `BIGINT` column. This is no longer the case. A `JOIN` requires the join columns to be of the same type. (See #4130 which tracks adding support for being able to `CAST` join criteria). Where joining on two `INT` columns would previously have resulted in a schema containing `ROWKEY STRING KEY`, it would not result in `ROWKEY INT KEY`.
apurvam
changed the title
|
Dropping a note here, but I need to explore more in depth - I'm wondering if this was properly implemented in first place. The partitioning strategy for Strings may not be the same as for Integers (for example) so the co-partitioning requirement may have been broken to begin with. |
|
As discussed, I still think its worthwhile |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Previously, as all keys were of type
STRINGyou could join on two columns of different types, e.g. aSTRINGcolumn that you know to contain a number with anINTEGERcolumn, or anINTEGERcolumn with aBIGINTcolumn.With primitive key support this is no longer the case. The
STRINGcontaining a number won't necessarily be the same as aINTEGER. Hence the join is not allowed.This can break users existing joins, with no easy way for them to fix because we don't support arbitrary expressions in the join criteria, i,.e. they can't do:
We could consider supporting casts, or just any expression, in the join criteria.
The text was updated successfully, but these errors were encountered: