[CALCITE-6690] Refactor the Arrow adapter type system #4052
Conversation
|
After this pr, for some data types, we can easily identify their precision and scale such as decimal, time, timestamp |
|
| LONG(Primitive.LONG), | ||
| BYTE(Primitive.BYTE), | ||
| SHORT(Primitive.SHORT); | ||
| class ArrowFieldType { |
There was a problem hiding this comment.
I think enums are preferable, since they are typechecked by the compiler.
A Class can be anything.
There was a problem hiding this comment.
Later, I may dynamically pass some parameters according to the data type of arrow, which is difficult to achieve through enumeration.
e.g.
new ArrowFieldType(BigDecimal.class,
((ArrowType.Decimal) arrowType).getPrecision(),
((ArrowType.Decimal) arrowType).getScale())
There was a problem hiding this comment.
Why later if you are refactoring the code now?
Do it correctly the first time.
I am suggesting to have 3 arguments for the enum constructor: Class, precision, scale.
There was a problem hiding this comment.
@mihaibudiu Sorry for the late reply, you are right. I looked at the arrow types and it seems that Class, precision, and scale are sufficient.
I will finish this work on the weekend
There was a problem hiding this comment.
@mihaibudiu Hi. I have a question, how to dynamically pass parameters in enumeration class? Precision and scale are not constants for decimal, timestamp and other types.
There was a problem hiding this comment.
I am not sure I understand the question.
There are lots of enum classes with parameters in the codebase.
See https://github.com/apache/calcite/blob/main/linq4j/src/main/java/org/apache/calcite/linq4j/tree/Primitive.java for example
| switch (vector.getMinorType()) { | ||
| case TINYINT: | ||
| switch (field.getName()) { | ||
| case "tinyIntField": |
There was a problem hiding this comment.
This is a necessary change? What is the difference between decimalField2 and decimalField?
There was a problem hiding this comment.
Required, the scale of decimalfield and decimalfiled2 are different, one is 2, the other is 3, the main purpose is to capture the accuracy of whether it is effective
https://issues.apache.org/jira/browse/CALCITE-6690