feat: Introduce schema definition. #19
Conversation
liurenjie1024
requested review from
JanKaul,
Xuanwo,
amogh-jahagirdar and
Fokko
| } | ||
|
|
||
| /// Set the field's initial default value. | ||
| pub fn with_write_default(mut self, value: impl ToString) -> Self { |
There was a problem hiding this comment.
write_default and initial_default seems should be a value?
There was a problem hiding this comment.
Yes, but currently Value is not defined yet 🤪
|
The Python and java implementation treats schema as immutable data structures, I think we should also follow that. Mutable makes things complicated, especially when we have indexes. For example the name indexing following. |
Got it, makes sense now. |
|
Should we also include identifier-field-ids? |
Just took a look at it. We need visitor pattern to verify them, for example the types, etc. So I want to postpone it after we introduce schema visitor. |
|
I don't entirely understand. Could you elaborate why we would need the visitor pattern for the identifier-field-ids? I was thinking about something similar to the serialized representation. |
I mean the verification check, you can find the implementation in java: https://github.com/apache/iceberg/blob/1bb853191fd378fb1dfda5a5cb297475b7fc204b/api/src/main/java/org/apache/iceberg/Schema.java#L104 cc @Fokko @JanKaul Why I don't include |
| r#struct: StructType, | ||
| schema_id: i32, | ||
| highest_field_id: i32, | ||
| } |
There was a problem hiding this comment.
I think the identifier-field-ids are missing: https://iceberg.apache.org/spec/#identifier-field-ids
Initial schema definition.
SchemaVisitor, name indexes will come later.