API, Core: Add formatVersion() to Table #11587
Conversation
nastra
force-pushed
the
add-format-version-to-table
branch
from
72a169d
to
c543a21
Compare
| @@ -212,6 +212,11 @@ public String toString() { | |||
| return name(); | |||
| } | |||
|
|
|||
| @Override | |||
| public int formatVersion() { | |||
| return table().formatVersion(); | |||
There was a problem hiding this comment.
Do metadata tables have format versions? I would argue no?
There was a problem hiding this comment.
Yeah I don't think it makes sense for metadata tables to have a format version. The entries in the tables could be spread across different "versions" of a table where the underlying table had different format versions.
There was a problem hiding this comment.
so the only table where this would be currently necessary to have is PositionDeletesTable. This is so that we can know in SparkPositionDeletesRewrite (which runs against PositionDeletesTable) whether existing position deletes need to be rewritten to either V2 positional deletes or to DVs
There was a problem hiding this comment.
I think the API makes a lot of sense, it'll shrink all the awkward call points which cast to HasTableOperations just to get the operations. The only doubt I have is if it makes sense for metadata tables to have this API. I don't think it makes complete sense since a metadata table could have entries which span different format versions for the underlying table
nastra
force-pushed
the
add-format-version-to-table
branch
from
61d0631
to
0548923
Compare
|
I've created an alternative impl to this in #11620 where we can achieve the same thing without adjusting the |
|
I think I'd prefer #11620 for now, not a strong opinion, though. |
No description provided.