Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#314] feat(catalog-lakehouse): support Iceberg table interface #350

Merged
merged 2 commits into from
Sep 12, 2023
Merged

[#314] feat(catalog-lakehouse): support Iceberg table interface #350

merged 2 commits into from
Sep 12, 2023

Conversation

FANNG1
Copy link
Contributor

@FANNG1 FANNG1 commented Sep 7, 2023

What changes were proposed in this pull request?

add table interface for Iceberg REST server

Why are the changes needed?

Iceberg REST api spec

Fix: #314

Does this PR introduce any user-facing change?

new api

How was this patch tested?

UT

@FANNG1 FANNG1 marked this pull request as draft September 7, 2023 06:09
@github-actions
Copy link

github-actions bot commented Sep 7, 2023

Code Coverage Report

Overall Project 63.75% -0.05% 🟢
Files changed 91.79% 🟢

Module Coverage
catalog-lakehouse 95.75% -2.83% 🟢
Files
Module File Coverage
catalog-lakehouse IcebergRestUtils.java 100% 🟢
IcebergExceptionMapper.java 100% 🟢
IcebergTableRenameOperations.java 100% 🟢
IcebergTableOperations.java 95.83% -4.17% 🟢
IcebergTableOps.java 88.89% -7.64% 🟢

@FANNG1 FANNG1 marked this pull request as ready for review September 7, 2023 07:27
@FANNG1
Copy link
Contributor Author

FANNG1 commented Sep 8, 2023

@jerryshao , it's ready to review now.

@FANNG1
Copy link
Contributor Author

FANNG1 commented Sep 8, 2023

some works of @yunqing-wei (IcebergCatalog table operations) depends on this pr, @jerryshao please have a review priorly

@jerryshao jerryshao changed the title [#314] feat(iceberg): support table interface [#314] feat(catalog-lakehouse): support Iceberg table interface Sep 11, 2023
@PathParam("namespace") String namespace,
@PathParam("table") String table,
@DefaultValue("all") @QueryParam("snapshots") String snapshots) {
// todo support snapshots
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When will you do this snapshot support.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I try to merge the logic to the CatalogHandlers in upstream, snapshots is a performance feature,does't impact the correctness.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the behavior of the current Iceberg? Does it support snapshot in load table?

@jerryshao
Copy link
Contributor

@yunqing-wei would you please also help to review this, thanks.

@Clearvive
Copy link
Contributor

+1

verifyCreateTableSucc("rename_foo3");
verifyRenameTableFail("rename_foo2", "rename_foo3", 409);
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if we fully cover all the return codes in this UT, if not, I would suggest adding more UTs to cover all the scenarios in Iceberg's REST spec.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll check it completely again today

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add more UTs to cover, take testRenameTable for example, covers the rename related logic:

  • rename succ, 200
  • namespace not exists. 404
  • source table not exists. 404
  • dest table exits 409

the other return codes are tested in TestIcebergExceptionMapper,

  • BadRequest. 400
  • Unauthorized 401
  • Forbidden. 403
  • UnsupportedOperation 406
  • ServiceUnavailable. 503
  • ServerError 500

AuthenticationTimeout is defined in api-spec, but not defined in Iceberg api and not handled in REST client code. not covered now.

@jerryshao jerryshao merged commit eeac24e into apache:main Sep 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Subtask] support table interface in Iceberg REST catalog
4 participants