Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#2678] improvement(test): Add e2e test for jdbc backend #2686

Merged
merged 15 commits into from
Apr 2, 2024

Conversation

yuqi1129
Copy link
Contributor

What changes were proposed in this pull request?

Add JDBC entity store in integration test.

Why are the changes needed?

To test whether JDBC backends works.

Fix: #2678

Does this PR introduce any user-facing change?

N/A.

How was this patch tested?

Existing ITs

@yuqi1129 yuqi1129 self-assigned this Mar 26, 2024
@yuqi1129 yuqi1129 closed this Mar 26, 2024
@yuqi1129 yuqi1129 reopened this Mar 26, 2024
@@ -120,7 +120,7 @@ SchemaPO selectSchemaMetaByCatalogIdAndName(
+ " AND schema_name = #{oldSchemaMeta.schemaName}"
+ " AND metalake_id = #{oldSchemaMeta.metalakeId}"
+ " AND catalog_id = #{oldSchemaMeta.catalogId}"
+ " AND schema_comment = #{oldSchemaMeta.schemaComment}"
+ " AND (schema_comment IS NULL OR schema_comment = #{oldSchemaMeta.schemaComment})"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to fix the problem that schema_comment is nullable

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we set this parameter to Not Null and use "" as the default value?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment null and empty are different for many catalogs, so I don't think it's better than the previous design.

@yuqi1129
Copy link
Contributor Author

After this PR, the backend integration test will take as long as 50 minutes.

@yuqi1129 yuqi1129 requested a review from jerryshao March 27, 2024 02:01
@yuqi1129
Copy link
Contributor Author

@xloya
Can you also assist in taking a look?

@xloya
Copy link
Contributor

xloya commented Mar 27, 2024

After this PR, the backend integration test will take as long as 50 minutes.

I'm not sure if it's because of the driver download. During my internal testing, downloading the mysql driver from https://repo1.maven.org/maven2 was very slow.

@yuqi1129
Copy link
Contributor Author

After this PR, the backend integration test will take as long as 50 minutes.

I'm not sure if it's because of the driver download. During my internal testing, downloading the mysql driver from https://repo1.maven.org/maven2 was very slow.

I'm afraid 50 minutes is as expected. Previous ITs took about 23 minutes on average, as this PR adds another test mode, the total time I think should be double, moreover, the MySQL entity store should be more time-consuming that the KV entity store.

Downdowing jars in CI are very fast as they won't be blocked by GFW.

@yuqi1129
Copy link
Contributor Author

@jerryshao Please help to review it if you have time, thanks

@yuqi1129 yuqi1129 requested review from FANNG1 and xloya April 2, 2024 07:32
@yuqi1129
Copy link
Contributor Author

yuqi1129 commented Apr 2, 2024

@jerryshao Please help to review it if you have time, thanks

Could you please help to review it, thanks

@xloya
Copy link
Contributor

xloya commented Apr 2, 2024

Overall LGTM.

@jerryshao
Copy link
Contributor

I have no further comment, my simple worry is that it will add more CI jobs that make the CI time much longer.

@jerryshao jerryshao merged commit 2bfefb9 into apache:main Apr 2, 2024
25 checks passed
yuqi1129 added a commit to yuqi1129/gravitino that referenced this pull request Apr 2, 2024
yuqi1129 added a commit to yuqi1129/gravitino that referenced this pull request Apr 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Improvement] Add e2e test for JDBC entity backend
4 participants