Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#2879] feat(docs): Add doc for Doris catalog #3051

Merged
merged 10 commits into from
Apr 22, 2024
Merged

Conversation

zhoukangcn
Copy link
Contributor

What changes were proposed in this pull request?

Add doc for Doris catalog

Why are the changes needed?

Fix: #2879

Does this PR introduce any user-facing change?

N/A

How was this patch tested?

N/A

docs/jdbc-doris-catalog.md Outdated Show resolved Hide resolved
docs/jdbc-doris-catalog.md Outdated Show resolved Hide resolved
@zhoukangcn zhoukangcn force-pushed the f-2879 branch 2 times, most recently from b5d0fcf to 446fe99 Compare April 20, 2024 07:45
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you provide a clearer screenshot here, the picture you provided is just a typical JDBC catalog creation page.
it seems unrelated to Doris.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done. Please take a look again.

### Catalog capabilities

- Gravitino catalog corresponds to the Doris instance.
- Supports metadata management of Doris (1.2.x).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does version 1.2.x is the mostly used version currently?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doris 2.x version utilizes a vectorized engine. Xiaomi is still in the testing phase and has not officially used it yet.

Also, there are updates in Doris 2 metadata, such as auto-increment column.

We could consider supporting Doris 2 later

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, let's change it after we have thoroughly tested other versions.


### Table operations

Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#table-operations) for more details.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please xxxx

@zhoukangcn zhoukangcn force-pushed the f-2879 branch 2 times, most recently from 5398346 to 3937e03 Compare April 20, 2024 08:34
@yuqi1129
Copy link
Contributor

@jerryshao
Do you need to take a look?

jerryshao pushed a commit that referenced this pull request Apr 20, 2024
…on more accurate (#3060)

### What changes were proposed in this pull request?

- postpone default schema creation after admin client
- reduced exception nesting

### Why are the changes needed?

Fix: #3015 

### Does this PR introduce _any_ user-facing change?

yes, when the Kafka catalog creation fails, use the correct error
message

### How was this patch tested?

IT added
@jerryshao
Copy link
Contributor

@jerryshao Do you need to take a look?

Yeah, I will.


### Schema properties

- Support schema properties
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What kind of schema properties do we support currently?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docs updated. Support schema properties, includes Doris database properties and user-defined properties.


### Table properties

Doris supports table properties, and you can set them in the table properties.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What kind of table properties do we support?

:::info
- Not all table alteration operations can be processed in batch.
- Schema change, such as add/modify/drop columns can be processed in batch
- Can modify multi column comments at same time
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Support modify multiple column comments at the same time.

- Not all table alteration operations can be processed in batch.
- Schema change, such as add/modify/drop columns can be processed in batch
- Can modify multi column comments at same time
- You can't modify the column type and column comment at the same time
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't support modify...

@zhoukangcn zhoukangcn force-pushed the f-2879 branch 3 times, most recently from 48668f7 to 9e6f28e Compare April 21, 2024 10:26
@zhoukangcn
Copy link
Contributor Author

@jerryshao Doc has been updated. Please help to review again, thanks.


### Catalog properties

You can pass to a Doris data source any property that isn't defined by Gravitino by adding `gravitino.bypass` prefix as a catalog property. For example, catalog property `gravitino.bypass.maxWaitMillis` will pass `maxWaitMillis` to the data source property.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually it should be "gravitino.bypass.", right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me help you polish the doc.


- Gravitino's table concept corresponds to the Doris table.
- Supports index.
- Supports [column default value](./manage-relational-metadata-using-gravitino.md#table-column-default-value) and [auto-increment](./manage-relational-metadata-using-gravitino.md#table-column-auto-increment)..
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mentioned that we don't support auto-increment for now in the below, but here it says it supports auto-increment, so which one is correct? @zhoukangcn @yuqi1129

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, It's a mistake here. I will update doc today

@jerryshao jerryshao merged commit 494c2df into apache:main Apr 22, 2024
9 checks passed
@zhoukangcn zhoukangcn deleted the f-2879 branch April 22, 2024 08:03
diqiu50 pushed a commit to diqiu50/gravitino that referenced this pull request Jun 13, 2024
…xception more accurate (apache#3060)

### What changes were proposed in this pull request?

- postpone default schema creation after admin client
- reduced exception nesting

### Why are the changes needed?

Fix: apache#3015 

### Does this PR introduce _any_ user-facing change?

yes, when the Kafka catalog creation fails, use the correct error
message

### How was this patch tested?

IT added
diqiu50 pushed a commit to diqiu50/gravitino that referenced this pull request Jun 13, 2024
### What changes were proposed in this pull request?

Add doc for Doris catalog

### Why are the changes needed?

Fix: apache#2879

### Does this PR introduce _any_ user-facing change?

N/A

### How was this patch tested?
N/A

---------

Co-authored-by: Jerry Shao <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Subtask] Add document about Doris catalog
4 participants