Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(shannon): Reading and writing open table format #31

Open
ShannonBase opened this issue Oct 30, 2023 · 2 comments
Open

feat(shannon): Reading and writing open table format #31

ShannonBase opened this issue Oct 30, 2023 · 2 comments
Assignees
Labels
feature it will be implemented as a new feature

Comments

@ShannonBase
Copy link
Contributor

ShannonBase commented Oct 30, 2023

To support reading and writing the open table format, such as parquets. which will enable Shannonbase lakehouse processing abilities. Nowdays, volume of data is getting larger, and the type of data is going to be more versatile, such as structured data, semi-structured data, unstructured data.

MySQL Heatwave, Databricks, etc. are all support data lakehouse.

@ShannonBase ShannonBase self-assigned this Oct 31, 2023
@RingsC
Copy link
Contributor

RingsC commented Dec 22, 2023

Using AFSL(Abstract File System Layer) or specific engine to operate on specific engine?

for example:
create table test (xxx) engine = innodb
ENGINE_ATTRIBUTE='{"file":
[{"region":"",
"namespace":"",
"bucket":"<bucket_name>",
"name":"<orders_file_location>"}]}';

@ShannonBase ShannonBase added the feature it will be implemented as a new feature label Jan 5, 2024
@ShannonBase
Copy link
Contributor Author

using create tablespace. When we ar creating a table space, we can specify the path of space file path. then using this tablespace to create a table.

by this, we can locate shannon rapid table to a NVME disk.

ref: https://dev.mysql.com/doc/refman/8.0/en/create-tablespace.html

@ShannonBase ShannonBase changed the title feat(shannon): Reading and writing open file format feat(shannon): Reading and writing open table format Aug 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature it will be implemented as a new feature
Projects
None yet
Development

No branches or pull requests

2 participants