Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Subtask] [spark-connector]: support CTAS and ITAS command #2402

Closed
Tracked by #1227
FANNG1 opened this issue Feb 29, 2024 · 6 comments · Fixed by #2669
Closed
Tracked by #1227

[Subtask] [spark-connector]: support CTAS and ITAS command #2402

FANNG1 opened this issue Feb 29, 2024 · 6 comments · Fixed by #2669
Assignees
@FANNG1
Labels
subtask Subtasks of umbrella issue
Milestone
Gravitino 0.5.0

Comments

@FANNG1
Copy link
Contributor

FANNG1 commented Feb 29, 2024

Describe the subtask

support CTAS and ITAS command

Parent issue

#1227
@FANNG1 FANNG1 added the subtask Subtasks of umbrella issue label Feb 29, 2024
@FANNG1 FANNG1 added this to the Gravitino 0.5.0 milestone Feb 29, 2024
@FANNG1 FANNG1 self-assigned this Feb 29, 2024
@FANNG1 FANNG1 mentioned this issue Feb 29, 2024
Open
23 tasks
@FANNG1 FANNG1 changed the title [Subtask] spark-connector: support CTAS and ITAS command [Subtask] [spark-connector]: support CTAS and ITAS command Mar 4, 2024
@caican00
Copy link
Collaborator

Hi @FANNG1 May I know what kind of sql does ITAS correspond to?

@FANNG1
Copy link
Contributor Author

FANNG1 commented Mar 14, 2024

Hi @FANNG1 May I know what kind of sql does ITAS correspond to?

Insert table as select xx

@FANNG1
Copy link
Contributor Author

FANNG1 commented Mar 14, 2024

Hi @FANNG1 May I know what kind of sql does ITAS correspond to?

Insert table as select, like

 INSERT INTO students PARTITION (student_id = 444444)
     SELECT name, address FROM persons WHERE name = "Dora Williams";

@caican00
Copy link
Collaborator

Hi @FANNG1 May I know what kind of sql does ITAS correspond to?

Insert table as select, like

 INSERT INTO students PARTITION (student_id = 444444)
     SELECT name, address FROM persons WHERE name = "Dora Williams";

I got it, thank you @FANNG1.
But why is it necessary in spark-connector?

@FANNG1
Copy link
Contributor Author

FANNG1 commented Mar 14, 2024

Normally they are not related to spark-connector, but we should add integrate test in SparkIT to verify it works as expected because the CTAS and ITAS are common on productive env.

@caican00
Copy link
Collaborator

Normally they are not related to spark-connector, but we should add integrate test in SparkIT to verify it works as expected because the CTAS and ITAS are common on productive env.

got it. thank you @FANNG1

@FANNG1 FANNG1 mentioned this issue Mar 25, 2024
Merged
coolderli pushed a commit to coolderli/gravitino that referenced this issue Apr 2, 2024
@FANNG1
[apache#2402] test(spark-connector): Add the integration tests for CT…
d957f27 
…AS and ITAS (apache#2669)

### What changes were proposed in this pull request?
add integrate test for `create table as select xx` and `insert table
select xx`

### Why are the changes needed?
Fix: apache#2402 

### Does this PR introduce _any_ user-facing change?
no

### How was this patch tested?
IT
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@FANNG1 FANNG1

Labels
subtask Subtasks of umbrella issue
Projects
None yet
Milestone
Gravitino 0.5.0
Development

Successfully merging a pull request may close this issue.

[#2402] test(spark-connector): Add the integration tests for CTAS and ITAS FANNG1/gravitino
2 participants
@FANNG1 @caican00