Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request]: SELECT INTO OUTFILE to work on top of fileservices and able to write to S3/Local stages. #17748

Closed
1 task done
cpegeric opened this issue Jul 26, 2024 · 1 comment

Comments

@cpegeric
Copy link
Contributor

Is there an existing issue for the same feature request?

  • I have checked the existing issues.

Is your feature request related to a problem?

select into outfile only support to write data to local file now.   Next step is to make sure stage works on top of fileservices, and make sure we can write to S3/Local stages.

Describe the feature you'd like

  1. SELECT INTO OUTFILE should write data file via fileservices to both local and S3.
  2. Support stage URL, e.g. stage://stagename/path/tosomewhere as a filepath. stage URL can be local stage or S3 stage.

Describe implementation you've considered

用 io.Pipe 构造一个 reader 一个 writer,然后开一个线程将 reader 写入 fileservice,另一个线程将 writer 包装成 csv writer 或者 json writer,然后把 batch 都写入 writer,最后 close 掉 writer,等 fileservice.Write 返回。

Documentation, Adoption, Use Case, Migration Strategy

No response

Additional information

No response

@Ariznawlll
Copy link
Contributor

test done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants