-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[fix](csv reader) fix csv parser incorrect if enclosing line_delimiter (#38347) #38446
Conversation
#38347) Csv reader parse data incorrect when data enclosing line_delimiter, for example, line_delimiter is \n and enclose is ', data as follows: ``` 'aaaaaaaaaaaa bbbb' ``` it will be parsed as two columns: `'aaaaaaaaaaaa` and `bbbb',` rather than one column ``` 'aaaaaaaaaaaa bbbb' ``` The reason why this happened is csv reader will not reset result when not match enclose in this `output_buf_read`, causing incorrect truncation was made. Co-authored-by: Xin Liao <[email protected]>
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
run buildall |
clang-tidy review says "All clean, LGTM! 👍" |
TPC-H: Total hot run time: 49520 ms
|
TeamCity be ut coverage result: |
TPC-DS: Total hot run time: 202838 ms
|
ClickBench: Total hot run time: 30.84 s
|
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
|
pick #38347
Csv reader parse data incorrect when data enclosing line_delimiter, for example, line_delimiter is \n and enclose is ', data as follows:
it will be parsed as two columns:
'aaaaaaaaaaaa
andbbbb',
rather than one columnThe reason why this happened is csv reader will not reset result when not match enclose in this
output_buf_read
, causing incorrect truncation was made.