-
Notifications
You must be signed in to change notification settings - Fork 211
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Intermittently fails for large csv file #102
Comments
I'm having the same issue on an 8k line csv. |
Not sure about whether latest version of fast-csv library has this issue. In our case we ended up using another csv parsing library at that time |
it still has it apparently, I just tried it and it just never returned... went to a different library as well and it worked like a charm. |
We are seeing the same errors happening randomly on Google Compute Engine (Linux) for files with around 30MB. @weagle08 What library did you end up using to resolve that issue? |
If it helps to resolve that issue we are seeing these errors on lines such as
In that case, the [
"123456",
"123456",
"2018-02-01T16:05:16Z",
"7",
"80428",
"65756",
"Unquoted_String_With_Underscores_Hypens"
] |
I just retested this with the latest version However, I did add a test to help catch this in the future. The test I added parses a 100K line file with the following line.
|
Hi,
I have a csv file with 15000 entries (1.3mb) . Csv parsing fails randomly on AWS amazon linux box, but then it passes sometimes cleanly parsing all the rows correctly.There is no issue of memory, the machine instance has 4Gb memory. The error received is:
Sometimes the parser cannot read the complete line and so it fails for matching ending quote in
parser.js:76 parsedEscapedItem
function call.The strange thing is the issue happens intermittently so it can actually read correctly few times.
the only option i pass on to fromStream method are
{ignoreEmpty:true}
rest are all defaults.Any pointers would be helpful.
thanks.
The text was updated successfully, but these errors were encountered: