Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix a bounds-checking bug in CSV parsing #1914

Merged
merged 2 commits into from
Aug 2, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions RELEASES.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
([#1906](https://github.com/enso-org/enso/pull/1906)).
- Added documentation for the new searcher categories
([#1910](https://github.com/enso-org/enso/pull/1910)).
- Fixed a bug where CSV files with very long lines could not be parsed
([#1914](https://github.com/enso-org/enso/pull/1914)).

# Enso 0.2.17 (2021-07-28)

Expand Down
9 changes: 7 additions & 2 deletions distribution/lib/Standard/Table/0.1.0/src/Io/Csv.enso
Original file line number Diff line number Diff line change
Expand Up @@ -64,14 +64,19 @@ from_csv : File.File | Text -> Boolean -> Text -> Table ! Parse_Error
from_csv csv has_header=True prefix='C' =
parser_inst = Parser.create has_header prefix

handle_error error = case error of
Polyglot_Error err -> Error.throw (Parse_Error err.getMessage)
_ -> Panic.throw error

case csv of
Text ->
input_stream = ByteArrayInputStream.new csv.utf_8.to_array
Table.Table (parser_inst.parse input_stream)
Panic.recover Table.Table (parser_inst.parse input_stream) . catch handle_error
File.File _ ->
csv.with_input_stream [File.Option.Read] stream->
maybe_err = Panic.recover <| csv.with_input_stream [File.Option.Read] stream->
stream.with_java_stream java_stream->
Table.Table (parser_inst.parse java_stream)
maybe_err.catch handle_error
_ ->
found_type_name = Meta.get_qualified_type_name csv
file_name = Meta.get_qualified_type_name File.File
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ public Table parse(InputStream inputStream) {
CsvParserSettings settings = new CsvParserSettings();
settings.setHeaderExtractionEnabled(hasHeader);
settings.detectFormatAutomatically();
settings.setMaxCharsPerColumn(-1);
CsvParser parser = new CsvParser(settings);
parser.beginParsing(inputStream);
StorageBuilder[] builders = null;
Expand Down