Skip to content

Commit

Permalink
Fix a bounds-checking bug in CSV parsing (#1914)
Browse files Browse the repository at this point in the history
  • Loading branch information
iamrecursion committed Aug 2, 2021
1 parent e6735d0 commit 20e7f35
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 2 deletions.
2 changes: 2 additions & 0 deletions RELEASES.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
([#1906](https://github.com/enso-org/enso/pull/1906)).
- Added documentation for the new searcher categories
([#1910](https://github.com/enso-org/enso/pull/1910)).
- Fixed a bug where CSV files with very long lines could not be parsed
([#1914](https://github.com/enso-org/enso/pull/1914)).

# Enso 0.2.17 (2021-07-28)

Expand Down
9 changes: 7 additions & 2 deletions distribution/lib/Standard/Table/0.1.0/src/Io/Csv.enso
Original file line number Diff line number Diff line change
Expand Up @@ -64,14 +64,19 @@ from_csv : File.File | Text -> Boolean -> Text -> Table ! Parse_Error
from_csv csv has_header=True prefix='C' =
parser_inst = Parser.create has_header prefix

handle_error error = case error of
Polyglot_Error err -> Error.throw (Parse_Error err.getMessage)
_ -> Panic.throw error

case csv of
Text ->
input_stream = ByteArrayInputStream.new csv.utf_8.to_array
Table.Table (parser_inst.parse input_stream)
Panic.recover Table.Table (parser_inst.parse input_stream) . catch handle_error
File.File _ ->
csv.with_input_stream [File.Option.Read] stream->
maybe_err = Panic.recover <| csv.with_input_stream [File.Option.Read] stream->
stream.with_java_stream java_stream->
Table.Table (parser_inst.parse java_stream)
maybe_err.catch handle_error
_ ->
found_type_name = Meta.get_qualified_type_name csv
file_name = Meta.get_qualified_type_name File.File
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ public Table parse(InputStream inputStream) {
CsvParserSettings settings = new CsvParserSettings();
settings.setHeaderExtractionEnabled(hasHeader);
settings.detectFormatAutomatically();
settings.setMaxCharsPerColumn(-1);
CsvParser parser = new CsvParser(settings);
parser.beginParsing(inputStream);
StorageBuilder[] builders = null;
Expand Down

0 comments on commit 20e7f35

Please sign in to comment.