Skip to content

Commit

Permalink
rename Lossy_Conversion to Conversion_Failure
Browse files Browse the repository at this point in the history
  • Loading branch information
radeusgd committed May 18, 2023
1 parent 34fdd74 commit df400f4
Show file tree
Hide file tree
Showing 7 changed files with 41 additions and 62 deletions.
18 changes: 6 additions & 12 deletions distribution/lib/Standard/Database/0.0.0-dev/src/Data/Column.enso
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import Standard.Table.Internal.Java_Problems
import Standard.Table.Internal.Problem_Builder.Problem_Builder
import Standard.Table.Internal.Widget_Helpers
from Standard.Table import Sort_Column, Data_Formatter, Value_Type, Auto
from Standard.Table.Errors import Floating_Point_Equality, Inexact_Type_Coercion, Invalid_Value_Type, Lossy_Conversion
from Standard.Table.Errors import Floating_Point_Equality, Inexact_Type_Coercion, Invalid_Value_Type, Conversion_Failure
from Standard.Table.Internal.Cast_Helpers import check_cast_compatibility

import project.Connection.Connection.Connection
Expand Down Expand Up @@ -1031,23 +1031,17 @@ type Column
- If a `Date` is to be converted to `Date_Time`, it will be set at
midnight of the default system timezone.

? Conversion Precision

In the in-memory backend, if the conversion is lossy, a
`Lossy_Conversion` warning will be reported. The only exception is when
truncating a column which is already a text column - as then the
truncation seems like an intended behaviour, so it is not reported. If
truncating needs to occur when converting a non-text column, a warning
will still be reported.

Currently, the warning is not reported for Database backends.
If the target type cannot fit some of the values (for example due to too
small range), a `Conversion_Failure` may be reported according to the
`on_problems` rules. The Database backends may fail with `SQL_Error`
instead.

? Inexact Target Type

If the backend does not support the requested target type, the closest
supported type is chosen and a `Inexact_Type_Coercion` problem is
reported.
cast : Value_Type -> Problem_Behavior -> Column ! Illegal_Argument | Inexact_Type_Coercion | Lossy_Conversion
cast : Value_Type -> Problem_Behavior -> Column ! Illegal_Argument | Inexact_Type_Coercion | Conversion_Failure
cast self value_type on_problems=Problem_Behavior.Report_Warning =
check_cast_compatibility self.value_type value_type <|
dialect = self.connection.dialect
Expand Down
16 changes: 5 additions & 11 deletions distribution/lib/Standard/Database/0.0.0-dev/src/Data/Table.enso
Original file line number Diff line number Diff line change
Expand Up @@ -1558,24 +1558,18 @@ type Table
- If a `Date` is to be converted to `Date_Time`, it will be set at
midnight of the default system timezone.

? Conversion Precision

In the in-memory backend, if the conversion is lossy, a
`Lossy_Conversion` warning will be reported. The only exception is when
truncating a column which is already a text column - as then the
truncation seems like an intended behaviour, so it is not reported. If
truncating needs to occur when converting a non-text column, a warning
will still be reported.

Currently, the warning is not reported for Database backends.
If the target type cannot fit some of the values (for example due to too
small range), a `Conversion_Failure` may be reported according to the
`on_problems` rules. The Database backends may fail with `SQL_Error`
instead.

? Inexact Target Type

If the backend does not support the requested target type, the closest
supported type is chosen and a `Inexact_Type_Coercion` problem is
reported.
@columns Widget_Helpers.make_column_name_vector_selector
cast : (Text | Integer | Column_Selector | Vector (Integer | Text | Column_Selector)) -> Value_Type -> Boolean -> Problem_Behavior -> Table ! Illegal_Argument | Inexact_Type_Coercion | Lossy_Conversion
cast : (Text | Integer | Column_Selector | Vector (Integer | Text | Column_Selector)) -> Value_Type -> Boolean -> Problem_Behavior -> Table ! Illegal_Argument | Inexact_Type_Coercion | Conversion_Failure
cast self columns=[0] value_type error_on_missing_columns=True on_problems=Problem_Behavior.Report_Warning =
selected = self.columns_helper.resolve_columns columns error_on_missing_columns=error_on_missing_columns on_problems=on_problems
selected.fold self table-> column_to_cast->
Expand Down
18 changes: 6 additions & 12 deletions distribution/lib/Standard/Table/0.0.0-dev/src/Data/Column.enso
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ import project.Data.Type.Value_Type_Helpers

from project.Data.Table import print_table
from project.Data.Type.Value_Type import Value_Type, Auto
from project.Errors import No_Index_Set_Error, Floating_Point_Equality, Invalid_Value_Type, Inexact_Type_Coercion, Lossy_Conversion
from project.Errors import No_Index_Set_Error, Floating_Point_Equality, Invalid_Value_Type, Inexact_Type_Coercion, Conversion_Failure
from project.Internal.Java_Exports import make_string_builder

polyglot java import org.enso.table.data.column.operation.map.MapOperationProblemBuilder
Expand Down Expand Up @@ -1308,23 +1308,17 @@ type Column
- If a `Date` is to be converted to `Date_Time`, it will be set at
midnight of the default system timezone.

? Conversion Precision

In the in-memory backend, if the conversion is lossy, a
`Lossy_Conversion` warning will be reported. The only exception is when
truncating a column which is already a text column - as then the
truncation seems like an intended behaviour, so it is not reported. If
truncating needs to occur when converting a non-text column, a warning
will still be reported.

Currently, the warning is not reported for Database backends.
If the target type cannot fit some of the values (for example due to too
small range), a `Conversion_Failure` may be reported according to the
`on_problems` rules. The Database backends may fail with `SQL_Error`
instead.

? Inexact Target Type

If the backend does not support the requested target type, the closest
supported type is chosen and a `Inexact_Type_Coercion` problem is
reported.
cast : Value_Type -> Problem_Behavior -> Column ! Illegal_Argument | Inexact_Type_Coercion | Lossy_Conversion
cast : Value_Type -> Problem_Behavior -> Column ! Illegal_Argument | Inexact_Type_Coercion | Conversion_Failure
cast self value_type on_problems=Problem_Behavior.Report_Warning =
Cast_Helpers.check_cast_compatibility self.value_type value_type <|
target_storage_type = Storage.from_value_type value_type on_problems
Expand Down
16 changes: 5 additions & 11 deletions distribution/lib/Standard/Table/0.0.0-dev/src/Data/Table.enso
Original file line number Diff line number Diff line change
Expand Up @@ -905,24 +905,18 @@ type Table
- If a `Date` is to be converted to `Date_Time`, it will be set at
midnight of the default system timezone.

? Conversion Precision

In the in-memory backend, if the conversion is lossy, a
`Lossy_Conversion` warning will be reported. The only exception is when
truncating a column which is already a text column - as then the
truncation seems like an intended behaviour, so it is not reported. If
truncating needs to occur when converting a non-text column, a warning
will still be reported.

Currently, the warning is not reported for Database backends.
If the target type cannot fit some of the values (for example due to too
small range), a `Conversion_Failure` may be reported according to the
`on_problems` rules. The Database backends may fail with `SQL_Error`
instead.

? Inexact Target Type

If the backend does not support the requested target type, the closest
supported type is chosen and a `Inexact_Type_Coercion` problem is
reported.
@columns Widget_Helpers.make_column_name_vector_selector
cast : (Text | Integer | Column_Selector | Vector (Integer | Text | Column_Selector)) -> Value_Type -> Boolean -> Problem_Behavior -> Table ! Illegal_Argument | Inexact_Type_Coercion | Lossy_Conversion
cast : (Text | Integer | Column_Selector | Vector (Integer | Text | Column_Selector)) -> Value_Type -> Boolean -> Problem_Behavior -> Table ! Illegal_Argument | Inexact_Type_Coercion | Conversion_Failure
cast self columns=[0] value_type error_on_missing_columns=True on_problems=Problem_Behavior.Report_Warning =
selected = self.columns_helper.resolve_columns columns error_on_missing_columns=error_on_missing_columns on_problems=on_problems
selected.fold self table-> column_to_cast->
Expand Down
15 changes: 9 additions & 6 deletions distribution/lib/Standard/Table/0.0.0-dev/src/Errors.enso
Original file line number Diff line number Diff line change
Expand Up @@ -552,9 +552,12 @@ type Inexact_Type_Coercion
to_text self =
"Inexact_Type_Coercion.Warning (requested_type = " + self.requested_type.to_text + ") (actual_type = " + self.actual_type.to_text + ")"

type Lossy_Conversion
## Indicates that some likely not-insignificant information was lost during
a conversion.
type Conversion_Failure
## Indicates that some values from the column could not be converted to the
desired type.

This may occur for example when a number does not fit the range of the
target type.
Error (target_type : Value_Type) (related_column : Text) (affected_rows_count : Nothing|Integer)

## PRIVATE
Expand All @@ -563,9 +566,9 @@ type Lossy_Conversion
to_display_text : Text
to_display_text self =
rows_info = case self.affected_rows_count of
Nothing -> ""
count -> " in "+count.to_text+" rows"
"When converting the column ["+self.related_column+"], the target type ("+self.target_type.to_display_text+") does not fill all the data, so some information was lost"+rows_info+"."
Nothing -> "Some values"
count -> count.to_text+" rows"
rows_info + "could not be converted into the target type "+self.target_type.to_display_text+" when converting the column ["+self.related_column+"]."

type Invalid_Value_For_Type
## PRIVATE
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import Standard.Base.Errors.Illegal_Argument.Illegal_Argument

import project.Data.Type.Value_Type.Value_Type
import project.Internal.Parse_Values_Helper
from project.Errors import Lossy_Conversion
from project.Errors import Conversion_Failure

polyglot java import org.enso.table.data.column.operation.CastProblemBuilder

Expand Down Expand Up @@ -48,7 +48,7 @@ type Cast_Problem_Builder

lossy_conversion_rows = java_instance.getLossyConversionRowCount
if lossy_conversion_rows > 0 then
builder.append (Lossy_Conversion.Error self.target_type self.column_name lossy_conversion_rows)
builder.append (Conversion_Failure.Error self.target_type self.column_name lossy_conversion_rows)

builder.to_vector

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import Standard.Base.Errors.Illegal_Argument.Illegal_Argument
from Standard.Table import Value_Type
import Standard.Table.Data.Type.Value_Type.Bits

from Standard.Table.Errors import Lossy_Conversion
from Standard.Table.Errors import Conversion_Failure
from Standard.Database.Errors import Unsupported_Database_Operation

from Standard.Test import Test, Problems
Expand Down Expand Up @@ -72,7 +72,7 @@ spec setup =
c.value_type . should_equal (Value_Type.Char size=3 variable_length=False)
c.to_vector . should_equal ["a ", "DEF", "a s"]

# No Lossy_Conversion warning here, because we started with text, so it was expected we will trim it if needed.
# No Conversion_Failure warning here, because we started with text, so it was expected we will trim it if needed.
Problems.assume_no_problems c

Test.specify "should allow casting a non-text column to fixed-length text" <|
Expand All @@ -82,12 +82,12 @@ spec setup =
c.to_vector . should_equal ["1 ", "22 ", "333"]
Problems.assume_no_problems c

Test.specify "should warn when losing data if the fixed-length text length is too short to fit the data" pending=(if setup.is_database then "Lossy_Conversion is not supported in Database yet.") <|
Test.specify "should warn when losing data if the fixed-length text length is too short to fit the data" pending=(if setup.is_database then "Conversion_Failure is not supported in Database yet.") <|
t = table_builder [["X", [15, 1000000, 123456]]]
c = t.at "X" . cast (Value_Type.Char size=3 variable_length=False)
c.value_type . should_equal (Value_Type.Char size=3 variable_length=False)
c.to_vector . should_equal ["15 ", "100", "123"]
Problems.expect_warning Lossy_Conversion c
Problems.expect_warning Conversion_Failure c

Test.group prefix+"Table/Column.cast - numeric" <|
Test.specify "should allow to cast a boolean column to integer" <|
Expand Down Expand Up @@ -115,10 +115,10 @@ spec setup =
c2.value_type . should_equal (Value_Type.Integer Bits.Bits_16)
c2.to_vector . should_equal [1, 2, Nothing]
# This can likely only be checked on in-memory.
Problems.expect_warning Lossy_Conversion c2
Problems.expect_warning Conversion_Failure c2

r3 = t2.at "X" . cast (Value_Type.Integer Bits.Bits_16) on_problems=Problem_Behavior.Report_Error
r3.should_fail_with Lossy_Conversion
r3.should_fail_with Conversion_Failure

# Now converting the 16-bit column `c` into 32 bits.
c3 = c.cast (Value_Type.Integer Bits.Bits_32)
Expand All @@ -142,7 +142,7 @@ spec setup =
[[1, 4], [1, 5]] . should_contain v2

if setup.is_database.not then
Test.specify "should report Lossy_Conversion if converting a huge float to an integer overflows it" <|
Test.specify "should report Conversion_Failure if converting a huge float to an integer overflows it" <|
max_long = Java_Long.MAX_VALUE
too_big_double = (max_long + 1.0) * 1000.0
(too_big_double > max_long) . should_be_true
Expand All @@ -159,7 +159,7 @@ spec setup =
c = t.at "X" . cast Value_Type.Integer
c.value_type . should_equal Value_Type.Integer
c.to_vector . should_equal [1, 2, max_long, Nothing, min_long, Nothing, 4]
warning = Problems.expect_warning Lossy_Conversion c
warning = Problems.expect_warning Conversion_Failure c
warning.to_display_text . should_contain "information was lost in 2 rows"

if supports_dates then
Expand Down

0 comments on commit df400f4

Please sign in to comment.