Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Python] MapType comparison ignores nullable flag of item_field #33574

Closed
asfimport opened this issue Nov 27, 2022 · 1 comment
Closed

[Python] MapType comparison ignores nullable flag of item_field #33574

asfimport opened this issue Nov 27, 2022 · 1 comment

Comments

@asfimport
Copy link
Collaborator

asfimport commented Nov 27, 2022

By default MapType value fields are nullable:

 pa.map_(pa.string(), pa.int32()).item_field.nullable == True 

It is possible to mark the value field of a MapType as not-nullable:

 pa.map_(pa.string(), pa.field("value", pa.int32(), nullable=False)).item_field.nullable == False

But comparing these two types, that are semantically different, returns True:

pa.map_(pa.string(), pa.int32()) == pa.map_(pa.string(), pa.field("value", pa.int32(), nullable=False)) # Returns True 

So it looks like the comparison omits the nullable flag. 

import pyarrow as pa

map_type = pa.map_(pa.string(), pa.int32())
non_null_map_type = pa.map_(pa.string(), pa.field("value", pa.int32(), nullable=False))
nullable_map_type = pa.map_(pa.string(), pa.field("value", pa.int32(), nullable=True))
map_type_different_field_name = pa.map_(pa.string(), pa.field("value", pa.int32(), nullable=True))

assert nullable_map_type == map_type  # Wrong
assert str(nullable_map_type) == str(map_type)
assert str(non_null_map_type) == str(map_type) # Wrong
assert non_null_map_type == map_type
assert non_null_map_type.item_type == map_type.item_type
assert non_null_map_type.item_field != map_type.item_field
assert non_null_map_type.item_field.nullable != map_type.item_field.nullable
assert non_null_map_type.item_field.name == map_type.item_field.name
assert map_type == map_type_different_field_name # This makes sense

Environment: pyarrow==10.0.1

Reporter: &res / @0x26res
Assignee: Will Jones / @wjones127

Related issues:

Note: This issue was originally created as ARROW-18411. Please see the migration documentation for further details.

@asfimport
Copy link
Collaborator Author

Will Jones / @wjones127:
Thanks for reporting this. This will be fixed by #13851

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants