Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: x.isin(<contains NULLs>) result depends on backend #8079

Closed
1 task done
NickCrews opened this issue Jan 23, 2024 · 1 comment · Fixed by #8625
Closed
1 task done

bug: x.isin(<contains NULLs>) result depends on backend #8079

NickCrews opened this issue Jan 23, 2024 · 1 comment · Fixed by #8625
Labels
bug Incorrect behavior inside of ibis polars The polars backend

Comments

@NickCrews
Copy link
Contributor

What happened?

t1 = ibis.memtable({"x": [1, 2]})
e = t1.x.isin([1, None])
print(ibis.duckdb.connect().execute(e))
print(ibis.sqlite.connect().execute(e))
print(ibis.polars.connect().execute(e))
print(ibis.pandas.connect().execute(e))

yields

0 True
1 None
Name: InValues(x), dtype: object
0 True
1 None
Name: InValues(x), dtype: object
0 True
1 False
Name: InValues(x), dtype: bool
0 True
1 None
Name: InValues(x), dtype: object

If a needle isn't present in the haystack, and the haystack contains NULL, then
for some backends this returns False, and for others NULL.

I think it is pretty standard SQL to return null. But this isn't what I expected to happen, I actually expected
pandas semantics. IDK which we want, but they should be the same, and this should be documented.

What version of ibis are you using?

main

What backend(s) are you using, if any?

varies

Relevant log output

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
@NickCrews NickCrews added the bug Incorrect behavior inside of ibis label Jan 23, 2024
@jcrist jcrist added the polars The polars backend label Jan 25, 2024
@cpcloud
Copy link
Member

cpcloud commented Mar 12, 2024

It looks like most (all?) backends return NULL and polars might be the odd one out here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Incorrect behavior inside of ibis polars The polars backend
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

3 participants