Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(python,rust,cli): add SQL support for null-aware equality checks #9332

Merged

Conversation

alexander-beedie
Copy link
Collaborator

@alexander-beedie alexander-beedie commented Jun 11, 2023

Now that we have expressions that distinguish between null-aware/unaware comparison, we can integrate the same into our SQL support; this PR covers both the spaceship operator <=>, and is [not] distinct from expressions.

Example

import polars as pl

df = pl.DataFrame({
    "a": [1, None, 3, 6, 5], 
    "b": [1, None, 3, 4, None],
})

with pl.SQLContext( frame_data=df ) as ctx:
    out = ctx.execute(
        """
        SELECT
          -- not null-aware
          (a = b)  as "1_eq_unaware",
          (a != b) as "2_neq_unaware",
          -- null-aware
          (a <=> b) as "3_eq_aware",
          (a IS NOT DISTINCT FROM b) as "4_eq_aware",
          (a IS DISTINCT FROM b) as "5_neq_aware",
        FROM frame_data
        """
    ).collect()

# shape: (5, 5)
# ┌──────────────┬───────────────┬────────────┬────────────┬─────────────┐
# │ 1_eq_unaware ┆ 2_neq_unaware ┆ 3_eq_aware ┆ 4_eq_aware ┆ 5_neq_aware │
# │ ---          ┆ ---           ┆ ---        ┆ ---        ┆ ---         │
# │ bool         ┆ bool          ┆ bool       ┆ bool       ┆ bool        │
# ╞══════════════╪═══════════════╪════════════╪════════════╪═════════════╡
# │ true         ┆ false         ┆ true       ┆ true       ┆ false       │
# │ null         ┆ null          ┆ true       ┆ true       ┆ false       │
# │ true         ┆ false         ┆ true       ┆ true       ┆ false       │
# │ false        ┆ true          ┆ false      ┆ false      ┆ true        │
# │ null         ┆ null          ┆ false      ┆ false      ┆ true        │
# └──────────────┴───────────────┴────────────┴────────────┴─────────────┘

@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars labels Jun 11, 2023
@alexander-beedie alexander-beedie added the A-sql Area: Polars SQL functionality label Jun 11, 2023
… (both the 'spaceship' operator and 'is [not] distinct from' expressions)
@alexander-beedie alexander-beedie force-pushed the sql-null-aware-equality branch from f4d278c to a1814f8 Compare June 11, 2023 06:18
@ritchie46
Copy link
Member

Nice.. learn more of SQL every day.

@alexander-beedie
Copy link
Collaborator Author

The "spaceship" operator has such a great name...
(You can see how I stumbled across the eq_missing doc entry now too ;)

@ritchie46
Copy link
Member

The "spaceship" operator has such a great

Reminds me of rusts "turbofish" ^^

@alexander-beedie alexander-beedie merged commit bf4ae9e into pola-rs:main Jun 11, 2023
@alexander-beedie alexander-beedie deleted the sql-null-aware-equality branch June 11, 2023 07:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-sql Area: Polars SQL functionality enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants