Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(sql): remove constants in order_by calls during select merging #10475

Merged
merged 1 commit into from
Dec 4, 2024
Merged

fix(sql): remove constants in order_by calls during select merging #10475

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion ibis/backends/sql/rewrites.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -320,7 +320,9 @@ def merge_select_select(_, **kwargs):
selections=selections,
predicates=unique_predicates,
qualified=unique_qualified,
sort_keys=unique_sort_keys,
sort_keys=tuple(
key for key in unique_sort_keys if not isinstance(key.expr, ops.Literal)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove all expressions which are constant across the relation? I am thinking of all pure scalar nodes.

Copy link
Member Author

@cpcloud cpcloud Nov 14, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean? We can't just start deleting anything that's a literal scalar. Expressions like

SELECT 1 as a
FROM t

are 100% valid and useful.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only mean the order by expressions.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No because it's possible to write things like 1 if x else 0, and use them in sort keys.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expr = 1 if x else 0 right, though we can identify that as well, like its .relations attribute is not empty

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In any case, it doesn't block this PR.

),
distinct=distinct,
)
return result if complexity(result) <= complexity(_) else _
Expand Down
7 changes: 7 additions & 0 deletions ibis/backends/tests/snapshots/test_sql/test_order_by_no_deference_literals/bigquery/out.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SELECT
`t0`.`a`,
9 AS `i`,
'foo' AS `s`
FROM `test` AS `t0`
ORDER BY
`t0`.`a` ASC NULLS LAST
7 changes: 7 additions & 0 deletions .../backends/tests/snapshots/test_sql/test_order_by_no_deference_literals/clickhouse/out.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SELECT
"t0"."a" AS "a",
9 AS "i",
'foo' AS "s"
FROM "test" AS "t0"
ORDER BY
"t0"."a" ASC
7 changes: 7 additions & 0 deletions .../backends/tests/snapshots/test_sql/test_order_by_no_deference_literals/databricks/out.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SELECT
`t0`.`a`,
9 AS `i`,
'foo' AS `s`
FROM `test` AS `t0`
ORDER BY
`t0`.`a` ASC NULLS LAST
7 changes: 7 additions & 0 deletions .../backends/tests/snapshots/test_sql/test_order_by_no_deference_literals/datafusion/out.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SELECT
"t0"."a",
9 AS "i",
'foo' AS "s"
FROM "test" AS "t0"
ORDER BY
"t0"."a" ASC
7 changes: 7 additions & 0 deletions ibis/backends/tests/snapshots/test_sql/test_order_by_no_deference_literals/druid/out.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SELECT
"t0"."a",
9 AS "i",
'foo' AS "s"
FROM "test" AS "t0"
ORDER BY
"t0"."a" ASC
7 changes: 7 additions & 0 deletions ibis/backends/tests/snapshots/test_sql/test_order_by_no_deference_literals/duckdb/out.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SELECT
"t0"."a",
9 AS "i",
'foo' AS "s"
FROM "test" AS "t0"
ORDER BY
"t0"."a" ASC
7 changes: 7 additions & 0 deletions ibis/backends/tests/snapshots/test_sql/test_order_by_no_deference_literals/exasol/out.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SELECT
"t0"."a",
9 AS "i",
'foo' AS "s"
FROM "test" AS "t0"
ORDER BY
"t0"."a" ASC
7 changes: 7 additions & 0 deletions ibis/backends/tests/snapshots/test_sql/test_order_by_no_deference_literals/flink/out.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SELECT
`t0`.`a`,
9 AS `i`,
'foo' AS `s`
FROM `test` AS `t0`
ORDER BY
`t0`.`a` ASC NULLS LAST
7 changes: 7 additions & 0 deletions ibis/backends/tests/snapshots/test_sql/test_order_by_no_deference_literals/impala/out.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SELECT
`t0`.`a`,
9 AS `i`,
'foo' AS `s`
FROM `test` AS `t0`
ORDER BY
`t0`.`a` ASC
7 changes: 7 additions & 0 deletions ibis/backends/tests/snapshots/test_sql/test_order_by_no_deference_literals/mssql/out.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SELECT
[t0].[a],
9 AS [i],
'foo' AS [s]
FROM [test] AS [t0]
ORDER BY
CASE WHEN [t0].[a] IS NULL THEN 1 ELSE 0 END, [t0].[a] ASC
7 changes: 7 additions & 0 deletions ibis/backends/tests/snapshots/test_sql/test_order_by_no_deference_literals/mysql/out.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SELECT
`t0`.`a`,
9 AS `i`,
'foo' AS `s`
FROM `test` AS `t0`
ORDER BY
CASE WHEN `t0`.`a` IS NULL THEN 1 ELSE 0 END, `t0`.`a` ASC
7 changes: 7 additions & 0 deletions ibis/backends/tests/snapshots/test_sql/test_order_by_no_deference_literals/oracle/out.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SELECT
"t0"."a",
9 AS "i",
'foo' AS "s"
FROM "test" "t0"
ORDER BY
"t0"."a" ASC
7 changes: 7 additions & 0 deletions ibis/backends/tests/snapshots/test_sql/test_order_by_no_deference_literals/postgres/out.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SELECT
"t0"."a",
9 AS "i",
'foo' AS "s"
FROM "test" AS "t0"
ORDER BY
"t0"."a" ASC
7 changes: 7 additions & 0 deletions ibis/backends/tests/snapshots/test_sql/test_order_by_no_deference_literals/pyspark/out.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SELECT
`t0`.`a`,
9 AS `i`,
'foo' AS `s`
FROM `test` AS `t0`
ORDER BY
`t0`.`a` ASC NULLS LAST
7 changes: 7 additions & 0 deletions .../backends/tests/snapshots/test_sql/test_order_by_no_deference_literals/risingwave/out.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SELECT
"t0"."a",
9 AS "i",
'foo' AS "s"
FROM "test" AS "t0"
ORDER BY
"t0"."a" ASC
7 changes: 7 additions & 0 deletions ibis/backends/tests/snapshots/test_sql/test_order_by_no_deference_literals/snowflake/out.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SELECT
"t0"."a",
9 AS "i",
'foo' AS "s"
FROM "test" AS "t0"
ORDER BY
"t0"."a" ASC
7 changes: 7 additions & 0 deletions ibis/backends/tests/snapshots/test_sql/test_order_by_no_deference_literals/sqlite/out.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SELECT
"t0"."a",
9 AS "i",
'foo' AS "s"
FROM "test" AS "t0"
ORDER BY
"t0"."a" ASC NULLS LAST
7 changes: 7 additions & 0 deletions ibis/backends/tests/snapshots/test_sql/test_order_by_no_deference_literals/trino/out.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SELECT
"t0"."a",
9 AS "i",
'foo' AS "s"
FROM "test" AS "t0"
ORDER BY
"t0"."a" ASC
10 changes: 10 additions & 0 deletions ibis/backends/tests/test_sql.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -259,3 +259,13 @@ def test_sample(backend_name, snapshot, subquery):
row = ibis.to_sql(t.sample(0.5, method="row"), dialect=backend_name)
snapshot.assert_match(block, "block.sql")
snapshot.assert_match(row, "row.sql")


@pytest.mark.parametrize("backend_name", _get_backends_to_test())
@pytest.mark.notimpl(["polars"], raises=ValueError, reason="not a SQL backend")
def test_order_by_no_deference_literals(backend_name, snapshot):
t = ibis.table({"a": "int"}, name="test")
s = t.select("a", i=ibis.literal(9), s=ibis.literal("foo"))
o = s.order_by("a", "i", "s")
sql = ibis.to_sql(o, dialect=backend_name)
snapshot.assert_match(sql, "out.sql")