Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] Diff on parquet agg #7494

Open
zml1206 opened this issue Oct 12, 2024 · 7 comments
Open

[VL] Diff on parquet agg #7494

zml1206 opened this issue Oct 12, 2024 · 7 comments
Labels
bug Something isn't working triage

Comments

@zml1206
Copy link
Contributor

zml1206 commented Oct 12, 2024

Backend

VL (Velox)

Bug description

spark.sql("set spark.gluten.enabled=false")
spark.range(100).selectExpr("id%2 as c1", "id%5 as c2", "id as c3").write.mode("overwrite").parquet("tmp/t1")
spark.sql("set spark.gluten.enabled=true")
spark.read.parquet("tmp/t1").createOrReplaceTempView("t1")
spark.sql("select c2, sum(c3)  from t1 where  c1= 1 group by c2").show

result

+---+---------------+
| c2|        sum(c3)|
+---+---------------+
|  0|559882429285360|
|  1|559885503421750|
|  3|839826576815406|
|  2|839827141809990|
|  4|559885785918562|
+---+---------------+

I tested three versions. The velox-08-27 version is normal, but the velox-10-11 and velox-10-04 versions are abnormal.

Spark version

Spark-3.4.x

Spark configurations

No response

System information

No response

Relevant logs

No response

@zml1206 zml1206 added bug Something isn't working triage labels Oct 12, 2024
@zml1206
Copy link
Contributor Author

zml1206 commented Oct 12, 2024

cc @rui-mo @FelixYBW @zhztheplayer
Have you encountered similar problems?

@FelixYBW
Copy link
Contributor

No. Looks like a new Velox bug. Would you debug it?

@zml1206
Copy link
Contributor Author

zml1206 commented Oct 12, 2024

No. Looks like a new Velox bug. Would you debug it?

Sorry, there are problems with local mac compilation of new version, I can’t debug it for the time being.

@zml1206
Copy link
Contributor Author

zml1206 commented Oct 12, 2024

One more thing, I cannot reproduce on mac.

@zml1206
Copy link
Contributor Author

zml1206 commented Oct 13, 2024

Through testing, found that facebookincubator/velox#11010 caused, it worked after reverted it.

@FelixYBW FelixYBW changed the title Diff on parquet agg [VL] Diff on parquet agg Oct 14, 2024
@FelixYBW
Copy link
Contributor

Thank you for update. Did you submit an issue to Velox?

@zml1206
Copy link
Contributor Author

zml1206 commented Oct 15, 2024

Thank you for update. Did you submit an issue to Velox?

facebookincubator/velox#11257

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage
Projects
None yet
Development

No branches or pull requests

2 participants