Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] [Delta Lake] Add support for new metrics in MERGE #8556

Closed
andygrove opened this issue Jun 12, 2023 · 2 comments · Fixed by #8573
Closed

[FEA] [Delta Lake] Add support for new metrics in MERGE #8556

andygrove opened this issue Jun 12, 2023 · 2 comments · Fixed by #8573
Assignees
Labels
feature request New feature or request

Comments

@andygrove
Copy link
Contributor

andygrove commented Jun 12, 2023

Is your feature request related to a problem? Please describe.

Delta Lake 2.4 adds some new metrics for MERGE operations:

'numTargetRowsMatchedUpdated': '0', 
'numTargetRowsMatchedDeleted': '0',
'numTargetRowsNotMatchedBySourceUpdated': '0',
'numTargetRowsNotMatchedBySourceDeleted': '0'

Describe the solution you'd like
We should ensure that we have tests that generate valid metrics on CPU and then determine what we need to do on GPU to match.

Describe alternatives you've considered

Additional context

@andygrove andygrove added feature request New feature or request ? - Needs Triage Need team to review and classify labels Jun 12, 2023
@andygrove
Copy link
Contributor Author

Note that we currently fall back to CPU for queries involving NOT MATCHED BY SOURCE. Tracking issue is #8415

@andygrove
Copy link
Contributor Author

This is related to #8415 but I don't think it is a duplicate since we should produce zero metrics in the case where we do run a MERGE on GPU and there is no NOT MATCHED BY SOURCE clause.

@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label Jun 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants