Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: only do row identity based joins when joining by index #356

Merged
merged 6 commits into from
Mar 7, 2024

Conversation

DLT1412
Copy link
Contributor

@DLT1412 DLT1412 commented Jan 29, 2024

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #355 🦕

@DLT1412 DLT1412 requested review from a team as code owners January 29, 2024 17:45
Copy link

conventional-commit-lint-gcf bot commented Jan 29, 2024

🤖 I detect that the PR title and the commit message differ and there's only one commit. To use the PR title for the commit history, you can use Github's automerge feature with squashing, or use automerge label. Good luck human!

-- conventional-commit-lint bot
https://conventionalcommits.org/

@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. labels Jan 29, 2024
@DLT1412
Copy link
Contributor Author

DLT1412 commented Feb 1, 2024

@TrevorBergeron can you take a look?

@@ -64,6 +64,8 @@ def join_by_column_ordered(
.equals(right._get_ibis_column(rcol).name("index"))
for lcol, rcol in join.conditions
)
and not left._predicates
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would likely be a regression with regards to our SQL size. We want to simulate a join in this case. I'll take a look at join_by_row_identity_ordered and join_by_row_identity_unordered to see if there has been a regression there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any update on this?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @DLT1412 Sorry for the wait. I was OOO for a couple weeks to get married. I hope to look at this more deeply later this week. Thanks again for raising this issue.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm able to reproduce this issue locally with inner, outer, and left join. Will look more tomorrow.

@tswast tswast changed the title fix: merge df same source after filter fix: only do row identity based joins when joining by index Mar 7, 2024
@tswast tswast added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Mar 7, 2024
@bigframes-bot bigframes-bot removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Mar 7, 2024
@tswast tswast added the owlbot:run Add this label to trigger the Owlbot post processor. label Mar 7, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Mar 7, 2024
@tswast tswast added the automerge Merge the pull request once unit tests and other checks pass. label Mar 7, 2024
@gcf-merge-on-green gcf-merge-on-green bot merged commit 76b252f into googleapis:main Mar 7, 2024
14 of 15 checks passed
@gcf-merge-on-green gcf-merge-on-green bot removed the automerge Merge the pull request once unit tests and other checks pass. label Mar 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Merge df with same source after filter return wrong result
4 participants