Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: read_pandas inline respects location #412

Merged
merged 14 commits into from
Mar 12, 2024
Merged

Conversation

GarrettWu
Copy link
Contributor

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes b/327544164 🦕

@GarrettWu GarrettWu requested review from a team as code owners March 5, 2024 22:26
@product-auto-label product-auto-label bot added size: s Pull request size is small. api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. labels Mar 5, 2024
@GarrettWu GarrettWu marked this pull request as draft March 6, 2024 19:45
@GarrettWu GarrettWu marked this pull request as ready for review March 7, 2024 01:10
options = bigframes.BigQueryOptions(location="europe-west1")
session = bigframes.Session(options)

df = session.read_pandas(pd.DataFrame([[1, 2, 3], [4, 5, 6]]))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add a test for inline data too, and verify that it creates the result tables in the intended location? bpd.Dataframe([[1, 2, 3], [4, 5, 6]])

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@GarrettWu GarrettWu requested a review from shobsi March 7, 2024 22:54
@GarrettWu GarrettWu added the automerge Merge the pull request once unit tests and other checks pass. label Mar 7, 2024
@@ -155,6 +155,7 @@ def __hash__(self):
@dataclass(frozen=True)
class ReadLocalNode(BigFrameNode):
feather_bytes: bytes
session: bigframes.session.Session

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Local data is session-independent, we don't want to add a session constrain to the node. Don't worry about a dataframe/block not having session, that just means you can execute it anywhere, as all the data sources are local.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The data itself is independent of session, yes. But when reading a local data, a specific session will be used. And when executing the query, we'd call that particular session. Do we have other options than keep a record here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A session will be used to execute the tree, yes, but the choice of session need not be constrained by the tree itself. You can check a tree to see if it has a required session, and otherwise, just use the default session to execute trees that don't depend on a specific session.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline, lets just make the session an optional field that we set when users have a specific session they used to read the local data with session.read_gbq.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

Copy link

Merge-on-green attempted to merge your PR for 6 hours, but it was not mergeable because either one of your required status checks failed, one of your required reviews was not approved, or there is a do not merge label. Learn more about your required status checks here: https://help.github.com/en/github/administering-a-repository/enabling-required-status-checks. You can remove and reapply the label to re-run the bot.

@gcf-merge-on-green gcf-merge-on-green bot removed the automerge Merge the pull request once unit tests and other checks pass. label Mar 8, 2024
@product-auto-label product-auto-label bot added size: m Pull request size is medium. and removed size: s Pull request size is small. labels Mar 8, 2024
@GarrettWu GarrettWu added the automerge Merge the pull request once unit tests and other checks pass. label Mar 12, 2024
@gcf-merge-on-green gcf-merge-on-green bot merged commit ae0e3ea into main Mar 12, 2024
14 of 15 checks passed
@gcf-merge-on-green gcf-merge-on-green bot deleted the garrettwu-fix branch March 12, 2024 18:50
@gcf-merge-on-green gcf-merge-on-green bot removed the automerge Merge the pull request once unit tests and other checks pass. label Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants