Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pandas extra for google-cloud-bigquery to pick up missing pyarrow #1153

Merged
merged 2 commits into from
Mar 28, 2024

Conversation

mikealfare
Copy link
Contributor

resolves #1152

Problem

google-cloud-bigquery==3.20 added pyarrow>=3.0 as a dependency for the pandas extra and we are not declaring the pandas extra. Either we are using functionality that requires the pandas extra and were not aware or this is a required dependency and not an extra dependency.

Solution

Install google-cloud-bigquery[pandas]>=3.0,<4.0

Checklist

  • I have read the contributing guide and understand what's expected of me
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX

@mikealfare mikealfare self-assigned this Mar 28, 2024
@cla-bot cla-bot bot added the cla:yes label Mar 28, 2024
@mikealfare mikealfare requested a review from a team March 28, 2024 01:07
@mikealfare
Copy link
Contributor Author

Integration tests were skipped, I think because the changed file (setup.py) did not trigger the run. We need to fix that, but for now they are running here: https://github.com/dbt-labs/dbt-bigquery/actions/runs/8460917158.

@mikealfare mikealfare merged commit 978a1ae into main Mar 28, 2024
71 checks passed
@mikealfare mikealfare deleted the deps/pyarrow-3.0 branch March 28, 2024 02:07
Copy link
Contributor

The backport to 1.6.latest failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.6.latest 1.6.latest
# Navigate to the new working tree
cd .worktrees/backport-1.6.latest
# Create a new branch
git switch --create backport-1153-to-1.6.latest
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 978a1aea7d18af79d6d78d0ab5e8d058c4b7c93c
# Push it to GitHub
git push --set-upstream origin backport-1153-to-1.6.latest
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.6.latest

Then, create a pull request where the base branch is 1.6.latest and the compare/head branch is backport-1153-to-1.6.latest.

Copy link
Contributor

The backport to 1.5.latest failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.5.latest 1.5.latest
# Navigate to the new working tree
cd .worktrees/backport-1.5.latest
# Create a new branch
git switch --create backport-1153-to-1.5.latest
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 978a1aea7d18af79d6d78d0ab5e8d058c4b7c93c
# Push it to GitHub
git push --set-upstream origin backport-1153-to-1.5.latest
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.5.latest

Then, create a pull request where the base branch is 1.5.latest and the compare/head branch is backport-1153-to-1.5.latest.

Copy link
Contributor

The backport to 1.4.latest failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.4.latest 1.4.latest
# Navigate to the new working tree
cd .worktrees/backport-1.4.latest
# Create a new branch
git switch --create backport-1153-to-1.4.latest
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 978a1aea7d18af79d6d78d0ab5e8d058c4b7c93c
# Push it to GitHub
git push --set-upstream origin backport-1153-to-1.4.latest
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.4.latest

Then, create a pull request where the base branch is 1.4.latest and the compare/head branch is backport-1153-to-1.4.latest.

Copy link
Contributor

The backport to 1.7.latest failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.7.latest 1.7.latest
# Navigate to the new working tree
cd .worktrees/backport-1.7.latest
# Create a new branch
git switch --create backport-1153-to-1.7.latest
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 978a1aea7d18af79d6d78d0ab5e8d058c4b7c93c
# Push it to GitHub
git push --set-upstream origin backport-1153-to-1.7.latest
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.7.latest

Then, create a pull request where the base branch is 1.7.latest and the compare/head branch is backport-1153-to-1.7.latest.

mikealfare added a commit that referenced this pull request Mar 28, 2024
…arrow` (#1153)

* Add `pandas` extra for `google-cloud-bigquery` to pick up missing `pyarrow`

(cherry picked from commit 978a1ae)
mikealfare added a commit that referenced this pull request Mar 28, 2024
…arrow` (#1153)

* Add `pandas` extra for `google-cloud-bigquery` to pick up missing `pyarrow`

(cherry picked from commit 978a1ae)
mikealfare added a commit that referenced this pull request Mar 28, 2024
…arrow` (#1153)

* Add `pandas` extra for `google-cloud-bigquery` to pick up missing `pyarrow`

(cherry picked from commit 978a1ae)
mikealfare added a commit that referenced this pull request Mar 28, 2024
…arrow` (#1153)

* Add `pandas` extra for `google-cloud-bigquery` to pick up missing `pyarrow`

(cherry picked from commit 978a1ae)
mikealfare added a commit that referenced this pull request Mar 28, 2024
…arrow` (#1153) (#1156)

* Add `pandas` extra for `google-cloud-bigquery` to pick up missing `pyarrow`

(cherry picked from commit 978a1ae)
mikealfare added a commit that referenced this pull request Mar 28, 2024
…arrow` (#1153) (#1155)

* Add `pandas` extra for `google-cloud-bigquery` to pick up missing `pyarrow`

(cherry picked from commit 978a1ae)
mikealfare added a commit that referenced this pull request Mar 28, 2024
…arrow` (#1153) (#1154)

* Add `pandas` extra for `google-cloud-bigquery` to pick up missing `pyarrow`

(cherry picked from commit 978a1ae)
@ddovbii
Copy link

ddovbii commented Mar 28, 2024

Hey Guys!
Will we get this fix if we use 1.5.7?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] pyarrow not found, install pyarrow>=3.0.0
3 participants