Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deps: support pandas 2.2 #492

Merged
merged 16 commits into from
Mar 22, 2024
Merged

Conversation

tswast
Copy link
Collaborator

@tswast tswast commented Mar 21, 2024

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

@product-auto-label product-auto-label bot added size: xs Pull request size is extra small. api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. labels Mar 21, 2024
@tswast tswast added kokoro:run Add this label to force Kokoro to re-run the tests. and removed api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. size: xs Pull request size is extra small. labels Mar 21, 2024
@bigframes-bot bigframes-bot removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Mar 21, 2024
@product-auto-label product-auto-label bot added the size: m Pull request size is medium. label Mar 21, 2024
@tswast tswast marked this pull request as ready for review March 21, 2024 18:08
@tswast tswast requested review from a team as code owners March 21, 2024 18:08
@tswast tswast requested a review from milkshakeiii March 21, 2024 18:08
from bigframes.series import Series
# Avoid cirular imports.
if typing.TYPE_CHECKING:
import bigframes.dataframe
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like disambiguating "DataFrame..." should we just always qualify it?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we should. I have a bug open to do that. Issue 296390934. Fully qualified names will also help with docs rendering.

@tswast
Copy link
Collaborator Author

tswast commented Mar 21, 2024

Note: I will wait for e2e tests to come back before merging this one to make sure all our code samples are still working with this change too.

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. label Mar 22, 2024
@tswast
Copy link
Collaborator Author

tswast commented Mar 22, 2024

FAILED tests/system/large/test_remote_function.py::test_remote_function_multiply_with_ibis
FAILED tests/system/large/test_remote_function.py::test_remote_function_stringify_with_ibis

These look like real failures. I'll investigate further this morning.

@tswast
Copy link
Collaborator Author

tswast commented Mar 22, 2024

FAILED notebooks/generative_ai/bq_dataframes_llm_kmeans.ipynb::

This seems to be the same it always has been. I'll try and fix this notebook with some better error handling.

FAILED notebooks/visualization/bq_dataframes_covid_line_graphs.ipynb::

stack trace
_ /tmpfs/src/github/python-bigquery-dataframes/notebooks/visualization/bq_dataframes_covid_line_graphs.ipynb _
[gw1] linux -- Python 3.9.18 /tmpfs/src/github/python-bigquery-dataframes/.nox/notebook-3-9/bin/python
---------------------------------------------------------------------------
import matplotlib.pyplot as plt

# matplotlin will draw a line graph by default
plt.plot(new_cases_usa)
# Rotate the labels on the x axis so that they don't overlap
plt.xticks(rotation=45)
# label the y axis for clarity
plt.ylabel("New Cases")

# Show the plot
plt.show()
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[9], line 4
      1 import matplotlib.pyplot as plt
      3 # matplotlin will draw a line graph by default
----> 4 plt.plot(new_cases_usa)
      5 # Rotate the labels on the x axis so that they don't overlap
      6 plt.xticks(rotation=45)

File /[tmpfs/src/github/python-bigquery-dataframes/.nox/notebook-3-9/lib/python3.9/site-packages/matplotlib/pyplot.py:3590](https://cs.corp.google.com/piper///depot/google3/tmpfs/src/github/python-bigquery-dataframes/.nox/notebook-3-9/lib/python3.9/site-packages/matplotlib/pyplot.py?l=3590), in plot(scalex, scaley, data, *args, **kwargs)
   3582 @_copy_docstring_and_deprecators(Axes.plot)
   3583 def plot(
   3584     *args: float | ArrayLike | str,
   (...)
   3588     **kwargs,
   3589 ) -> list[Line2D]:
-> 3590     return gca().plot(
   3591         *args,
   3592         scalex=scalex,
   3593         scaley=scaley,
   3594         **({"data": data} if data is not None else {}),
   3595         **kwargs,
   3596     )

File /[tmpfs/src/github/python-bigquery-dataframes/.nox/notebook-3-9/lib/python3.9/site-packages/matplotlib/axes/_axes.py:1724](https://cs.corp.google.com/piper///depot/google3/tmpfs/src/github/python-bigquery-dataframes/.nox/notebook-3-9/lib/python3.9/site-packages/matplotlib/axes/_axes.py?l=1724), in Axes.plot(self, scalex, scaley, data, *args, **kwargs)
   1481 """
   1482 Plot y versus x as lines and/or markers.
   1483 
   (...)
   1721 (``'green'``) or hex strings (``'#008000'``).
   1722 """
   1723 kwargs = cbook.normalize_kwargs(kwargs, mlines.Line2D)
-> 1724 lines = [*self._get_lines(self, *args, data=data, **kwargs)]
   1725 for line in lines:
   1726     self.add_line(line)

File /[tmpfs/src/github/python-bigquery-dataframes/.nox/notebook-3-9/lib/python3.9/site-packages/matplotlib/axes/_base.py:303](https://cs.corp.google.com/piper///depot/google3/tmpfs/src/github/python-bigquery-dataframes/.nox/notebook-3-9/lib/python3.9/site-packages/matplotlib/axes/_base.py?l=303), in _process_plot_var_args.__call__(self, axes, data, *args, **kwargs)
    301     this += args[0],
    302     args = args[1:]
--> 303 yield from self._plot_args(
    304     axes, this, kwargs, ambiguous_fmt_datakey=ambiguous_fmt_datakey)

File /[tmpfs/src/github/python-bigquery-dataframes/.nox/notebook-3-9/lib/python3.9/site-packages/matplotlib/axes/_base.py:491](https://cs.corp.google.com/piper///depot/google3/tmpfs/src/github/python-bigquery-dataframes/.nox/notebook-3-9/lib/python3.9/site-packages/matplotlib/axes/_base.py?l=491), in _process_plot_var_args._plot_args(self, axes, tup, kwargs, return_kwargs, ambiguous_fmt_datakey)
    489     y = _check_1d(xy[1])
    490 else:
--> 491     x, y = index_of(xy[-1])
    493 if axes.xaxis is not None:
    494     axes.xaxis.update_units(x)

File /[tmpfs/src/github/python-bigquery-dataframes/.nox/notebook-3-9/lib/python3.9/site-packages/matplotlib/cbook.py:1662](https://cs.corp.google.com/piper///depot/google3/tmpfs/src/github/python-bigquery-dataframes/.nox/notebook-3-9/lib/python3.9/site-packages/matplotlib/cbook.py?l=1662), in index_of(y)
   1641 """
   1642 A helper function to create reasonable x values for the given *y*.
   1643 
   (...)
   1659    The x and y values to plot.
   1660 """
   1661 try:
-> 1662     return y.index.to_numpy(), y.to_numpy()
   1663 except AttributeError:
   1664     pass

File /[tmpfs/src/github/python-bigquery-dataframes/bigframes/core/log_adapter.py:44](https://cs.corp.google.com/piper///depot/google3/tmpfs/src/github/python-bigquery-dataframes/bigframes/core/log_adapter.py?l=44), in method_logger..wrapper(*args, **kwargs)
     42 if api_method_name.startswith("__") or not api_method_name.startswith("_"):
     43     add_api_method(full_method_name)
---> 44 return method(*args, **kwargs)

File /[tmpfs/src/github/python-bigquery-dataframes/bigframes/dataframe.py:2709](https://cs.corp.google.com/piper///depot/google3/tmpfs/src/github/python-bigquery-dataframes/bigframes/dataframe.py?l=2709), in DataFrame.to_numpy(self, dtype, copy, na_value, **kwargs)
   2706 def to_numpy(
   2707     self, dtype=None, copy=False, na_value=None, **kwargs
   2708 ) -> numpy.ndarray:
-> 2709     return self.to_pandas().to_numpy(dtype, copy, na_value, **kwargs)

File /[tmpfs/src/github/python-bigquery-dataframes/.nox/notebook-3-9/lib/python3.9/site-packages/pandas/core/frame.py:1981](https://cs.corp.google.com/piper///depot/google3/tmpfs/src/github/python-bigquery-dataframes/.nox/notebook-3-9/lib/python3.9/site-packages/pandas/core/frame.py?l=1981), in DataFrame.to_numpy(self, dtype, copy, na_value)
   1979 if dtype is not None:
   1980     dtype = np.dtype(dtype)
-> 1981 result = self._mgr.as_array(dtype=dtype, copy=copy, na_value=na_value)
   1982 if result.dtype is not dtype:
   1983     result = np.array(result, dtype=dtype, copy=False)

File /[tmpfs/src/github/python-bigquery-dataframes/.nox/notebook-3-9/lib/python3.9/site-packages/pandas/core/internals/managers.py:1701](https://cs.corp.google.com/piper///depot/google3/tmpfs/src/github/python-bigquery-dataframes/.nox/notebook-3-9/lib/python3.9/site-packages/pandas/core/internals/managers.py?l=1701), in BlockManager.as_array(self, dtype, copy, na_value)
   1699     pass
   1700 else:
-> 1701     arr[isna(arr)] = na_value
   1703 return arr.transpose()

TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

Learn more about nbmake at [https://github.com/treebeardtech/nbmake](https://www.google.com/url?q=https://github.com/treebeardtech/nbmake&sa=D)

This likes potentially a real failure.

Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@tswast tswast added the kokoro:run Add this label to force Kokoro to re-run the tests. label Mar 22, 2024
@bigframes-bot bigframes-bot removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Mar 22, 2024
@tswast tswast merged commit e2cf50e into main Mar 22, 2024
16 checks passed
@tswast tswast deleted the b322845272-pandas-2.2-no-dtype_backend branch March 22, 2024 22:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants