Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FR: dtype for Decimal data #49

Open
tswast opened this issue Dec 15, 2021 · 2 comments
Open

FR: dtype for Decimal data #49

tswast opened this issue Dec 15, 2021 · 2 comments
Labels
api: bigquery Issues related to the googleapis/python-db-dtypes-pandas API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@tswast
Copy link
Collaborator

tswast commented Dec 15, 2021

Currently, if one wants to store decimal.Decimal data, the pandas object dtype must be used.

Alternatively, one could use one of the pyarrow decimal types with fletcher https://github.com/xhochy/fletcher

I'm opening this issue to investigate how a (db)decimal dtype(s) might work and what the benefits would be beyond the obvious performance improvement potential.

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-db-dtypes-pandas API. label Dec 15, 2021
@tswast tswast added the type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. label Dec 15, 2021
@tswast
Copy link
Collaborator Author

tswast commented Jan 25, 2022

There's already a DecimalDtype defined in tests.

https://github.com/pandas-dev/pandas/blob/ae5b2b7bab84a9c276ca14d0a46060735a0bd601/pandas/tests/extension/decimal/array.py#L38

Added 4 years ago in pandas-dev/pandas#19520

I imagine we'd want something backed by pyarrow these days, as there isn't really a natural numpy type, especially if we wanted to support NULL values.

@tswast
Copy link
Collaborator Author

tswast commented Aug 6, 2024

Possibly redundant with pandas ArrowDtype + arrow decimal128 / decimal256, as these can unambiguously describe BigQuery's NUMERIC and BIGNUMERIC types, respectively.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-db-dtypes-pandas API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

1 participant