You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
UBSAN detects this from running test_sql.py (first error only shown):
/pandas/_libs/src/vendored/numpy/datetime/np_datetime.c:359:33: runtime error: signed integer overflow: 253370764800000000 * 1000 cannot be represented in type 'long int'
What will be tricky about this function is it has an integral return value, where all values are technically valid. Ideally we have some way to avoid the overflow and signal an error via CPython
Issue Description
see above
Expected Behavior
see above
Installed Versions
main
The text was updated successfully, but these errors were encountered:
running also into this issue since today. getting data from bigquery to dataframe and then pandas fails to convert it to json
File "<ipython-input-2-fa9883de5fe4>", line 82, in my_function
rows = df.to_json(orient="records")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/PycharmProjects/data-etl/venv/lib/python3.11/site-packages/pandas/util/_decorators.py", line 333, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/PycharmProjects/data-etl/venv/lib/python3.11/site-packages/pandas/core/generic.py", line 2699, in to_json
return json.to_json(
^^^^^^^^^^^^^
File "/PycharmProjects/data-etl/venv/lib/python3.11/site-packages/pandas/io/json/_json.py", line 210, in to_json
).write()
^^^^^^^
File "/PycharmProjects/data-etl/venv/lib/python3.11/site-packages/pandas/io/json/_json.py", line 263, in write
return ujson_dumps(
^^^^^^^^^^^^
OverflowError: Overflow occurred in npy_datetimestruct_to_datetime
Pandas version checks
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
We have a lot of multiplication in npy_datetimestruct_to_datetime that can overflow:
pandas/pandas/_libs/src/vendored/numpy/datetime/np_datetime.c
Line 308 in 79067a7
UBSAN detects this from running test_sql.py (first error only shown):
/pandas/_libs/src/vendored/numpy/datetime/np_datetime.c:359:33: runtime error: signed integer overflow: 253370764800000000 * 1000 cannot be represented in type 'long int'
What will be tricky about this function is it has an integral return value, where all values are technically valid. Ideally we have some way to avoid the overflow and signal an error via CPython
Issue Description
see above
Expected Behavior
see above
Installed Versions
main
The text was updated successfully, but these errors were encountered: