Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

date_range produces wrong values when overflowing #14187

Closed
jorisvandenbossche opened this issue Sep 8, 2016 · 2 comments · Fixed by #23345
Closed

date_range produces wrong values when overflowing #14187

jorisvandenbossche opened this issue Sep 8, 2016 · 2 comments · Fixed by #23345
Labels
Compat pandas objects compatability with Numpy or Python functions Datetime Datetime data dtype Error Reporting Incorrect or improved errors from pandas
Milestone

Comments

@jorisvandenbossche
Copy link
Member

In the following case, you now get a RuntimeWarning. But shouldn't we rather error on this?

In [23]: pd.date_range('1/1/2000', periods=100000, freq='D')
/home/joris/scipy/pandas/pandas/tseries/index.py:1944: RuntimeWarning: overflow encountered in long_scalars
  e = b + np.int64(periods) * stride
/home/joris/scipy/pandas/pandas/tseries/index.py:1954: RuntimeWarning: overflow encountered in long_scalars
  data = np.arange(b, e, stride, dtype=np.int64)
Out[23]: 
DatetimeIndex([          '2000-01-01 00:00:00',
                         '2000-01-02 00:00:00',
                         '2000-01-03 00:00:00',
                         '2000-01-04 00:00:00',
                         '2000-01-05 00:00:00',
                         '2000-01-06 00:00:00',
                         '2000-01-07 00:00:00',
                         '2000-01-08 00:00:00',
                         '2000-01-09 00:00:00',
                         '2000-01-10 00:00:00',
               ...
               '1689-03-17 00:25:26.290448384',
               '1689-03-18 00:25:26.290448384',
               '1689-03-19 00:25:26.290448384',
               '1689-03-20 00:25:26.290448384',
               '1689-03-21 00:25:26.290448384',
               '1689-03-22 00:25:26.290448384',
               '1689-03-23 00:25:26.290448384',
               '1689-03-24 00:25:26.290448384',
               '1689-03-25 00:25:26.290448384',
               '1689-03-26 00:25:26.290448384'],
              dtype='datetime64[ns]', length=100000, freq='D')

In [24]: pd.__version__
Out[24]: '0.19.0rc1+2.gd8cd33b'

In [25]: np.__version__
Out[25]: '1.11.1'

@jreback
Copy link
Contributor

jreback commented Sep 8, 2016

might be now showing because of the np.errstate changes. yes an overflow is not good here (in general we can/should catch the overflow in an offset and just return a datetime, rather than a Timestamp) But here I think its an error as we can't represent that range.

@jreback jreback added Datetime Datetime data dtype Error Reporting Incorrect or improved errors from pandas Compat pandas objects compatability with Numpy or Python functions Difficulty Intermediate labels Sep 8, 2016
@jreback jreback added this to the Next Major Release milestone Sep 8, 2016
@jreback
Copy link
Contributor

jreback commented Sep 8, 2016

something like this would work I think

try:
    with np.errstate(over='raise'):
         e = b + .....
except FloatingPointError:
    raise ValueError("cannot create a date range like that")

@jreback jreback modified the milestones: Next Minor Release, Next Major Release Apr 19, 2017
@jreback jreback modified the milestones: Interesting Issues, Next Major Release Nov 26, 2017
@jreback jreback modified the milestones: Next Major Release, 0.23.0 Feb 19, 2018
@jreback jreback modified the milestones: 0.23.0, Next Major Release Apr 14, 2018
@jreback jreback modified the milestones: Contributions Welcome, 0.24.0 Oct 26, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Compat pandas objects compatability with Numpy or Python functions Datetime Datetime data dtype Error Reporting Incorrect or improved errors from pandas
Projects
None yet
2 participants