BUG: ``Series.interpolate`` regression in latest Pandas 3.0.0 nightly (method 'linear' behaves like 'index')
Closed this issue · 7 comments
Pandas version checks
-
I have checked that this issue has not already been reported.
-
I have confirmed this bug exists on the latest version of pandas.
-
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
import numpy as np
import pandas as pd
s = pd.Series([1.0, np.nan, 3.0], index=[1, 3, 4])
s.interpolate(method='linear')
s.interpolate(method='index')Issue Description
The interpolation method 'linear' behaves like the method 'index' with current Pandas 3.0.0 nightly. This is a regression from 2.2.3.
According to the documentation (stable and dev):
Interpolation technique to use. One of:
- ‘linear’: Ignore the index and treat the values as equally spaced. This is the only method supported on MultiIndexes.
[...]- ‘index’: The interpolation uses the numerical values of the DataFrame’s index to linearly calculate missing values.
In the example above, the index is not linearly spaced. But both interpolation methods return the output that is expected for the 'index' method when using the latest Pandas 3.0.0 nightly.
>>> s.interpolate(method='linear')
1 1.000000
3 2.333333
4 3.000000
dtype: float64
>>> s.interpolate(method='index')
1 1.000000
3 2.333333
4 3.000000
dtype: float64
Expected Behavior
The output should be different and 'linear' should ignore the non-linearly spaced index. The expected output should be the same as with Pandas 2.2.3:
>>> s.interpolate(method='linear')
1 1.0
3 2.0
4 3.0
dtype: float64
>>> s.interpolate(method='index')
1 1.000000
3 2.333333
4 3.000000
dtype: float64
Installed Versions
INSTALLED VERSIONS
commit : ddd0aa8
python : 3.12.1
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19044
machine : AMD64
processor : AMD64 Family 23 Model 113 Stepping 0, AuthenticAMD
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_United Kingdom.1252
pandas : 3.0.0.dev0+2010.gddd0aa8dc7
numpy : 2.3.0.dev0+git20250311.a651643
dateutil : 2.9.0.post0
pip : 23.2.1
Cython : None
sphinx : None
IPython : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : None
blosc : None
bottleneck : None
fastparquet : None
fsspec : None
html5lib : None
hypothesis : None
gcsfs : None
jinja2 : None
lxml.etree : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
psycopg2 : None
pymysql : None
pyarrow : None
pyreadstat : None
pytest : None
python-calamine : None
pytz : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlsxwriter : None
zstandard : None
tzdata : 2025.1
qtpy : None
pyqt5 : None
Thanks for the report! A git-bisect reveals:
4f7cb743533d21d3025f9b4fd2f4f1854977cc63 is the first bad commit
commit 4f7cb743533d21d3025f9b4fd2f4f1854977cc63
Author: Carlo Barth
Date: Wed Apr 24 21:41:58 2024 +0200
Fix/time series interpolation is wrong 21351 (#56515)
I believe this should be a simple partial revert of the linked PR. See #56515 (comment)
nice, thanks for looking into it
I'm surprised the tests didn't catch this, probably too many tests using just the default (range) index
@MarcoGorelli - the tests did catch this. Doing the revert I suggested, three tests fail. In particular, this test:
oooh that's bad, I missed it when reviewing, this is on me (I probably thought it was one of the originally broken test cases which needed updating)
thanks @theOehrly for trying out the nightly build and reporting this!
@rhshadrach did you already have a fix ready?
I don't have capacity to think about this problem deeply enough at the moment (and in hindsight i shouldn't have taken on review of it to begin with), so if you don't i would be for just reverting the linked pr, as the previous behavior may well have been undesirable but at least it was as-documented
@MarcoGorelli - yes, indicated in #56515 (comment). I can take this up.