pandas-dev/pandas

.loc assignment with empty label list should not convert dtypes

Closed this issue · 8 comments

jondo commented

Code Sample

import pandas as pd

df = pd.DataFrame({'a':[2,3]})
print(df.a.dtype) # int64
df.loc[[]] = 0.1
print(df.a.dtype) # float64!

Problem description

The .loc-Assignment with empty label list does not change any dataframe row, so it should not convert the column datatype from integer to float.

Seen with the current pandas version 0.25.3.

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit : None python : 3.6.8.final.0 python-bits : 64 OS : Linux OS-release : 4.15.0-62-generic machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : en_DK.utf8 LOCALE : en_DK.UTF-8

pandas : 0.25.3
numpy : 1.17.4
pytz : 2019.3
dateutil : 2.8.1
pip : 19.3.1
setuptools : 41.6.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None

jondo commented

A counter argument would be that the length of the label list should not influence whether the conversion should happen or not. So the behavior might be by design.

mroeschke commented

Thanks for the report! At minimum this doesn't look like friendly behavior and should probably change. I would argue that df.loc[[]] should possibly even raise in this case.

SteffenMeinecke commented

Same issue exists when .loc gets an boolean including only False values, e.g.:

import numpy as np
import pandas as pd

DF = pd.DataFrame([])
DF["col0"] = np.array([1, 4])
DF["col1"] = np.array([6, 5], dtype=object)
print(DF.dtypes)  # int, object
DF.col1.loc[DF.col0 > 10] = "string"
print(DF.dtypes)  # int, int
phofl commented

Works now

parthi-siva commented

Hi @MarvinGravert are you working on it?

MarvinGravert commented

Hi @parthi-siva ,
I am and have posed a PR though i will need to incorporate the feedback i have received on that PR before this issue can be closed. I hope i can get to that in the next few days

parthi-siva commented

Hi @MarvinGravert ..
Sure, Thanks for the reply..
All the best :-)