BUG: Unexpected behavior of setting item to DataFrame slice

[joannasendorek]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

df = pd.DataFrame(data={'col1': [1, 2], 'col2': [3, 4], 'col3': [6, 7]})
df[1:] = [1, 2, 3]

Problem description

I get the ValueError: cannot set using a slice indexer with a different length than the value, when running the above code snippet. However, when running the following one, the error doesn't appear:

df = pd.DataFrame(data={'col1': [1, 2], 'col2': [3, 4], 'col3': [6, 7]})
df[1:] = ['a', 'b', 'c']
df[1:] = [1, 2, 3]

Looks like assigning value to slice have different behavior depending on type on the value, which doesn't seem to be intended and is not documented.

Another example is:

df = pd.DataFrame(data={'col1': [1, 2], 'col2': [3, 4], 'e': [6, 7]})
df[1:] = ['a', 'b', 'c']
df[pd.DataFrame({'col1': [True, True], 'col2': [False, True]})] = [2, 3, 4]

raises ValueError: cannot assign mismatch length to masked array for line 3, but this code doesn't:

df = pd.DataFrame(data={'col1': [1, 2], 'col2': [3, 4], 'e': [6, 7]})
df[pd.DataFrame({'col1': [True, True], 'col2': [False, True]})] = [2, 3, 4]

Expected Output

Exceptions thrown should be the same regardless of the previous calls on the given data frame. If assigning value of array of strings have some special meaning, it should be documented. String array assignment shouldn't change behavior of succeeding calls.

Output of pd.show_versions()

INSTALLED VERSIONS

commit : f2c8480
python : 3.8.6.final.0
python-bits : 64
OS : Darwin
OS-release : 20.3.0
Version : Darwin Kernel Version 20.3.0: Thu Jan 21 00:07:06 PST 2021; root:xnu-7195.81.3~1/RELEASE_X86_64
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : None
LOCALE : pl_PL.UTF-8
pandas : 1.2.3
numpy : 1.20.1
pytz : 2021.1
dateutil : 2.8.1
pip : 20.2.3
setuptools : 50.3.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : None

[MarcoGorelli]

Thanks @joannasendorek for the report, can reproduce on master

I don't know what the cause of this is (cc @jbrockmendel ) but using .iloc/.loc works fine here:

In [10]: df = pd.DataFrame(data={'col1': [1, 2], 'col2': [3, 4], 'col3': [6, 7]})

In [11]: df.iloc[1, :] = [1, 2, 3]

In [12]: df.iloc[1, :] = ['a', 'b', 'c']

[phofl]

Not really @MarcoGorelli

df.iloc[1:] = [1, 2, 3]

fails too, because the example in the op is translated into this.

[MarcoGorelli]

@phofl I've probably misunderstood something, but in what sense does df.iloc[1:] = [1, 2, 3] it fail?

In [1]: df = pd.DataFrame(data={'col1': [1, 2], 'col2': [3, 4], 'col3': [6, 7]})
   ...: 

In [2]: df.iloc[1, :] = [1, 2, 3]

In [3]: df
Out[3]: 
   col1  col2  col3
0     1     3     6
1     1     2     3

In [4]: df[1:] = [1, 2, 3]
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-4-1b86f492883a> in <module>
----> 1 df[1:] = [1, 2, 3]

~/pandas-marco/pandas/core/frame.py in __setitem__(self, key, value)
   3539             # either we have a slice or we have a string that can be converted
   3540             #  to a slice for partial-string date indexing
-> 3541             return self._setitem_slice(indexer, value)
   3542 
   3543         if isinstance(key, DataFrame) or getattr(key, "ndim", None) == 2:

~/pandas-marco/pandas/core/frame.py in _setitem_slice(self, key, value)
   3561         #  backwards-compat, xref GH#31469
   3562         self._check_setitem_copy()
-> 3563         self.iloc[key] = value
   3564 
   3565     def _setitem_array(self, key, value):

~/pandas-marco/pandas/core/indexing.py in __setitem__(self, key, value)
    714 
    715         iloc = self if self.name == "iloc" else self.obj.iloc
--> 716         iloc._setitem_with_indexer(indexer, value, self.name)
    717 
    718     def _validate_key(self, key, axis: int):

~/pandas-marco/pandas/core/indexing.py in _setitem_with_indexer(self, indexer, value, name)
   1694             self._setitem_with_indexer_split_path(indexer, value, name)
   1695         else:
-> 1696             self._setitem_single_block(indexer, value, name)
   1697 
   1698     def _setitem_with_indexer_split_path(self, indexer, value, name: str):

~/pandas-marco/pandas/core/indexing.py in _setitem_single_block(self, indexer, value, name)
   1922 
   1923         # actually do the set
-> 1924         self.obj._mgr = self.obj._mgr.setitem(indexer=indexer, value=value)
   1925         self.obj._maybe_update_cacher(clear=True)
   1926 

~/pandas-marco/pandas/core/internals/managers.py in setitem(self, indexer, value)
    594 
    595     def setitem(self, indexer, value) -> BlockManager:
--> 596         return self.apply("setitem", indexer=indexer, value=value)
    597 
    598     def putmask(self, mask, new, align: bool = True):

~/pandas-marco/pandas/core/internals/managers.py in apply(self, f, align_keys, ignore_failures, **kwargs)
    523                     applied = b.apply(f, **kwargs)
    524                 else:
--> 525                     applied = getattr(b, f)(**kwargs)
    526             except (TypeError, NotImplementedError):
    527                 if not ignore_failures:

~/pandas-marco/pandas/core/internals/blocks.py in setitem(self, indexer, value)
    961 
    962         # length checking
--> 963         check_setitem_lengths(indexer, value, values)
    964         exact_match = is_exact_shape_match(values, arr_value)
    965 

~/pandas-marco/pandas/core/indexers.py in check_setitem_lengths(indexer, value, values)
    182         if is_list_like(value):
    183             if len(value) != length_of_indexer(indexer, values):
--> 184                 raise ValueError(
    185                     "cannot set using a slice indexer with a "
    186                     "different length than the value"

ValueError: cannot set using a slice indexer with a different length than the value

In [5]: df.iloc[1, :] = [1, 2, 3]

[phofl]

Without , just df.iloc[1:] = [1,2,3]

[MarcoGorelli]

My bad, I hadn't seen the difference (extra comma), indeed that does fail, thanks

In [1]: df = pd.DataFrame(data={'col1': [1, 2], 'col2': [3, 4], 'col3': [6, 7]})

In [2]: df.iloc[1:] = [1, 2, 3]
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-2-5a2946ddcf10> in <module>
----> 1 df.iloc[1:] = [1, 2, 3]

~/pandas-marco/pandas/core/indexing.py in __setitem__(self, key, value)
    714 
    715         iloc = self if self.name == "iloc" else self.obj.iloc
--> 716         iloc._setitem_with_indexer(indexer, value, self.name)
    717 
    718     def _validate_key(self, key, axis: int):

~/pandas-marco/pandas/core/indexing.py in _setitem_with_indexer(self, indexer, value, name)
   1694             self._setitem_with_indexer_split_path(indexer, value, name)
   1695         else:
-> 1696             self._setitem_single_block(indexer, value, name)
   1697 
   1698     def _setitem_with_indexer_split_path(self, indexer, value, name: str):

~/pandas-marco/pandas/core/indexing.py in _setitem_single_block(self, indexer, value, name)
   1922 
   1923         # actually do the set
-> 1924         self.obj._mgr = self.obj._mgr.setitem(indexer=indexer, value=value)
   1925         self.obj._maybe_update_cacher(clear=True)
   1926 

~/pandas-marco/pandas/core/internals/managers.py in setitem(self, indexer, value)
    594 
    595     def setitem(self, indexer, value) -> BlockManager:
--> 596         return self.apply("setitem", indexer=indexer, value=value)
    597 
    598     def putmask(self, mask, new, align: bool = True):

~/pandas-marco/pandas/core/internals/managers.py in apply(self, f, align_keys, ignore_failures, **kwargs)
    523                     applied = b.apply(f, **kwargs)
    524                 else:
--> 525                     applied = getattr(b, f)(**kwargs)
    526             except (TypeError, NotImplementedError):
    527                 if not ignore_failures:

~/pandas-marco/pandas/core/internals/blocks.py in setitem(self, indexer, value)
    961 
    962         # length checking
--> 963         check_setitem_lengths(indexer, value, values)
    964         exact_match = is_exact_shape_match(values, arr_value)
    965 

~/pandas-marco/pandas/core/indexers.py in check_setitem_lengths(indexer, value, values)
    182         if is_list_like(value):
    183             if len(value) != length_of_indexer(indexer, values):
--> 184                 raise ValueError(
    185                     "cannot set using a slice indexer with a "
    186                     "different length than the value"

ValueError: cannot set using a slice indexer with a different length than the value