BUG: Unclear error message assign a column with an empty DataFrame

[fieryash]

Edit: This turned out to be an issue with the error message when setting a single column using a DataFrame with no columns; see #55956 (comment) - rhshadach

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
df = pd.DataFrame()
df['test1'] = df.apply(lambda x: x['test'] + "_",axis=1)

Issue Description

I was expecting to see KeyError 'test' as an error, but I get an error as follows:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[4], line 1
----> 1 df['test1'] = df.apply(lambda x: x['test'] + "_",axis=1)

File ~\AppData\Local\anaconda3\lib\site-packages\pandas\core\frame.py:4081, in DataFrame.__setitem__(self, key, value)
   4079     self._setitem_array(key, value)
   4080 elif isinstance(value, DataFrame):
-> 4081     self._set_item_frame_value(key, value)
   4082 elif (
   4083     is_list_like(value)
   4084     and not self.columns.is_unique
   4085     and 1 < len(self.columns.get_indexer_for([key])) == len(value)
   4086 ):
   4087     # Column to set is duplicated
   4088     self._setitem_array([key], value)

File ~\AppData\Local\anaconda3\lib\site-packages\pandas\core\frame.py:4239, in DataFrame._set_item_frame_value(self, key, value)
   4236     return self.isetitem(locs, value)
   4238 if len(value.columns) != 1:
-> 4239     raise ValueError(
   4240         "Cannot set a DataFrame with multiple columns to the single "
   4241         f"column {key}"
   4242     )
   4244 self[key] = value[value.columns[0]]

ValueError: Cannot set a DataFrame with multiple columns to the single column test1

Expected Behavior

I was expecting to see KeyError 'test' in the error traceback

Installed Versions

Python 3.10.9, pandas 2.1.3, numpy 1.24.3 - Windows 10

---

ImportError Traceback (most recent call last)
Cell In[6], line 1
----> 1 pd.show_versions()

File ~\AppData\Local\anaconda3\lib\site-packages\pandas\util_print_versions.py:141, in show_versions(as_json)
104 """
105 Provide useful information, important for bug reports.
106
(...)
138 ...
139 """
140 sys_info = _get_sys_info()
--> 141 deps = _get_dependency_info()
143 if as_json:
144 j = {"system": sys_info, "dependencies": deps}

File ~\AppData\Local\anaconda3\lib\site-packages\pandas\util_print_versions.py:98, in _get_dependency_info()
96 result: dict[str, JSONSerializable] = {}
97 for modname in deps:
---> 98 mod = import_optional_dependency(modname, errors="ignore")
99 result[modname] = get_version(mod) if mod else None
100 return result

File ~\AppData\Local\anaconda3\lib\site-packages\pandas\compat_optional.py:147, in import_optional_dependency(name, extra, errors, min_version)
145 minimum_version = min_version if min_version is not None else VERSIONS.get(parent)
146 if minimum_version:
--> 147 version = get_version(module_to_get)
148 if version and Version(version) < Version(minimum_version):
149 msg = (
150 f"Pandas requires version '{minimum_version}' or newer of '{parent}' "
151 f"(version '{version}' currently installed)."
152 )

File ~\AppData\Local\anaconda3\lib\site-packages\pandas\compat_optional.py:74, in get_version(module)
71 version = getattr(module, "version", None)
73 if version is None:
---> 74 raise ImportError(f"Can't determine version for {module.name}")
75 if module.name == "psycopg2":
76 # psycopg2 appends " (dt dec pq3 ext lo64)" to it's version
77 version = version.split()[0]

ImportError: Can't determine version for xlrd

[rhshadrach]

I was expecting to see KeyError 'test' in the error traceback

Why are you expecting the provided lambda to get called? E.g.

for x in []:
	raise ValueError

does not raise an error. In your case, there are now rows to iterate over, so you get an empty DataFrame.

df = pd.DataFrame()
print(df.apply(lambda x: x['test'] + "_",axis=1))
# Empty DataFrame
# Columns: []
# Index: []

[fieryash]

I understand that the right side produces an empty DataFrame and it is the assignment that fails.
But, the error message indicates that there are multiple columns involved, which seems confusing. Maybe the error message could be made clearer.

[rob-sil]

Assigning an empty data frame is fine, as long as it only has one column:

df1 = pd.DataFrame()
df1["test1"] = pd.DataFrame() # the same ValueError as in the issue

df2 = pd.DataFrame()
df2["test1"] = pd.DataFrame(columns=["column"]) # successfully creates an empty "test1" column

df3 = pd.DataFrame()
df3["test1"] = pd.DataFrame(columns=["column1", "column2]) # fails with the same Value Error

The "multiple columns" error message is written for the third case, but is raised in your zero-column case as well. Would you find the error clearer if it said something like "Cannot set a DataFrame without any columns to the single column test1"?

[rhshadrach]

Thanks @rob-sil - agreed the error message can be approved.

[JIeJaitt]

Has this issue been fixed? I still encountered this issue in the latest version