BUG: groupby ignores sort=False when observed=True breaking transform

[dhorkel]

• I have checked that this issue has not already been reported.
  Maybe related to but distinct from Bug: groupby(..., observed=True) doesn't respect sort key #27369

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Code Sample

import pandas as pd
df = pd.DataFrame({'cat_col':pd.Categorical(['a','c','a','b']),'other_col':['q','w','w','q'],'val':[1,2,3,4]})
df.groupby(['cat_col','other_col'],observed=False,sort=False)['val'].transform(max)

returns

0    1.0
1    3.0
2    NaN
3    2.0
Name: val, dtype: float64

whereas

df.groupby(['cat_col','other_col'],observed=True,sort=False)['val'].transform(max)

returns

0    1
1    2
2    3
3    4
Name: val, dtype: int64

What is going on is more clear when using .agg()

df.groupby(['cat_col','other_col'],observed=False,sort=False)['val'].agg(max)

returns

a        q            1.0
         w            3.0
c        q            NaN
         w            2.0
b        q            4.0
         w            NaN

whereas

df.groupby(['cat_col','other_col'],observed=True,sort=False)['val'].agg(max)

returns

a        q            1
c        w            2
a        w            3
b        q            4
Name: val, dtype: int64

Problem description

The sort=False is not respected when observed=False. It appears transform() assumes the original order still holds and maps the results to the shape of the original dataframe.

This is an issue when using .transform() to create a new column in the original (unsorted) dataframe as the rows will not be correctly associated.

Expected Output

transform() should give the same result whether observed=True or observed=False in the groupby().

Output of pd.show_versions()

INSTALLED VERSIONS

commit : None
python : 3.7.7.final.0
python-bits : 64
OS : Darwin
OS-release : 18.7.0
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.0.3
numpy : 1.18.2
pytz : 2019.1
dateutil : 2.8.1
pip : 20.0.2
setuptools : 46.0.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : 0.9.3
psycopg2 : 2.8.5 (dt dec pq3 ext lo64)
jinja2 : 2.10.1
IPython : 7.5.0
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : 0.3.1
gcsfs : None
lxml.etree : None
matplotlib : 3.1.1
numexpr : 2.7.1
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 0.16.0
pytables : None
pytest : None
pyxlsb : None
s3fs : 0.4.2
scipy : 1.4.1
sqlalchemy : 1.3.13
tables : 3.6.1
tabulate : None
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None
numba : 0.48.0

[MarcoGorelli]

Thanks @sapientderek for the report!

I think this is the same as #32494, so closing as duplicate, but feel free to reopen if you think it's different