Skip to content

BUG: groupby + rolling with a non-integer window on multi-index fails- #40579

Closed
@velikod

Description

@velikod
  • I have checked that this issue has not already been reported. This seems to be a related issue, but I guess groupby is a complication.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Code Sample, a copy-pastable example

import pandas as pd
data = pd.DataFrame(
    [
        [1, "2020-01-01", "a", 1],
        [2, "2020-01-01", "a", 13],
        [2, "2020-01-02", "a", 4],
        [1, "2020-01-01", "b", 5],
        [1, "2020-01-02", "b", 6],
        [1, "2020-01-03", "c", 9],
        [2, "2020-01-03", "c", 11],
        [2, "2020-01-03", "c", 12],
    ],
    columns=["gr1", "date", "gr2", "value"],
)
data["date"] = pd.to_datetime(data["date"])
data.set_index(["date", "gr1", "gr2"], inplace=True)
data.sort_index(level="date", inplace=True)
print(data)
data.groupby(level="gr1").rolling(window="2D", level="date")["value"].max()

Output:

                    value
date       gr1 gr2       
2020-01-01 1   a        1
               b        5
           2   a       13
2020-01-02 1   b        6
           2   a        4
2020-01-03 1   c        9
           2   c       11
               c       12
data.groupby(level="gr1").rolling(window="2D", level="date")["value"].max()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/pandas/core/groupby/groupby.py", line 1866, in rolling
    return RollingGroupby(self, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/pandas/core/window/rolling.py", line 752, in __init__
    super().__init__(obj, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/pandas/core/window/rolling.py", line 113, in __init__
    self.validate()
  File "/usr/local/lib/python3.9/site-packages/pandas/core/window/rolling.py", line 1921, in validate
    raise ValueError("window must be an integer")
ValueError: window must be an integer

Problem description

Non-integer rolling window seems to not be compatible with multi-index although the level is passed in the rolling. I am actually not sure if this is a Bug or design choice/design flaw. Note that the gr2 column needs to stay in the index otherwise it will be "lost".

Expected Output

                    value
date       gr1 gr2       
2020-01-01 1   a        5
               b        5
           2   a       13
2020-01-02 1   b        6
           2   a       13
2020-01-03 1   c        9
           2   c       12
               c       12

Output of pd.show_versions()

commit : f2c8480
python : 3.9.2.final.0
python-bits : 64
OS : Darwin
OS-release : 20.3.0
Version : Darwin Kernel Version 20.3.0: Thu Jan 21 00:06:51 PST 2021; root:xnu-7195.81.3~1/RELEASE_ARM64_T8101
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : None
LOCALE : None.UTF-8

pandas : 1.2.3
numpy : 1.20.1
pytz : 2021.1
dateutil : 2.8.1
pip : 21.0.1
setuptools : 54.1.2
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : None

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugNeeds TriageIssue that has not been reviewed by a pandas team member

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions