Skip to content

Rolling correlation for DataFrame with MultiIndex columns broken in Pandas 0.23 #21157

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jjbarton opened this issue May 21, 2018 · 3 comments · Fixed by #43261
Closed

Rolling correlation for DataFrame with MultiIndex columns broken in Pandas 0.23 #21157

jjbarton opened this issue May 21, 2018 · 3 comments · Fixed by #43261
Labels
Bug MultiIndex Regression Functionality that used to work in a prior pandas version Window rolling, ewma, expanding
Milestone

Comments

@jjbarton
Copy link

jjbarton commented May 21, 2018

Code Sample

import numpy as np
import pandas as pd

index = range(5)
data = np.random.random(size=(len(index), 2))

simple_cols = ['M', 'N']
simple_df = pd.DataFrame(data, index=index, columns=simple_cols)

multi_cols = pd.MultiIndex.from_arrays([['M', 'N'], ['P', 'Q']], names=['a', 'b'])
multi_df = pd.DataFrame(data, index=index, columns=multi_cols)

# Works:
simple_df.rolling(3).corr()

# Fails
multi_df.rolling(3).corr()

Problem description

Since the Pandas 0.23 release it is no longer possible to calculate rolling correlation on a pd.DataFrame which has a pd.MultiIndex column index. An exception is raised in 0.23. In 0.22 a valid rolling correlation result is returned.

Expected Output

Pandas should return the same rolling correlation matrix as it returns for a data frame with simple index, but with the multi index levels in columns and index as they appeared in 0.22.

@WillAyd
Copy link
Member

WillAyd commented May 21, 2018

Thanks for the report! Investigation and PRs are always welcome

@WillAyd WillAyd added Regression Functionality that used to work in a prior pandas version MultiIndex labels May 21, 2018
@WillAyd WillAyd added this to the 0.23.1 milestone May 21, 2018
@jreback jreback modified the milestones: 0.23.1, Next Major Release Jun 4, 2018
@WillAyd WillAyd added the Window rolling, ewma, expanding label Sep 4, 2018
@andyshermanash
Copy link

hello, I am also having this same issue in 0.23, which has broken my working code that was previously using 0.22 @jjbarton were you able to solve this issue without reverting back to pandas 0.22?

@jjbarton
Copy link
Author

jjbarton commented Nov 17, 2018 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug MultiIndex Regression Functionality that used to work in a prior pandas version Window rolling, ewma, expanding
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants