Skip to content

BUG: Fix a bug when using DataFrame.to_records with unicode column names #13462

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 11 commits into from
Closed

BUG: Fix a bug when using DataFrame.to_records with unicode column names #13462

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
0a8edfa
BUG: Fix a bug occuring when using DataFrame.to_records with unicode
AlexisMignon Jun 16, 2016
dd64e91
BUG: Add a non regression test for to_records() with unicode column n…
AlexisMignon Jun 16, 2016
2f9b9ec
Resyncing pull request with pydata/master
AlexisMignon Sep 26, 2016
cc94866
Resyncing fork with pydata/pandas
AlexisMignon Nov 25, 2016
35a3569
Changed the way dtype is specified in to_records
AlexisMignon Dec 27, 2016
587b0f2
BUG: Reformatted the test as requested
AlexisMignon Dec 27, 2016
81e80a3
[BUG] Added forgotten assert in test
AlexisMignon Dec 27, 2016
f92632c
CLN: Make the test complient with flake8
AlexisMignon Dec 27, 2016
4fc4a1b
CLN: Remove flake8 warnings
AlexisMignon Dec 27, 2016
a827844
Added bug fix entry in what's new
AlexisMignon Jan 3, 2017
39e8226
Merge branch 'master' into master
AlexisMignon Jan 3, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions pandas/core/frame.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1062,10 +1062,11 @@ def to_records(self, index=True, convert_datetime64=True):
count += 1
elif index_names[0] is None:
index_names = ['index']
names = lmap(str, index_names) + lmap(str, self.columns)
names = (lmap(compat.text_type, index_names) +
lmap(compat.text_type, self.columns))
else:
arrays = [self[c].get_values() for c in self.columns]
names = lmap(str, self.columns)
names = lmap(compat.text_type, self.columns)

dtype = np.dtype([(x, v.dtype) for x, v in zip(names, arrays)])
return np.rec.fromarrays(arrays, dtype=dtype, names=names)
Expand Down
8 changes: 8 additions & 0 deletions pandas/tests/frame/test_convert_to.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,3 +179,11 @@ def test_to_records_with_unicode_index(self):
.to_records()
expected = np.rec.array([('x', 'y')], dtype=[('a', 'O'), ('b', 'O')])
tm.assert_almost_equal(result, expected)

def test_to_records_with_unicode_column_names(self):
# Issue #11879. to_records used to raise an exception when used
# with column names containing non ascii caracters in Python 2
try:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this being fixed by this PR? If so, remove try-except block and compare the result with the expected result.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The expected result is that it doesn't raise an exception.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then, you can remove try-except and compare the result with expected array. We don't use self.fail often because any error results in failure.

DataFrame(columns=[u"accented_name_é"]).to_records()
except UnicodeEncodeError:
self.fail("to_records() raised a UnicodeEncodeError exception")