BUG: stack doesn't preserve lower precision floating dtypes #51602
Conversation
simonjayhawkins
added
Bug
Reshaping
| if common_type.kind == "f" and not is_extension_array_dtype(common_type): | ||
| fill_value = common_type.type(np.nan) | ||
| else: | ||
| fill_value = np.nan |
There was a problem hiding this comment.
Any reason for not using common_dtype.na_value?
Right now "Float32" would add np.nan, but this should be pd.NA, Same for Int64 for example
There was a problem hiding this comment.
The default fill_value is np.nan for reindex, even for EAs. That's probably something to fix elsewhere, though (IIRC, the fill value gets converted somewhere in the EA methods, maybe take? It's been a while since I looked into it though).
There was a problem hiding this comment.
But why can't we supply the correct one?
| fill_value = np.nan | ||
| value_slice = chunk.reindex( | ||
| columns=level_vals_used, fill_value=fill_value | ||
| ).values |
There was a problem hiding this comment.
This is something we'll have to figure out at some point, right now this coerces EA dtypes to object. But ok for another pr since we had this problem before as well...
|
This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this. |
|
This looked okay. Could you merge in main once more and add a whatsnew? |
|
Going to mothball for now but feel free to reopen |
mroeschke
added
the
Mothballed
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.