We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
I have confirmed this bug exists on the main branch of pandas.
import pandas as pd df = pd.DataFrame( { "dates": [ pd.to_datetime("2025-05-21 18:44:22"), pd.to_datetime("2025-05-21 19:12:42"), ], "tags": [ 12, 45, ] }, ) df["dates"] = df["dates"].astype("datetime64[ms]") print(df.dtypes) print(df) df.to_hdf("dates.h5", key="dates") df2 = pd.read_hdf("dates.h5", key="dates") print(df2) df2["corrected"] = df2["dates"].astype("i8").astype("datetime64[ms]") print(df2)
Dataframes containing dtype of "datetime64[ms]" seem to be correctly written in hdf format, but the readback is misinterpreted as “datetime64[ns]”.
The output of the code above is:
tags int64 dtype: object dates tags 0 2025-05-21 18:44:22 12 1 2025-05-21 19:12:42 45 dates tags 0 1970-01-01 00:29:07.853062 12 1 1970-01-01 00:29:07.854762 45 dates tags corrected 0 1970-01-01 00:29:07.853062 12 2025-05-21 18:44:22 1 1970-01-01 00:29:07.854762 45 2025-05-21 19:12:42``` ### Expected Behavior Correct dates when read back. ### Installed Versions ```INSTALLED VERSIONS ------------------ commit : d9cdd2ee5a58015ef6f4d15c7226110c9aab8140 python : 3.11.9.final.0 python-bits : 64 OS : Windows OS-release : 10 Version : 10.0.22631 machine : AMD64 processor : Intel64 Family 6 Model 183 Stepping 1, GenuineIntel byteorder : little LC_ALL : None LANG : en_US.UTF-8 LOCALE : de_DE.cp1252 pandas : 2.2.2 numpy : 1.26.4 pytz : 2024.1 dateutil : 2.9.0.post0 setuptools : 80.8.0 pip : 25.1.1 Cython : 3.1.1 pytest : 8.3.5 hypothesis : 6.131.20 sphinx : 8.2.3 blosc : None feather : None xlsxwriter : 3.2.3 lxml.etree : 5.4.0 html5lib : 1.1 pymysql : None psycopg2 : None jinja2 : 3.1.6 IPython : 8.36.0 pandas_datareader : None adbc-driver-postgresql: None adbc-driver-sqlite : None bs4 : 4.13.4 bottleneck : 1.5.0 dataframe-api-compat : None fastparquet : None fsspec : 2025.5.0 gcsfs : None matplotlib : 3.8.4 numba : 0.61.2 numexpr : 2.10.2 odfpy : None openpyxl : 3.1.2 pandas_gbq : None pyarrow : 17.0.0 pyreadstat : None python-calamine : None pyxlsb : None s3fs : None scipy : 1.15.3 sqlalchemy : 2.0.30 tables : 3.10.2 tabulate : None xarray : 2025.4.0 xlrd : 2.0.1 zstandard : 0.23.0 tzdata : 2025.2 qtpy : 2.4.3 pyqt5 : None```
The text was updated successfully, but these errors were encountered:
No branches or pull requests
Pandas version checks
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
Issue Description
Dataframes containing dtype of "datetime64[ms]" seem to be correctly written in hdf format, but the readback is misinterpreted as “datetime64[ns]”.
The output of the code above is:
The text was updated successfully, but these errors were encountered: