Pandas 025 has made RangeIndex start, stop & step public and dropped the like-named internal atiributes

[topper-123]

Hi,

Pandas has in master refactored RangeIndex to use a single python range in a internal ._range attribute, rather than three separate _start, _stop & _step integer attributes. Those have been deprecated, emitting a DeprecationWarning.

Public-facing start, stop & step attributes have been added to RangeIndex, and they replace the private ones.

If I understand it correctly, there has been some discussion with pandas maintainers about fastparquet could use those private attributes. If this change causes too much trouble, please let me know.

See pandas-dev/pandas#26581 for the relevant PR.

[martindurant]

So when the next release comes out, users will start getting the deprecation messages, but errors will only happen in Pandas 26? I'd be ok with that, I can make the fix when the time comes and perhaps depend on the latest pandas at that point.

By the way, is either 25 or 26 going py3-only?

[topper-123]

So when the next release comes out, users will start getting the deprecation messages, but errors will only happen in Pandas 26?

Yes, thought the next version after 0.25 will probably be 1.0.

By the way, is either 25 or 26 going py3-only?

Yes, pandas 0.25 will only support python3. The Pandas master branch is already python3 only.

[martindurant]

OK, thank you for letting me know. I'll leave this issue open until after the release.

[toddrme2178]

Pandas 0.25 is out, and the warnings are apparently causing a unit test failure:

=================================== FAILURES ===================================
_________________________ test_datetime_roundtrip[df1] _________________________

tempdir = '/tmp/tmpn3bwybce'
df =                           x
0 2012-03-06 00:00:00+00:00
1 2012-03-06 01:00:00+00:00
2 2012-03-06 02:00:00+00:00
3 2012...0+00:00
6 2012-03-06 06:00:00+00:00
7 2012-03-06 07:00:00+00:00
8 2012-03-06 08:00:00+00:00
9 2012-03-06 09:00:00+00:00
capsys = <_pytest.capture.CaptureFixture object at 0x7f4f25592ef0>

    @pytest.mark.parametrize('df', [
        pd.util.testing.makeMixedDataFrame(),
        pd.DataFrame({'x': pd.date_range('3/6/2012 00:00',
                      periods=10, freq='H', tz='Europe/London')}),
        pd.DataFrame({'x': pd.date_range('3/6/2012 00:00',
                      periods=10, freq='H', tz='Europe/Berlin')}),
        pd.DataFrame({'x': pd.date_range('3/6/2012 00:00',
                      periods=10, freq='H', tz='UTC')})
        ])
    def test_datetime_roundtrip(tempdir, df, capsys):
        fname = os.path.join(tempdir, 'test.parquet')
        w = False
        if 'x' in df and 'Europe/' in str(df.x.dtype.tz):
            with pytest.warns(UserWarning) as w:
                write(fname, df)
        else:
            write(fname, df)
        r = ParquetFile(fname)
    
        if w:
>           assert "UTC" in str(w.list[0].message)
E           AssertionError: assert 'UTC' in 'RangeIndex._start is deprecated and will be removed in a future version. Use RangeIndex.start instead'
E            +  where 'RangeIndex._start is deprecated and will be removed in a future version. Use RangeIndex.start instead' = str(DeprecationWarning('RangeIndex._start is deprecated and will be removed in a future version. Use RangeIndex.start instead'))
E            +    where DeprecationWarning('RangeIndex._start is deprecated and will be removed in a future version. Use RangeIndex.start instead') = <warnings.WarningMessage object at 0x7f4f25592da0>.message

[martindurant]

That is an interesting error! I suppose there are more than just one messages coming out of the operation now; but it should be enough that any one of them contains "UTC". I had not wanted to adopt the new pandas structure so quickly, but maybe this is the time to do that, as well as switch to py3.

[jprafael]

@martindurant From the commenst above and the 332c8dc fix I deduce that this issue was closed because the tests are no longer failing. However the end-users of fastparquet still receive the deprecation warnings, which will only be fixed when fastparquet switches to new pandas/py3. Is this correct? If so, is there a tracking issue for those developments?

[martindurant]

Are people seeing depr warnings now? You may well be right. I'd be happy to see the code try the public attributes first.