[Data] Support pandas 2.2.0 for pandas tensor extensions

[scottjlee]

Why are these changes needed?

ExtensionArrayFormatter is replaced with _ ExtensionArrayFormatter in pandas 2.2.0. If Ray Data users have this version of pandas installed, it will result in the error in #42842.

This PR adds the case to support the change to ExtensionArrayFormatter. Tested locally with pandas 2.2.0 and Ray Data usage works without the error.

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[c21]

@bveeramani can you help do a double check?

btw @scottjlee is it possible we add a unit test?

[scottjlee]

@bveeramani can you help do a double check?

btw @scottjlee is it possible we add a unit test?

yeah i was attempting this, but wasn't sure how to test different versions of pandas other than adding an entirely new test suite (like we do for various arrow versions). Do you suggest this approach?

I did test the PR locally with pandas 1.5 and pandas 2.2.0, and the fix worked for both cases.

Moreover, we may want to generalize this pattern to also test floor/ceiling versions of pandas and other critical dependencies.

[c21]

SG, we can put the CI test as a followup.

[bveeramani]

LGTM

[fehtemam]

Does anyone know which release has included this merged PR? 2.9.3 didn't have it I think. 2.10 is out now. Does that one include this fix? I couldn't tell by looking at the release notes.

[scottjlee]

Does anyone know which release has included this merged PR? 2.9.3 didn't have it I think. 2.10 is out now. Does that one include this fix? I couldn't tell by looking at the release notes.

@fehtemam Yes, this fix is included in 2.10.