TST/CI: PyArrow Test Failures

[gfyoung]

Weird pyarrow failures going on with 37e23d:

https://travis-ci.org/pandas-dev/pandas/jobs/276857797
https://travis-ci.org/pandas-dev/pandas/jobs/276857799

Looks unrelated to commit, so something probably went wrong with pyarrow

cc @jreback @jorisvandenbossche @wesm

[wesm]

Just updated packages to 0.7.0 today, so let's investigate and resolve

[jorisvandenbossche]

Looking at the green build before, there has been a pyarrow update from 0.6.0 to 0.7.0

[wesm]

Yeah, so it looks like a feature that was unsupported before is now supported, so this is a good kind of failure

[wesm]

I recommend that we pin pandas builds at a particular pyarrow version so that we can update pandas and fix bugs in a PR rather than letting builds randomly start failing

[jreback]

most of the builds are pinned

but to catch things like this on purpose i did not pin several

[wesm]

Oh nice. Can we set one of the builds in the build matrix to run against the nightlies (conda install pyarrow -c twosigma) with allowed failures? That might be helpful to get early feedback in case there is a regression or some other failure

[gfyoung]

@jreback : Maybe it would be better to add pyarrow builds like these to allowed failures? No reason to block PR's just because of a 3rd-party library update that is unrelated to their changes.

[gfyoung]

@wesm : Along similar lines to what you said, perhaps we could just run an allowed-failure build of pandas against a nightly build as one of our allowed failures just so we can catch these compat issues in advance.

[jreback]

no its much better to flag this in a highly visible manner - allowed failures are only marked this way to make the build turn 'finish' faster - they r still not allowed

the pr to fix will take about 5 minutes
welcome for someone to do it otherwise i will tomorrow

[jreback]

further his is very common and has happened many times with matplotlib and dateutil among others

[jreback]

we already have a build that hits numpy / scipy dailies. easy enough to add pyarrow

[gfyoung]

they r still not allowed

True, it's just that I think people freak out more when they see a failure on the non-allowed failures.

[wesm]

I am fine either way, but pyarrow at this point is no longer alpha and while APIs may change this package should be working all the time in production from PyPI and conda-forge. So whether or not failures are allowed in Travis CI they should be fixed immediately

[gfyoung]

So whether or not failures are allowed in Travis CI they should be fixed immediately

@wesm : Absolutely. I'm just proposing this change for the sake of not scarring people who are less familiar with our code when they contribute to pandas.

[wesm]

Totally, I'm with you on that. I think allowed failures on the unpinned builds is reasonable