Type annotate pytest.mark.{skip,skipif,xfail,parametrize}

[bluetech]

These marks are some of the most used user-facing APIs in pytest, but because they are marks, which accept any arguments and are evaluated dynamically, there aren't any places to type them like regular functions.

This PR does a little cheating by special-casing them inside MarkGenerator. This is a gross breach of abstraction and other bad stuff, in particular it is not generalizable to 3rd-party marks (at least, if accessed through pytest.mark). However, due to the pervasiveness of these marks I think it is worth it, until/if we find a better solution.

I typed the marks as complete functions, i.e. building "partial" marks (like get_empty_parameterset_mark in the diff does) is not allowed. I think that's fine for these marks at least.

[nicoddemus]

Thanks! I agree the workaround is worth it, and only affects typing, so it shouldn't introduce unexpected problems.

FTR there is some relation with #5424.

[RonnyPfannschmidt]

Can users opt out of this, as this introduces partial mark incompatible typing

[bluetech]

Can users opt out of this, as this introduces partial mark incompatible typing

As currently written, they can't. Mypy doesn't support functools.partial so I'm pretty sure there isn't a proper way currently to type partial marks.

It is possible for the user to "cheat" by using e.g. getattr(pytest.mark, "xfail") which will give them a MarkDecorator instead of the specific _XfailMarkDecorator subtype, but that of course is not obvious nor recommended.

[nicoddemus]

introduces partial mark incompatible typing

What's that exactly, is it related to functools.partial as @bluetech commented? Can we see an example?

I ask because this is in line with the work started in #5424, where one of the outcomes (at least in my mind) is that we would not generate the builtin marks dynamically anymore, but would have proper signatures declared, very similar to what we have here.

[RonnyPfannschmidt]

@bluetech i believe its possible to type this correct as long as the actual return type supports a overridden call and a override with_params (so stored partial markers could be applied correctly to classes and be expanded at usage sites and still type correct

[bluetech]

Let me clarify the partial mark comment. There is a minor issue there but it's mostly a red herring, I didn't understand it quite right.

This patch only affects the mark-generator (pytest.mark.XXX) step, after the mark is created (pytest.mark.XXX(...)) it becomes just a generic Mark which you can do anything you want with.

So the difference is, that for the builtin marks, the pytest.mark.XXX.__call__ becomes typed:

1. Cannot provide extra args/kwargs that are unexpected
2. Must provide all required args/kwargs

(1) I think is fine, shouldn't do that. If for some reason want to provide extra args, can add them as a second step after the mark is created, e.g. pytest.mark.skip("reason")(extra="foo").

(2) is actually mostly a non-issue because out of all of pytest's builtin marks, only pytest.parametrize has required arguments. And I think it's pretty weird for someone to write pytest.mark.parametrize("key")(["value1", "value2"]). But maybe that's a thing?

@RonnyPfannschmidt mentioned with_args, I haven't noticed it before, but because I don't override it it can also be used as a workaround: pytest.mark.parametrize.with_args("key")(["value1", "value2"]).

[bluetech]

BTW there are a few builtin marks I forgot (looking at pytest --markers), I might add (some of) them:

• pytest.mark.filterwarnings
• pytest.mark.usefixtures
• pytest.mark.tryfirst
• pytest.mark.trylast

[RonnyPfannschmidt]

Thanks for elaborating the details, with that my misunderstanding is cleared

[bluetech]

I added typing also for filterwarnings and usefixtures.