Decorator infers Never when using protocols

[brandonchinn178]

I'm trying to add types to a decorator that can be used on methods in classes containing a database_store property. The decorator logs the start/end time to the instance's database_store.

For simplicity, I'll just use a logging.Logger here. (mypy playground)

import logging
import time
from typing import *

class Loggable(Protocol):
    logger: logging.Logger

P = ParamSpec("P")
T = TypeVar("T", covariant=True)
class TimedFunc(Protocol, Generic[P, T]):
    def __call__(_self, self: Loggable, *args: P.args, **kwargs: P.kwargs) -> T:
        ...

def timer(func: TimedFunc[P, T]) -> TimedFunc[P, T]:
    def wrapper(self: Loggable, *args: P.args, **kwargs: P.kwargs) -> T:
        self.logger.info(f"{func} - Start {time.time()}")
        resp = func(self, *args, **kwargs)
        self.logger.info(f"{func} - End {time.time()}")
        return resp
    return wrapper

class Foo:
    def __init__(self) -> None:
        self.logger = logging.getLogger()

    @timer
    def bar(self, value: str) -> str:
        return value

This results in an error

main.py:26: error: Argument 1 to "timer" has incompatible type "Callable[[Foo, str], str]"; expected "TimedFunc[Never, Never]"  [arg-type]
main.py:26: note: "TimedFunc[Never, Never].__call__" has type "Callable[[Arg(Loggable, 'self'), VarArg(Never), KwArg(Never)], Never]"

It works if I hardcode self: Foo instead of self: Loggable.

1. Why are the type variables being inferred as Never?
2. Is this intended to work, or am I doing something wrong?

Other things I tried

• I tried using Concatenate[Loggable, P], but with --strict, this forces self to be a positional argument, and I don't want to update all the methods to do def foo(self, /, ...).

Possibly related

• Self and self are not bound until after class definition, becoming Any/Never when passed through a decorator #16554
• Can't seem to type a decorator that works with ParamSpec on methods #16333

[sterliakov]

If I'm not mistaken, what you're doing is not supposed to work: timer expects a callable that can take any Loggable as the first arg, while your method only takes some specific type of Loggable (namely Foo). So it works if you annotate def bar(self: Loggable, value: str) -> str (but then you aren't able to use self as Foo) or if you opt in for a third type variable to spell "some Loggable":

import logging
import time
from typing import Generic, Protocol, ParamSpec, TypeVar

class Loggable(Protocol):
    logger: logging.Logger

P = ParamSpec("P")
T = TypeVar("T", covariant=True)
S = TypeVar("S", contravariant=True, bound="Loggable")

class TimedFunc(Protocol, Generic[P, T, S]):
    def __call__(_self, self: S, *args: P.args, **kwargs: P.kwargs) -> T:
        ...

def timer(func: TimedFunc[P, T, S]) -> TimedFunc[P, T, S]:
    def wrapper(self: S, *args: P.args, **kwargs: P.kwargs) -> T:
        self.logger.info(f"{func} - Start {time.time()}")
        resp = func(self, *args, **kwargs)
        self.logger.info(f"{func} - End {time.time()}")
        return resp
    return wrapper

class Foo:
    def __init__(self) -> None:
        self.logger = logging.getLogger()

    @timer
    def bar(self, value: str) -> str:
        return value

playground

[brandonchinn178]

Thanks! That gets past the first error. Now I'm encountering an issue where Foo is a subclass and it says the TimedFunc protocol is not compatible with the superclass function.

from typing import *

S = TypeVar("S", contravariant=True)

class FooLike(Protocol, Generic[S]):
    def __call__(_self, self: S, k: str) -> str:
        ...

def decorator(f: FooLike[S]) -> FooLike[S]:
    return f

class Base:
    def foo(self, k: str) -> str:
        return "base"

class Child(Base):
    @decorator
    def foo(self, k: str) -> str:
        return "child"

mypy_protocol_subclass.py:20: error: Signature of "foo" incompatible with supertype "Base"  [override]
mypy_protocol_subclass.py:20: note:      Superclass:
mypy_protocol_subclass.py:20: note:          def foo(self, k: str) -> str
mypy_protocol_subclass.py:20: note:      Subclass:
mypy_protocol_subclass.py:20: note:          FooLike[Child]

(My actual code makes the args + return type generic like def __call__(_self, self: S, *args: P.args, **kwargs: P.kwargs) -> T, but it also fails if I hardcode the signature here, so I left it simplified)

[sterliakov]

Oh, and that's because

from typing import Protocol, TypeVar

S = TypeVar("S", contravariant=True)

class FooLike(Protocol[S]):
    def __call__(_self, self: S, k: str) -> str:
        ...

class A:
    foo: FooLike['A']

A().foo('')  # E: Missing positional argument "k" in call to "__call__" of "FooLike"  [call-arg] \
             # E: Argument 1 to "__call__" of "FooLike" has incompatible type "str"; expected "A"  [arg-type]

FooLike is a callable, yes, but is not guaranteed to undergo method binding (cf. staticmethod). My bad: it was actually broken in my last suggestion as well.

How about this?

import logging
import time
from collections.abc import Callable
from typing import Concatenate, Protocol, ParamSpec, TypeVar

class Loggable(Protocol):
    logger: logging.Logger

P = ParamSpec("P")
T = TypeVar("T", covariant=True)
S = TypeVar("S", contravariant=True, bound="Loggable")

class TimedFunc(Protocol[P, T, S]):
    def __call__(_self, self: S, *args: P.args, **kwargs: P.kwargs) -> T:
        ...

def timer(func: TimedFunc[P, T, S]) -> Callable[Concatenate[S, P], T]:
    def wrapper(self: S, /, *args: P.args, **kwargs: P.kwargs) -> T:
        self.logger.info(f"{func} - Start {time.time()}")
        resp = func(self, *args, **kwargs)
        self.logger.info(f"{func} - End {time.time()}")
        return resp
    return wrapper

class Foo:
    def __init__(self) -> None:
        self.logger = logging.getLogger()

    @timer
    def bar(self, value: str) -> str:
        return value

Foo().bar('')

Yes, there's some asymmetry in the definition, but it should work as long as you're fine with self being posonly in the methods returned by decorator. Works with inheritance too:

# snip

class Base:
    def __init__(self) -> None:
        self.logger = logging.getLogger()
        
    def bar(self, value: str) -> str:
        return ""

class Derived(Base):
    @timer
    def bar(self, value: str) -> str:
        return value

(it probably shouldn't as it violates LSP and .bar(self=Derived(), value="") is allowed on Base but not on Derived, but mypy is sometimes lax with self pos-only-ness; if you want to be fully correct - make self posonly in Base; pyright also accepts this as-is)

[sterliakov]

As this doesn't appear to be a mypy deficiency, I'll close this issue for now. Please feel free to open another ticket if you encounter some other problem with mypy!

[brandonchinn178]

That seems to work @sterliakov, thanks a ton! (Sorry for the late response, got sick)