classmethod's in generic protocols with self-types

[juanarrivillaga]

• Are you reporting a bug, or opening a feature request?
  A potential bug

So, I've posted a corresponding question on StackOverflow, but it hasn't gotten much traction.

Here is my basic situation: I'm trying to use a Protocol that would include a couple of classmethods.
The protocol is generic too, and the signatures involves self-types:

from dataclasses import dataclass
from typing import Union, ClassVar, TypeVar, Generic, Type

from typing_extensions import Protocol


_P = TypeVar('_P', bound='PType')
class PType(Protocol):
    @classmethod
    def maximum_type_value(cls: Type[_P]) -> _P:
        ...
    @classmethod
    def minimum_type_value(cls: Type[_P]) -> _P:
        ...
    def predecessor(self: _P) -> _P:
        ...
    def successor(self: _P) -> _P:
        ...

@dataclass
class MyInteger:
    value: int
    _MAX: ClassVar[int] = 42
    _MIN: ClassVar[int] = -42
    def __post_init__(self) -> None:
        if not (self._MIN <= self.value <= self._MAX):
            msg = f"Integers must be in range [{self._MIN}, {self._MAX}]"
            raise ValueError(msg)
    @classmethod
    def maximum_type_value(cls) -> MyInteger:
        return MyInteger(cls._MAX)
    @classmethod
    def minimum_type_value(cls) -> MyInteger:
        return MyInteger(cls._MIN)
    def predecessor(self) -> MyInteger:
        return MyInteger(self.value - 1)
    def successor(self) -> MyInteger:
        return MyInteger(self.value + 1)


@dataclass
class Interval(Generic[_P]):
    low: _P
    high: _P

interval = Interval(MyInteger(1), MyInteger(2))
def foo(x: PType) -> PType:
    return x
foo(MyInteger(42))

When I try to type-check this, I get the following errors:

(py37) Juans-MacBook-Pro: juan$ mypy mcve.py
mcve.py:46: error: Value of type variable "_P" of "Interval" cannot be "MyInteger"
mcve.py:49: error: Argument 1 to "foo" has incompatible type "MyInteger"; expected "PType"
mcve.py:49: note: Following member(s) of "MyInteger" have conflicts:
mcve.py:49: note:     Expected:
mcve.py:49: note:         def maximum_type_value(cls) -> <nothing>
mcve.py:49: note:     Got:
mcve.py:49: note:         def maximum_type_value(cls) -> MyInteger
mcve.py:49: note:     Expected:
mcve.py:49: note:         def minimum_type_value(cls) -> <nothing>
mcve.py:49: note:     Got:
mcve.py:49: note:         def minimum_type_value(cls) -> MyInteger

Why are the return types registering as <nothing>? When omit the annotation for the cls argument in the protocol, I get the following error:

mcve.py:46: error: Value of type variable "_P" of "Interval" cannot be "MyInteger"
mcve.py:49: error: Argument 1 to "foo" has incompatible type "MyInteger"; expected "PType"
mcve.py:49: note: Following member(s) of "MyInteger" have conflicts:
mcve.py:49: note:     Expected:
mcve.py:49: note:         def [_P <: PType] maximum_type_value(cls) -> _P
mcve.py:49: note:     Got:
mcve.py:49: note:         def maximum_type_value(cls) -> MyInteger
mcve.py:49: note:     Expected:
mcve.py:49: note:         def [_P <: PType] minimum_type_value(cls) -> _P
mcve.py:49: note:     Got:
mcve.py:49: note:         def minimum_type_value(cls) -> MyInteger

Which, quite frankly, makes even less sense to me.

If I make the classmethods instance methods, I receive no error. I would want to avoid this, since it would require a more hefty re-design.

If I try to use a staticmethod (which would be an easier workaround) it also fails:

_P = TypeVar('_P', bound='PType')
class PType(Protocol):
    @staticmethod
    def maximum_type_value() -> _P:
        ...
    @staticmethod
    def minimum_type_value() -> _P:
        ...
    def predecessor(self: _P) -> _P:
        ...
    def successor(self: _P) -> _P:
        ...

@dataclass
class MyInteger:
    value: int
    _MAX: ClassVar[int] = 42
    _MIN: ClassVar[int] = -42
    def __post_init__(self) -> None:
        if not (self._MIN <= self.value <= self._MAX):
            msg = f"Integers must be in range [{self._MIN}, {self._MAX}]"
            raise ValueError(msg)
    @staticmethod
    def maximum_type_value() -> MyInteger:
        return MyInteger(MyInteger._MAX)
    @staticmethod
    def minimum_type_value() -> MyInteger:
        return MyInteger(MyInteger._MIN)
    def predecessor(self) -> MyInteger:
        return MyInteger(self.value - 1)
    def successor(self) -> MyInteger:
        return MyInteger(self.value + 1)

error:

mcve.py:46: error: Value of type variable "_P" of "Interval" cannot be "MyInteger"
mcve.py:49: error: Argument 1 to "foo" has incompatible type "MyInteger"; expected "PType"
mcve.py:49: note: Following member(s) of "MyInteger" have conflicts:
mcve.py:49: note:     Expected:
mcve.py:49: note:         def [_P <: PType] maximum_type_value() -> _P
mcve.py:49: note:     Got:
mcve.py:49: note:         def maximum_type_value() -> MyInteger
mcve.py:49: note:     Expected:
mcve.py:49: note:         def [_P <: PType] minimum_type_value() -> _P
mcve.py:49: note:     Got:
mcve.py:49: note:         def minimum_type_value() -> MyInteger

What is wrong with my Protocol / class definition? Am I missing something obvious?

I am using mypy version 0.620

[ilevkivskyi]

I think this is actually a duplicate of #2511. Here is a repro that doesn't include protocols and dataclasses:

T = TypeVar('T', bound=B)
class B:
    @classmethod
    def m(cls: Type[T]) -> T: ...

class C(B):
    @classmethod
    def m(cls) -> C: ...

[ilevkivskyi]

(As for a workaround you can use # type: ignore to silence that error.)

[juanarrivillaga]

@ilevkivskyi I see, thank you for the response.

Just to clarify, the issue is generics in classmethods that return self-types?

Is it the same issue here: #3645 ?

I'm not sure silencing would be an option for me, because I would have to silence the error every time this generic container class is instantiated, because it never recognizes anything as fulfilling the protocol (unless I misunderstood your suggestion).

I'll probably just refactor to work with an instance method. Thanks for the suggestion anyway.

[ilevkivskyi]

This is different from #3645, but a proper fix will likely fix both.

Yes, using an instance method is a good option.

[SergeyTsaplin]

Excuse me, but why the issue is closed? There is no solution how to use class methods in Protocols, i.e I have the following Protocol and implementation:

from typing import Protocol, TypeVar, Dict, Type, Generic
from dataclasses import dataclass


ModelType = TypeVar("ModelType", bound="DictSerializable")


class DictSerializable(Protocol):
    @classmethod
    def from_dict(cls: Type[ModelType], data: Dict) -> ModelType:
        ...

    def to_dict(self: ModelType) -> Dict:
        ...


@dataclass
class Item:
    value: int

    @classmethod
    def from_dict(cls: Type[Item], data: Dict) -> Item:
        return Item(value=data["value"])

    def to_dict(self) -> Dict:
        return {"value": self.value}


item: DictSerializable = Item(value=1)

Error:

main.py:29: error: Incompatible types in assignment (expression has type "Item", variable has type "DictSerializable")
main.py:29: note: Following member(s) of "Item" have conflicts:
main.py:29: note:     Expected:
main.py:29: note:         def from_dict(cls, data: Dict[Any, Any]) -> <nothing>
main.py:29: note:     Got:
main.py:29: note:         def from_dict(cls, data: Dict[Any, Any]) -> Item
Found 1 error in 1 file (checked 1 source file)

Playground https://mypy-play.net/?mypy=latest&python=3.9&gist=cd5f30078d13e2cac70376a0bdc2ff06

The main problem that the DictSerializable structures possibly are not controlled by the Protocol provider as well as the Protocol user. How can I avoid the errors in the mypy checks without # type: ignore? Ignoring the type makes the type checking useless...

[claudio-ebel]

Hey @SergeyTsaplin,
I stumbled over this issue and your comment.

Excuse me, but why the issue is closed? There is no solution how to use class methods in Protocols, i.e I have the following Protocol and implementation:

…

…

I tried to reproduce the errors you mentioned in your comment with an upgraded version of your minimal example, incorporating the following changes:

• Upgrade the python 3.9 typing style to python 3.10, i.e. Dict → dict
• Change the abstract dict you used in class Item into TypeAlias ItemDict
• Remove the unused import of Generic
• Change the type annotations of the Item method into str since python complains about NameError: name 'Item' is not defined. Did you mean: 'iter'? otherwise

The upgraded code:

from typing import Protocol, TypeAlias, Type, TypeVar
from dataclasses import dataclass

ModelType = TypeVar("ModelType", bound="DictSerializable")
ItemDict: TypeAlias = dict[str, int]

class DictSerializable(Protocol):
    @classmethod
    def from_dict(cls: ModelType, data: dict) -> ModelType:
        ...

    def to_dict(self: ModelType) -> dict:
        ...


@dataclass
class Item:
    value: int

    @classmethod
    def from_dict(cls: Type['Item'], data: ItemDict) -> 'Item':
        return Item(value=data["value"])

    def to_dict(self) -> ItemDict:
        return {"value": self.value}


item: DictSerializable = Item(value=1)
print(item)

I can run this code without problems:

$ mypy spam.py && python spam.py
Success: no issues found in 1 source file
Item(value=1)

using Python 3.10.6 and mypy 1.3.0 (compiled: yes). This issue is now rightfully closed, isn't it?