The exclude option is being ignored

[r-owen]

I am using pytest-mypy 0.8.0 to check ts_salobj (see link below). It had a generated version.py file that makes mypy unhappy so I am trying to exclude that (on branch tickets/DM-31000, see link below). My configuration works if I run mypy from the command line:

mypy python tests

but if run mypy using pytest-mypy then version.py is processed and the test fails:

pytest -v -m mypy

My configuration includes verbosity=1 and the output shows that version\.py is marked as excluded in both cases, but somehow pytest is pulling it in anyway.

I am not sure how pytest is running mypy. Even with verbosity 2 I can't tell pytest-mypy tells mypy which files to test. I suspect that pytest-mypy is explicitly ordering mypy to test each file and am hoping there is some way to tell it to exclude version.py.

The packages is here https://github.com/lsst-ts/ts_salobj/tree/tickets/DM-31000 and that ticket branch is the one with my attempted changes (including committing the offending version.py file, which normally is not in git).

I admit that I am specifying the mypy config in setup.cfg and the pytest-mypy docs say to use mypy.ini. However, pytest-mypy is clearly using the configuration, based on the verbosity and the resulting log. Also moving or copying the mypy configuration to a new mypy.ini file makes no difference. Here is an excerpt from the printed output from pytest -v -m mypy:

LOG:  Mypy Version:           0.812
LOG:  Config File:            setup.cfg
LOG:  Configured Executable:  /opt/lsst/software/stack/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-0.6.0/bin/python
LOG:  Current Executable:     /opt/lsst/software/stack/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-0.6.0/bin/python
LOG:  Cache Dir:              .mypy_cache
LOG:  Compiled:               True
LOG:  Exclude:                version\.py
LOG:  Found source:           BuildSource(path='/home/saluser/tsrepos/ts_salobj/setup.py', module='setup', has_text=False, base_dir='/home/saluser/tsrepos/ts_salobj')
...
LOG:  Found source:           BuildSource(path='/home/saluser/tsrepos/ts_salobj/python/lsst/ts/salobj/validator.py', module='salobj.validator', has_text=False, base_dir='/home/saluser/tsrepos/ts_salobj/python/lsst/ts')
LOG:  Found source:           BuildSource(path='/home/saluser/tsrepos/ts_salobj/python/lsst/ts/salobj/version.py', module='salobj.version', has_text=False, base_dir='/home/saluser/tsrepos/ts_salobj/python/lsst/ts')
LOG:  Found source:           BuildSource(path='/home/saluser/tsrepos/ts_salobj/python/lsst/ts/salobj/topics/__init__.py', module='salobj.topics', has_text=False, base_dir='/home/saluser/tsrepos/ts_salobj/python/lsst/ts')

and a log from the command line:

LOG:  Mypy Version:           0.812
LOG:  Config File:            setup.cfg
LOG:  Configured Executable:  /opt/lsst/software/stack/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-0.6.0/bin/python
LOG:  Current Executable:     /opt/lsst/software/stack/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-0.6.0/bin/python
LOG:  Cache Dir:              .mypy_cache
LOG:  Compiled:               True
LOG:  Exclude:                version\.py
LOG:  Found source:           BuildSource(path='python/lsst/ts/salobj/__init__.py', module='salobj', has_text=False, base_dir='/home/saluser/tsrepos/ts_salobj/python/lsst/ts')
...
LOG:  Found source:           BuildSource(path='python/lsst/ts/salobj/validator.py', module='salobj.validator', has_text=False, base_dir='/home/saluser/tsrepos/ts_salobj/python/lsst/ts')
LOG:  Found source:           BuildSource(path='tests/data/minimal_salobj_controller.py', module='minimal_salobj_controller', has_text=False, base_dir='/home/saluser/tsrepos/ts_salobj/tests/data')
LOG:  Found source:           BuildSource(path='/home/saluser/tsrepos/ts_salobj/python/lsst/ts/salobj/topics/__init__.py', module='salobj.topics', has_text=False, base_dir='/home/saluser/tsrepos/ts_salobj/python/lsst/ts')

notice that version.py is not listed as a "Found source" in the second case.

[r-owen]

The problem is that pytest is explicitly telling mypy to run version.py, which overrides the exclude = version\.py in the [mypy] section.

The way to prevent this is to add --ignore-glob=*version.py to the addopts section of my config file ([tool:pytest] since I am using setup.cfg).

Having both is a good idea because the [mypy] section applies when running mypy from the command line.

[miketheman]

@r-owen Thanks for noting this, as well as your solution! I've hit a similar issue, and I am trying to figure out how to have the pytest-mypy extension exclude a subdir from mypy checking only, but the --ignore-glob solution applies to all plugins (like pytest-black).

Do you have a similar problem?

[dmtucker]

@miketheman, have you tried the mypy-specific exclude directive?
https://mypy.readthedocs.io/en/stable/config_file.html#confval-exclude

[miketheman]

@dmtucker I did - and it works well enough for mypy CLI invocation, but with pytest-mypy the config doesn't appear to be applied.

[dmtucker]

Wow, sorry... I didn't page this issue back in before responding 😅

Ya, the issue is that the plugin specifies collected files by name, which mypy gives precedence to over exclude:

$ mypy src
src/pytest_mypy.py:223: error: Name "_abspath_errors_type" is not defined
src/pytest_mypy.py:339: error: Incompatible types in assignment (expression has type "None", variable has type "List[str]")
Found 2 errors in 1 file (checked 1 source file)

$ mypy --exclude pytest_mypy.py src
There are no .py[i] files in directory 'src'

$ mypy --exclude pytest_mypy.py src/pytest_mypy.py 
src/pytest_mypy.py:223: error: Name "_abspath_errors_type" is not defined
src/pytest_mypy.py:339: error: Incompatible types in assignment (expression has type "None", variable has type "List[str]")
Found 2 errors in 1 file (checked 1 source file)

This reminds me of #67 (basically the inverse of this: specify which files are ok to check instead of which aren't).

I would endorse @r-owen's solution along the same lines as something I called out in that issue:

In some sense, the point of this plugin is to let pytest handle collection

I think that the issue of that affecting other plugins is interesting, though.

Couple thoughts:

• Would it feasible for you to --ignore-glob the file(s) and explicitly include it for checking by the other plugins, @miketheman?
• Or, you might try implementing pytest_collection_modifyitems like so:

  def pytest_collection_modifyitems(session, config, items):
      for item in items:
          if isinstance(item, MypyFileItem) and item.fspath == "path/to/ignore.py":
              items.remove(item)
              break
      # Remove the status check if we removed the only file check:
      if not any(isinstance(item, MypyFileItem) for item in items):
          for item in items:
              if isinstance(item, MypyStatusItem):
                  items.remove(item)

  • Be advised: I'm not certain this is a pattern Pytest encourages (see Injecting a Session-level Item pytest-dev/pytest#7655).

I admit that I am specifying the mypy config in setup.cfg and the pytest-mypy docs say to use mypy.ini.

FTR, this plugin aims to support passing through config from whatever file(s) mypy chooses to support (i.e. setup.cfg should be fine).

[miketheman]

@dmtucker excellent breakdown of the problems, thanks!

• Would it feasible for you to --ignore-glob the file(s) and explicitly include it for checking by the other plugins?

Maybe? My current flow is to run pytest and the config does all the work for me - and passing the ignore param impacts all other plugins to ignore that file as well (flake8, black, isort, etc) So that's undesirable.

Sure, I could work around it by writing a script/wrapper/etc to run pytest twice - once for mypy and again for everything else, but that seems counter-intuitive.

• Or, you might try implementing pytest_collection_modifyitems like so:

This is interesting, but I think it has the same affect as excluding the item from evaluation in another plugin, doesn't it? If the mypy plugin detects it as a MypyFileItem and then pops it from collection, will it still get evaluated by other plugins?

If so - maybe this is something that gets put into the plugin itself? So plugin doesn't have to concern itself with which config file is in use, and can use mypy's config parser to obtain the user's Options.exclude programmatically, and thus reduce the user-side config?

[astrojuanlu]

Came here to report this same thing. Shouldn't the issue be reopened?

[r-owen]

You can reopen it if you want. But it feels more like an unfortunate side effect than a bug.

[cmartinopen]

2025 here and still having this issue