Skip to content

GH-119668: expose importlib.machinery.NamespacePath #119669

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
11 changes: 11 additions & 0 deletions Doc/library/importlib.rst
Original file line number Diff line number Diff line change
Expand Up @@ -985,6 +985,17 @@ find and load modules.
:exc:`ImportError` is raised.


.. class:: NamespacePath(name, path, path_finder)

Represents a namespace package's path.

It uses the module *name* to find its parent module, and from there it looks
up the parent's :attr:`module.__path__`. When this changes, the module's own
path is recomputed, using *path_finder*. The initial value is set to *path*.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we perhaps mention the type of the path finder? (namely using some class Sphinx directive).

Btw, when you say that the initial value is set to path it seems that you are talking about the initial value of the path finder.

In addition, maybe mention that invalidating the caches forces the path to be recomputed (it might not necessarily be the case that the module parent's path has changed).

Finally, could you document the public methods of this new class? For instance there is no insert() method nor remove() as for lists.

I think an example of how to use it could be useful though I don't know whether we want to burden the docs even more.


For top-level modules, the parent module's path is :data:`sys.path`.


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use .. versionadded:: next + what's new entry? While this only exposes a private class, I think users should be notified more than just with a blurb.

.. class:: SourceFileLoader(fullname, path)

A concrete implementation of :class:`importlib.abc.SourceLoader` by
Expand Down
28 changes: 17 additions & 11 deletions Lib/importlib/_bootstrap_external.py
Original file line number Diff line number Diff line change
Expand Up @@ -1079,12 +1079,15 @@ def get_filename(self, fullname):
return self.path


class _NamespacePath:
"""Represents a namespace package's path. It uses the module name
to find its parent module, and from there it looks up the parent's
__path__. When this changes, the module's own path is recomputed,
using path_finder. For top-level modules, the parent module's path
is sys.path."""
class NamespacePath:
"""Represents a namespace package's path.

It uses the module *name* to find its parent module, and from there it looks
up the parent's __path__. When this changes, the module's own path is
recomputed, using *path_finder*. The initial value is set to *path*.

For top-level modules, the parent module's path is sys.path.
"""

# When invalidate_caches() is called, this epoch is incremented
# https://bugs.python.org/issue45703
Expand Down Expand Up @@ -1138,7 +1141,7 @@ def __len__(self):
return len(self._recalculate())

def __repr__(self):
return f'_NamespacePath({self._path!r})'
return f'NamespacePath({self._path!r})'

def __contains__(self, item):
return item in self._recalculate()
Expand All @@ -1147,12 +1150,15 @@ def append(self, item):
self._path.append(item)


_NamespacePath = NamespacePath # for backwards compatibility
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we make it a deprecated alias? (and emit a warning if we were to access the private name? that could help transition from the private to the public name).



# This class is actually exposed publicly in a namespace package's __loader__
# attribute, so it should be available through a non-private name.
# https://github.com/python/cpython/issues/92054
class NamespaceLoader:
def __init__(self, name, path, path_finder):
self._path = _NamespacePath(name, path, path_finder)
self._path = NamespacePath(name, path, path_finder)

def is_package(self, fullname):
return True
Expand Down Expand Up @@ -1207,9 +1213,9 @@ def invalidate_caches():
del sys.path_importer_cache[name]
elif hasattr(finder, 'invalidate_caches'):
finder.invalidate_caches()
# Also invalidate the caches of _NamespacePaths
# Also invalidate the caches of NamespacePaths
# https://bugs.python.org/issue45703
_NamespacePath._epoch += 1
NamespacePath._epoch += 1

from importlib.metadata import MetadataPathFinder
MetadataPathFinder.invalidate_caches()
Expand Down Expand Up @@ -1295,7 +1301,7 @@ def find_spec(cls, fullname, path=None, target=None):
# We found at least one namespace path. Return a spec which
# can create the namespace package.
spec.origin = None
spec.submodule_search_locations = _NamespacePath(fullname, namespace_path, cls._get_spec)
spec.submodule_search_locations = NamespacePath(fullname, namespace_path, cls._get_spec)
return spec
else:
return None
Expand Down
1 change: 1 addition & 0 deletions Lib/importlib/machinery.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
from ._bootstrap_external import ExtensionFileLoader
from ._bootstrap_external import AppleFrameworkLoader
from ._bootstrap_external import NamespaceLoader
from ._bootstrap_external import NamespacePath
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Contrary to the title and what's new entry, this doesn't expose importlib.NamespacePath, but importlib.machinery.NamespacePath:

>>> from importlib import NamespacePath
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    from importlib import NamespacePath
ImportError: cannot import name 'NamespacePath' from 'importlib' (...)
>>> from importlib.machinery import NamespacePath
>>> 

Aside from what you intended, I'm not sure that this isn't actually better. importlib.NamespacePath seems a little odd to me, and while something like importlib.abc might be better, but this isn't really an ABC. So importlib.machinery.NamespacePath seems fine. Care to chime in, @brettcannon ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, sorry! I am okay with either namespace. From the lack of feedback, I think it should be okay to proceed with importlib.machinery.NamespacePath.



def all_suffixes():
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Expose :class:`importlib.machinery.NamespacePath`.
Loading