Merge branch 'main' into dictwatch

* main: (94 commits)
  Revert "bpo-46986: Upgrade bundled setuptools to 60.9.3 (GH-31820)" (GH-31879)
  bpo-30677: [doc] mention that os.mkdir() can raise FileNotFoundError (GH-31548)
  git ignore Lib/site-packages (GH-31862)
  bpo-31415: importtime was made by Inada Naoki (GH-31875)
  bpo-46920: Remove code that has explainers why it was disabled (GH-31813)
  bpo-46920: Remove disabled debug code added decades ago and likely unnecessary (GH-31812)
  bpo-46920: Remove code that has no explainer why it was disabled (GH-31814)
  bpo-46906: Mention native endian in PyFloat_Pack8() doc (GH-31866)
  bpo-40280: select: Use NULL for empty fdset (GH-31865)
  CI: Fix patchcheck (GH-31708)
  bpo-46987: Remove _PySys_GetObjectId / _PySys_GetObjectId (GH-31835)
  bpo-46994: Accept explicit contextvars.Context in asyncio create_task() API (GH-31837)
  bpo-39829: Fix `__len__()` is called twice in list() constructor (GH-31816)
  bpo-47003: Cleanup _overlapped module (GH-31848)
  bpo-47004: Sync with importlib_metadata 4.11.3. (#31854)
  bpo-46986: Upgrade bundled setuptools to 60.9.3 (GH-31820)
  bpo-46985: Upgrade bundled pip to 22.0.4 (GH-31819)
  bpo-46805: Add low level UDP socket functions to asyncio (GH-31455)
  bpo-46995: Deprecate missing asyncio.Task.set_name() for third-party task implementations (GH-31838)
  bpo-43215: Document Happy Eyeballs args of asyncio.open_connection (GH-24525)
  ...

Author: Carl Meyer

.azure-pipelines/posix-steps.yml

@@ -66,7 +66,9 @@ steps:
         COMMAND: make
 
 - ${{ if eq(parameters.patchcheck, 'true') }}:
-  - script: ./python Tools/scripts/patchcheck.py --ci true
+  - script: |
+      git fetch origin
+      ./python Tools/scripts/patchcheck.py --ci true
     displayName: 'Run patchcheck.py'
     condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest'))
 

.github/CODEOWNERS

@@ -32,13 +32,13 @@ Python/traceback.c            @iritkatriel
 Python/pythonrun.c            @iritkatriel
 
 # Hashing
-**/*hashlib*                  @python/crypto-team @tiran
-**/*pyhash*                   @python/crypto-team @tiran
-**/*sha*                      @python/crypto-team @tiran
-**/*md5*                      @python/crypto-team @tiran
-**/*blake*                    @python/crypto-team @tiran
-/Modules/_blake2/**           @python/crypto-team @tiran
-/Modules/_sha3/**             @python/crypto-team @tiran
+**/*hashlib*                  @tiran
+**/*pyhash*                   @tiran
+**/*sha*                      @tiran
+**/*md5*                      @tiran
+**/*blake*                    @tiran
+/Modules/_blake2/**           @tiran
+/Modules/_sha3/**             @tiran
 
 # logging
 **/*logging*                  @vsajip
@@ -61,14 +61,6 @@ Python/pythonrun.c            @iritkatriel
 **/*import*.c                 @brettcannon @encukou @ericsnowcurrently @ncoghlan @warsaw
 **/*import*.py                @brettcannon @encukou @ericsnowcurrently @ncoghlan @warsaw
 
-
-# SSL
-**/*ssl*                      @python/crypto-team
-**/*.pem                      @python/crypto-team
-
-# CSPRNG
-Python/bootstrap_hash.c       @python/crypto-team
-
 # Dates and times
 **/*datetime*                 @pganssle @abalkin
 **/*str*time*                 @pganssle @abalkin
@@ -132,7 +124,7 @@ Lib/ast.py                    @isidentical
 **/*bisect*                   @rhettinger
 **/*heapq*                    @rhettinger
 **/*functools*                @rhettinger
-**/*decimal*                  @rhettinger @skrah
+**/*decimal*                  @rhettinger
 
 **/*dataclasses*              @ericvsmith
 
@@ -149,6 +141,8 @@ Lib/ast.py                    @isidentical
 **/*cgi*                      @ethanfurman
 **/*tarfile*                  @ethanfurman
 
+**/*tomllib*                  @encukou
+
 # macOS
 /Mac/                         @python/macos-team
 **/*osx_support*              @python/macos-team

.github/workflows/build.yml

@@ -74,12 +74,14 @@ jobs:
           grep "aclocal 1.16.3" aclocal.m4
           grep -q "runstatedir" configure
           grep -q "PKG_PROG_PKG_CONFIG" aclocal.m4
-      - name: Regenerate autoconf files
-        run: docker run --rm -v $(pwd):/src quay.io/tiran/cpython_autoconf:269
-      - name: Build CPython
+      - name: Configure CPython
         run: |
           # Build Python with the libpython dynamic library
           ./configure --with-pydebug --enable-shared
+      - name: Regenerate autoconf files with container image
+        run: make regen-configure
+      - name: Build CPython
+        run: |
           make -j4 regen-all
           make regen-stdlib-module-names
       - name: Check for changes

.gitignore

@@ -58,6 +58,8 @@ Doc/.env/
 Include/pydtrace_probes.h
 Lib/distutils/command/*.pdb
 Lib/lib2to3/*.pickle
+Lib/site-packages/*
+!Lib/site-packages/README.txt
 Lib/test/data/*
 !Lib/test/data/README
 /_bootstrap_python
@@ -75,6 +77,7 @@ Mac/pythonw
 Misc/python.pc
 Misc/python-embed.pc
 Misc/python-config.sh
+Modules/Setup.bootstrap
 Modules/Setup.config
 Modules/Setup.local
 Modules/Setup.stdlib

Doc/c-api/float.rst

@@ -76,3 +76,89 @@ Floating Point Objects
 .. c:function:: double PyFloat_GetMin()
 
    Return the minimum normalized positive float *DBL_MIN* as C :c:type:`double`.
+
+
+Pack and Unpack functions
+=========================
+
+The pack and unpack functions provide an efficient platform-independent way to
+store floating-point values as byte strings. The Pack routines produce a bytes
+string from a C :c:type:`double`, and the Unpack routines produce a C
+:c:type:`double` from such a bytes string. The suffix (2, 4 or 8) specifies the
+number of bytes in the bytes string.
+
+On platforms that appear to use IEEE 754 formats these functions work by
+copying bits. On other platforms, the 2-byte format is identical to the IEEE
+754 binary16 half-precision format, the 4-byte format (32-bit) is identical to
+the IEEE 754 binary32 single precision format, and the 8-byte format to the
+IEEE 754 binary64 double precision format, although the packing of INFs and
+NaNs (if such things exist on the platform) isn't handled correctly, and
+attempting to unpack a bytes string containing an IEEE INF or NaN will raise an
+exception.
+
+On non-IEEE platforms with more precision, or larger dynamic range, than IEEE
+754 supports, not all values can be packed; on non-IEEE platforms with less
+precision, or smaller dynamic range, not all values can be unpacked. What
+happens in such cases is partly accidental (alas).
+
+.. versionadded:: 3.11
+
+Pack functions
+--------------
+
+The pack routines write 2, 4 or 8 bytes, starting at *p*. *le* is an
+:c:type:`int` argument, non-zero if you want the bytes string in little-endian
+format (exponent last, at ``p+1``, ``p+3``, or ``p+6`` ``p+7``), zero if you
+want big-endian format (exponent first, at *p*). The :c:data:`PY_BIG_ENDIAN`
+constant can be used to use the native endian: it is equal to ``1`` on big
+endian processor, or ``0`` on little endian processor.
+
+Return value: ``0`` if all is OK, ``-1`` if error (and an exception is set,
+most likely :exc:`OverflowError`).
+
+There are two problems on non-IEEE platforms:
+
+* What this does is undefined if *x* is a NaN or infinity.
+* ``-0.0`` and ``+0.0`` produce the same bytes string.
+
+.. c:function:: int PyFloat_Pack2(double x, unsigned char *p, int le)
+
+   Pack a C double as the IEEE 754 binary16 half-precision format.
+
+.. c:function:: int PyFloat_Pack4(double x, unsigned char *p, int le)
+
+   Pack a C double as the IEEE 754 binary32 single precision format.
+
+.. c:function:: int PyFloat_Pack8(double x, unsigned char *p, int le)
+
+   Pack a C double as the IEEE 754 binary64 double precision format.
+
+
+Unpack functions
+----------------
+
+The unpack routines read 2, 4 or 8 bytes, starting at *p*.  *le* is an
+:c:type:`int` argument, non-zero if the bytes string is in little-endian format
+(exponent last, at ``p+1``, ``p+3`` or ``p+6`` and ``p+7``), zero if big-endian
+(exponent first, at *p*). The :c:data:`PY_BIG_ENDIAN` constant can be used to
+use the native endian: it is equal to ``1`` on big endian processor, or ``0``
+on little endian processor.
+
+Return value: The unpacked double.  On error, this is ``-1.0`` and
+:c:func:`PyErr_Occurred` is true (and an exception is set, most likely
+:exc:`OverflowError`).
+
+Note that on a non-IEEE platform this will refuse to unpack a bytes string that
+represents a NaN or infinity.
+
+.. c:function:: double PyFloat_Unpack2(const unsigned char *p, int le)
+
+   Unpack the IEEE 754 binary16 half-precision format as a C double.
+
+.. c:function:: double PyFloat_Unpack4(const unsigned char *p, int le)
+
+   Unpack the IEEE 754 binary32 single precision format as a C double.
+
+.. c:function:: double PyFloat_Unpack8(const unsigned char *p, int le)
+
+   Unpack the IEEE 754 binary64 double precision format as a C double.

Doc/library/asyncio-eventloop.rst

@@ -330,7 +330,7 @@ Creating Futures and Tasks
 
    .. versionadded:: 3.5.2
 
-.. method:: loop.create_task(coro, *, name=None)
+.. method:: loop.create_task(coro, *, name=None, context=None)
 
    Schedule the execution of a :ref:`coroutine`.
    Return a :class:`Task` object.
@@ -342,17 +342,24 @@ Creating Futures and Tasks
    If the *name* argument is provided and not ``None``, it is set as
    the name of the task using :meth:`Task.set_name`.
 
+   An optional keyword-only *context* argument allows specifying a
+   custom :class:`contextvars.Context` for the *coro* to run in.
+   The current context copy is created when no *context* is provided.
+
    .. versionchanged:: 3.8
       Added the *name* parameter.
 
+   .. versionchanged:: 3.11
+      Added the *context* parameter.
+
 .. method:: loop.set_task_factory(factory)
 
    Set a task factory that will be used by
    :meth:`loop.create_task`.
 
    If *factory* is ``None`` the default task factory will be set.
    Otherwise, *factory* must be a *callable* with the signature matching
-   ``(loop, coro)``, where *loop* is a reference to the active
+   ``(loop, coro, context=None)``, where *loop* is a reference to the active
    event loop, and *coro* is a coroutine object.  The callable
    must return a :class:`asyncio.Future`-compatible object.
 
@@ -922,6 +929,29 @@ convenient.
 
    .. versionadded:: 3.7
 
+.. coroutinemethod:: loop.sock_recvfrom(sock, bufsize)
+
+   Receive a datagram of up to *bufsize* from *sock*.  Asynchronous version of
+   :meth:`socket.recvfrom() <socket.socket.recvfrom>`.
+
+   Return a tuple of (received data, remote address).
+
+   *sock* must be a non-blocking socket.
+
+   .. versionadded:: 3.11
+
+.. coroutinemethod:: loop.sock_recvfrom_into(sock, buf, nbytes=0)
+
+   Receive a datagram of up to *nbytes* from *sock* into *buf*.
+   Asynchronous version of
+   :meth:`socket.recvfrom_into() <socket.socket.recvfrom_into>`.
+
+   Return a tuple of (number of bytes received, remote address).
+
+   *sock* must be a non-blocking socket.
+
+   .. versionadded:: 3.11
+
 .. coroutinemethod:: loop.sock_sendall(sock, data)
 
    Send *data* to the *sock* socket. Asynchronous version of
@@ -940,6 +970,18 @@ convenient.
       method, before Python 3.7 it returned a :class:`Future`.
       Since Python 3.7, this is an ``async def`` method.
 
+.. coroutinemethod:: loop.sock_sendto(sock, data, address)
+
+   Send a datagram from *sock* to *address*.
+   Asynchronous version of
+   :meth:`socket.sendto() <socket.socket.sendto>`.
+
+   Return the number of bytes sent.
+
+   *sock* must be a non-blocking socket.
+
+   .. versionadded:: 3.11
+
 .. coroutinemethod:: loop.sock_connect(sock, address)
 
    Connect *sock* to a remote socket at *address*.

Doc/library/asyncio-llapi-index.rst

@@ -189,9 +189,18 @@ See also the main documentation section about the
     * - ``await`` :meth:`loop.sock_recv_into`
       - Receive data from the :class:`~socket.socket` into a buffer.
 
+    * - ``await`` :meth:`loop.sock_recvfrom`
+      - Receive a datagram from the :class:`~socket.socket`.
+
+    * - ``await`` :meth:`loop.sock_recvfrom_into`
+      - Receive a datagram from the :class:`~socket.socket` into a buffer.
+
     * - ``await`` :meth:`loop.sock_sendall`
       - Send data to the :class:`~socket.socket`.
 
+    * - ``await`` :meth:`loop.sock_sendto`
+      - Send a datagram via the :class:`~socket.socket` to the given address.
+
     * - ``await`` :meth:`loop.sock_connect`
       - Connect the :class:`~socket.socket`.
 

Doc/library/asyncio-stream.rst

@@ -51,7 +51,8 @@ and work with streams:
 .. coroutinefunction:: open_connection(host=None, port=None, *, \
                           limit=None, ssl=None, family=0, proto=0, \
                           flags=0, sock=None, local_addr=None, \
-                          server_hostname=None, ssl_handshake_timeout=None)
+                          server_hostname=None, ssl_handshake_timeout=None, \
+                          happy_eyeballs_delay=None, interleave=None)
 
    Establish a network connection and return a pair of
    ``(reader, writer)`` objects.
@@ -69,6 +70,9 @@ and work with streams:
    .. versionchanged:: 3.7
       Added the *ssl_handshake_timeout* parameter.
 
+   .. versionadded:: 3.8
+      Added *happy_eyeballs_delay* and *interleave* parameters.
+
    .. versionchanged:: 3.10
       Removed the *loop* parameter.
 

Doc/library/asyncio-task.rst

@@ -244,14 +244,18 @@ Running an asyncio Program
 Creating Tasks
 ==============
 
-.. function:: create_task(coro, *, name=None)
+.. function:: create_task(coro, *, name=None, context=None)
 
    Wrap the *coro* :ref:`coroutine <coroutine>` into a :class:`Task`
    and schedule its execution.  Return the Task object.
 
    If *name* is not ``None``, it is set as the name of the task using
    :meth:`Task.set_name`.
 
+   An optional keyword-only *context* argument allows specifying a
+   custom :class:`contextvars.Context` for the *coro* to run in.
+   The current context copy is created when no *context* is provided.
+
    The task is executed in the loop returned by :func:`get_running_loop`,
    :exc:`RuntimeError` is raised if there is no running loop in
    current thread.
@@ -281,6 +285,9 @@ Creating Tasks
    .. versionchanged:: 3.8
       Added the *name* parameter.
 
+   .. versionchanged:: 3.11
+      Added the *context* parameter.
+
 
 Sleeping
 ========

Doc/library/fileformats.rst

@@ -12,6 +12,7 @@ that aren't markup languages and are not related to e-mail.
 
    csv.rst
    configparser.rst
+   tomllib.rst
    netrc.rst
    xdrlib.rst
    plistlib.rst

Doc/library/importlib.metadata.rst

@@ -264,6 +264,7 @@ Python packages or modules::
 
 .. versionadded:: 3.10
 
+.. _distributions:
 
 Distributions
 =============
@@ -335,6 +336,3 @@ a custom finder, return instances of this derived ``Distribution`` in the
 .. _`entry point API`: https://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points
 .. _`metadata API`: https://setuptools.readthedocs.io/en/latest/pkg_resources.html#metadata-api
 .. _`importlib_resources`: https://importlib-resources.readthedocs.io/en/latest/index.html
-
-
-.. rubric:: Footnotes

Doc/library/math.rst

@@ -649,6 +649,9 @@ Constants
    A floating-point "not a number" (NaN) value.  Equivalent to the output of
    ``float('nan')``.
 
+   .. versionchanged:: 3.11
+      It is now always available.
+
    .. versionadded:: 3.5
 
 

Doc/library/os.rst

@@ -2050,7 +2050,8 @@ features:
 
    Create a directory named *path* with numeric mode *mode*.
 
-   If the directory already exists, :exc:`FileExistsError` is raised.
+   If the directory already exists, :exc:`FileExistsError` is raised. If a parent
+   directory in the path does not exist, :exc:`FileNotFoundError` is raised.
 
    .. _mkdir_modebits:
 
@@ -4755,6 +4756,9 @@ Miscellaneous System Information
 
    .. availability:: Unix.
 
+   .. versionchanged:: 3.11
+      Add ``'SC_MINSIGSTKSZ'`` name.
+
 The following data values are used to support path manipulation operations.  These
 are defined for all platforms.