Merge branch 'main' into code_hash

Author: sweeneyde

.github/workflows/doc.yml

@@ -50,18 +50,8 @@ jobs:
       run: make -C Doc/ venv
     - name: 'Check documentation'
       run: make -C Doc/ check
-    - name: 'Upload NEWS'
-      uses: actions/upload-artifact@v3
-      with:
-        name: NEWS
-        path: Doc/build/NEWS
     - name: 'Build HTML documentation'
       run: make -C Doc/ SPHINXOPTS="-q" SPHINXERRORHANDLING="-W --keep-going" html
-    - name: 'Upload docs'
-      uses: actions/upload-artifact@v3
-      with:
-        name: doc-html
-        path: Doc/build/html
 
   # Run "doctest" on HEAD as new syntax doesn't exist in the latest stable release
   doctest:

Doc/c-api/apiabiversion.rst

@@ -58,6 +58,8 @@ See :ref:`stable` for a discussion of API and ABI stability across versions.
    Thus ``3.4.1a2`` is hexversion ``0x030401a2`` and ``3.10.0`` is
    hexversion ``0x030a00f0``.
 
+   Use this for numeric comparisons, e.g. ``#if PY_VERSION_HEX >= ...``.
+
    This version is also available via the symbol :data:`Py_Version`.
 
 .. c:var:: const unsigned long Py_Version

Doc/c-api/structures.rst

@@ -228,29 +228,30 @@ Implementing functions and methods
    Structure used to describe a method of an extension type.  This structure has
    four fields:
 
-   +------------------+---------------+-------------------------------+
-   | Field            | C Type        | Meaning                       |
-   +==================+===============+===============================+
-   | :attr:`ml_name`  | const char \* | name of the method            |
-   +------------------+---------------+-------------------------------+
-   | :attr:`ml_meth`  | PyCFunction   | pointer to the C              |
-   |                  |               | implementation                |
-   +------------------+---------------+-------------------------------+
-   | :attr:`ml_flags` | int           | flag bits indicating how the  |
-   |                  |               | call should be constructed    |
-   +------------------+---------------+-------------------------------+
-   | :attr:`ml_doc`   | const char \* | points to the contents of the |
-   |                  |               | docstring                     |
-   +------------------+---------------+-------------------------------+
-
-The :attr:`ml_meth` is a C function pointer.  The functions may be of different
+   .. c:member:: const char* ml_name
+
+      name of the method
+
+   .. c:member:: PyCFunction ml_meth
+
+      pointer to the C implementation
+
+   .. c:member:: int ml_flags
+
+      flags bits indicating how the call should be constructed
+
+   .. c:member:: const char* ml_doc
+
+      points to the contents of the docstring
+
+The :c:member:`ml_meth` is a C function pointer.  The functions may be of different
 types, but they always return :c:expr:`PyObject*`.  If the function is not of
 the :c:type:`PyCFunction`, the compiler will require a cast in the method table.
 Even though :c:type:`PyCFunction` defines the first parameter as
 :c:expr:`PyObject*`, it is common that the method implementation uses the
 specific C type of the *self* object.
 
-The :attr:`ml_flags` field is a bitfield which can include the following flags.
+The :c:member:`ml_flags` field is a bitfield which can include the following flags.
 The individual flags indicate either a calling convention or a binding
 convention.
 

Doc/faq/programming.rst

@@ -113,6 +113,8 @@ Yes.  The coding style required for standard library modules is documented as
 Core Language
 =============
 
+.. _faq-unboundlocalerror:
+
 Why am I getting an UnboundLocalError when the variable has a value?
 --------------------------------------------------------------------
 

Doc/library/contextvars.rst

@@ -144,6 +144,11 @@ Manual Context Management
    To get a copy of the current context use the
    :func:`~contextvars.copy_context` function.
 
+   Every thread will have a different top-level :class:`~contextvars.Context`
+   object. This means that a :class:`ContextVar` object behaves in a similar
+   fashion to :func:`threading.local()` when values are assigned in different
+   threads.
+
    Context implements the :class:`collections.abc.Mapping` interface.
 
    .. method:: run(callable, *args, **kwargs)

Doc/library/imaplib.rst

@@ -187,7 +187,7 @@ IMAP4 Objects
 -------------
 
 All IMAP4rev1 commands are represented by methods of the same name, either
-upper-case or lower-case.
+uppercase or lowercase.
 
 All arguments to commands are converted to strings, except for ``AUTHENTICATE``,
 and the last argument to ``APPEND`` which is passed as an IMAP4 literal.  If

Doc/library/inspect.rst

@@ -343,15 +343,36 @@ attributes (see :ref:`import-mod-attrs` for module attributes):
 
 .. function:: iscoroutinefunction(object)
 
-   Return ``True`` if the object is a :term:`coroutine function`
-   (a function defined with an :keyword:`async def` syntax).
+   Return ``True`` if the object is a :term:`coroutine function` (a function
+   defined with an :keyword:`async def` syntax), a :func:`functools.partial`
+   wrapping a :term:`coroutine function`, or a sync function marked with
+   :func:`markcoroutinefunction`.
 
    .. versionadded:: 3.5
 
    .. versionchanged:: 3.8
       Functions wrapped in :func:`functools.partial` now return ``True`` if the
       wrapped function is a :term:`coroutine function`.
 
+   .. versionchanged:: 3.12
+      Sync functions marked with :func:`markcoroutinefunction` now return
+      ``True``.
+
+
+.. function:: markcoroutinefunction(func)
+
+   Decorator to mark a callable as a :term:`coroutine function` if it would not
+   otherwise be detected by :func:`iscoroutinefunction`.
+
+   This may be of use for sync functions that return a :term:`coroutine`, if
+   the function is passed to an API that requires :func:`iscoroutinefunction`.
+
+   When possible, using an :keyword:`async def` function is preferred. Also
+   acceptable is calling the function and testing the return with
+   :func:`iscoroutine`.
+
+   .. versionadded:: 3.12
+
 
 .. function:: iscoroutine(object)
 

Doc/library/re.rst

@@ -591,10 +591,9 @@ character ``'$'``.
 
 ``\w``
    For Unicode (str) patterns:
-      Matches Unicode word characters; this includes most characters
-      that can be part of a word in any language, as well as numbers and
-      the underscore. If the :const:`ASCII` flag is used, only
-      ``[a-zA-Z0-9_]`` is matched.
+      Matches Unicode word characters; this includes alphanumeric characters (as defined by :meth:`str.isalnum`)
+      as well as the underscore (``_``).
+      If the :const:`ASCII` flag is used, only ``[a-zA-Z0-9_]`` is matched.
 
    For 8-bit (bytes) patterns:
       Matches characters considered alphanumeric in the ASCII character set;

Doc/library/ssl.rst

@@ -823,6 +823,13 @@ Constants
 
    .. versionadded:: 3.12
 
+.. data:: OP_LEGACY_SERVER_CONNECT
+
+   Allow legacy insecure renegotiation between OpenSSL and unpatched servers
+   only.
+
+   .. versionadded:: 3.12
+
 .. data:: HAS_ALPN
 
    Whether the OpenSSL library has built-in support for the *Application-Layer

Doc/library/stdtypes.rst

@@ -1624,25 +1624,28 @@ expression support in the :mod:`re` module).
 
 .. method:: str.encode(encoding="utf-8", errors="strict")
 
-   Return an encoded version of the string as a bytes object. Default encoding
-   is ``'utf-8'``. *errors* may be given to set a different error handling scheme.
-   The default for *errors* is ``'strict'``, meaning that encoding errors raise
-   a :exc:`UnicodeError`. Other possible
-   values are ``'ignore'``, ``'replace'``, ``'xmlcharrefreplace'``,
-   ``'backslashreplace'`` and any other name registered via
-   :func:`codecs.register_error`, see section :ref:`error-handlers`. For a
-   list of possible encodings, see section :ref:`standard-encodings`.
-
-   By default, the *errors* argument is not checked for best performances, but
-   only used at the first encoding error. Enable the :ref:`Python Development
-   Mode <devmode>`, or use a :ref:`debug build <debug-build>` to check
-   *errors*.
+   Return the string encoded to :class:`bytes`.
+
+   *encoding* defaults to ``'utf-8'``;
+   see :ref:`standard-encodings` for possible values.
+
+   *errors* controls how encoding errors are handled.
+   If ``'strict'`` (the default), a :exc:`UnicodeError` exception is raised.
+   Other possible values are ``'ignore'``,
+   ``'replace'``, ``'xmlcharrefreplace'``, ``'backslashreplace'`` and any
+   other name registered via :func:`codecs.register_error`.
+   See :ref:`error-handlers` for details.
+
+   For performance reasons, the value of *errors* is not checked for validity
+   unless an encoding error actually occurs,
+   :ref:`devmode` is enabled
+   or a :ref:`debug build <debug-build>` is used.
 
    .. versionchanged:: 3.1
-      Support for keyword arguments added.
+      Added support for keyword arguments.
 
    .. versionchanged:: 3.9
-      The *errors* is now checked in development mode and
+      The value of the *errors* argument is now checked in :ref:`devmode` and
       in :ref:`debug mode <debug-build>`.
 
 
@@ -2759,29 +2762,32 @@ arbitrary binary data.
 .. method:: bytes.decode(encoding="utf-8", errors="strict")
             bytearray.decode(encoding="utf-8", errors="strict")
 
-   Return a string decoded from the given bytes.  Default encoding is
-   ``'utf-8'``. *errors* may be given to set a different
-   error handling scheme.  The default for *errors* is ``'strict'``, meaning
-   that encoding errors raise a :exc:`UnicodeError`.  Other possible values are
-   ``'ignore'``, ``'replace'`` and any other name registered via
-   :func:`codecs.register_error`, see section :ref:`error-handlers`. For a
-   list of possible encodings, see section :ref:`standard-encodings`.
+   Return the bytes decoded to a :class:`str`.
+
+   *encoding* defaults to ``'utf-8'``;
+   see :ref:`standard-encodings` for possible values.
+
+   *errors* controls how decoding errors are handled.
+   If ``'strict'`` (the default), a :exc:`UnicodeError` exception is raised.
+   Other possible values are ``'ignore'``, ``'replace'``,
+   and any other name registered via :func:`codecs.register_error`.
+   See :ref:`error-handlers` for details.
 
-   By default, the *errors* argument is not checked for best performances, but
-   only used at the first decoding error. Enable the :ref:`Python Development
-   Mode <devmode>`, or use a :ref:`debug build <debug-build>` to check *errors*.
+   For performance reasons, the value of *errors* is not checked for validity
+   unless a decoding error actually occurs,
+   :ref:`devmode` is enabled or a :ref:`debug build <debug-build>` is used.
 
    .. note::
 
       Passing the *encoding* argument to :class:`str` allows decoding any
       :term:`bytes-like object` directly, without needing to make a temporary
-      bytes or bytearray object.
+      :class:`!bytes` or :class:`!bytearray` object.
 
    .. versionchanged:: 3.1
       Added support for keyword arguments.
 
    .. versionchanged:: 3.9
-      The *errors* is now checked in development mode and
+      The value of the *errors* argument is now checked in :ref:`devmode` and
       in :ref:`debug mode <debug-build>`.
 
 
@@ -5480,7 +5486,7 @@ to mitigate denial of service attacks. This limit *only* applies to decimal or
 other non-power-of-two number bases. Hexadecimal, octal, and binary conversions
 are unlimited. The limit can be configured.
 
-The :class:`int` type in CPython is an abitrary length number stored in binary
+The :class:`int` type in CPython is an arbitrary length number stored in binary
 form (commonly known as a "bignum"). There exists no algorithm that can convert
 a string to a binary integer or a binary integer to a string in linear time,
 *unless* the base is a power of 2. Even the best known algorithms for base 10
@@ -5544,7 +5550,7 @@ and :class:`str` or :class:`bytes`:
 * ``int(string)`` with default base 10.
 * ``int(string, base)`` for all bases that are not a power of 2.
 * ``str(integer)``.
-* ``repr(integer)``
+* ``repr(integer)``.
 * any other string conversion to base 10, for example ``f"{integer}"``,
   ``"{}".format(integer)``, or ``b"%d" % integer``.
 
@@ -5572,7 +5578,7 @@ command line flag to configure the limit:
   :envvar:`PYTHONINTMAXSTRDIGITS` or :option:`-X int_max_str_digits <-X>`.
   If both the env var and the ``-X`` option are set, the ``-X`` option takes
   precedence. A value of *-1* indicates that both were unset, thus a value of
-  :data:`sys.int_info.default_max_str_digits` was used during initilization.
+  :data:`sys.int_info.default_max_str_digits` was used during initialization.
 
 From code, you can inspect the current limit and set a new one using these
 :mod:`sys` APIs:

Doc/library/weakref.rst

@@ -172,6 +172,30 @@ See :ref:`__slots__ documentation <slots>` for details.
    application without adding attributes to those objects.  This can be especially
    useful with objects that override attribute accesses.
 
+   Note that when a key with equal value to an existing key (but not equal identity)
+   is inserted into the dictionary, it replaces the value but does not replace the
+   existing key. Due to this, when the reference to the original key is deleted, it
+   also deletes the entry in the dictionary::
+
+      >>> class T(str): pass
+      ...
+      >>> k1, k2 = T(), T()
+      >>> d = weakref.WeakKeyDictionary()
+      >>> d[k1] = 1   # d = {k1: 1}
+      >>> d[k2] = 2   # d = {k1: 2}
+      >>> del k1      # d = {}
+
+   A workaround would be to remove the key prior to reassignment::
+
+      >>> class T(str): pass
+      ...
+      >>> k1, k2 = T(), T()
+      >>> d = weakref.WeakKeyDictionary()
+      >>> d[k1] = 1   # d = {k1: 1}
+      >>> del d[k1]
+      >>> d[k2] = 2   # d = {k2: 2}
+      >>> del k1      # d = {k2: 2}
+
    .. versionchanged:: 3.9
       Added support for ``|`` and ``|=`` operators, specified in :pep:`584`.
 

Doc/reference/executionmodel.rst

@@ -128,6 +128,8 @@ lead to errors when a name is used within a block before it is bound.  This rule
 is subtle.  Python lacks declarations and allows name binding operations to
 occur anywhere within a code block.  The local variables of a code block can be
 determined by scanning the entire text of the block for name binding operations.
+See :ref:`the FAQ entry on UnboundLocalError <faq-unboundlocalerror>`
+for examples.
 
 If the :keyword:`global` statement occurs within a block, all uses of the names
 specified in the statement refer to the bindings of those names in the top-level

Doc/whatsnew/3.11.rst

@@ -2319,7 +2319,7 @@ Porting to Python 3.11
   can define the following macros and use them throughout
   the code (credit: these were copied from the ``mypy`` codebase)::
 
-    #if PY_MAJOR_VERSION >= 3 && PY_MINOR_VERSION >= 8
+    #if PY_VERSION_HEX >= 0x03080000
     #  define CPy_TRASHCAN_BEGIN(op, dealloc) Py_TRASHCAN_BEGIN(op, dealloc)
     #  define CPy_TRASHCAN_END(op) Py_TRASHCAN_END
     #else

Doc/whatsnew/3.12.rst

@@ -225,6 +225,15 @@ asyncio
   a custom event loop factory.
   (Contributed by Kumar Aditya in :gh:`99388`.)
 
+* Add C implementation of :func:`asyncio.current_task` for 4x-6x speedup.
+  (Contributed by Itamar Ostricher and Pranav Thulasiram Bhat in :gh:`100344`.)
+
+inspect
+-------
+
+* Add :func:`inspect.markcoroutinefunction` to mark sync functions that return
+  a :term:`coroutine` for use with :func:`iscoroutinefunction`.
+  (Contributed Carlton Gibson in :gh:`99247`.)
 
 pathlib
 -------

Include/internal/pycore_long.h

@@ -110,6 +110,25 @@ PyAPI_FUNC(char*) _PyLong_FormatBytesWriter(
     int base,
     int alternate);
 
+/* Return 1 if the argument is positive single digit int */
+static inline int
+_PyLong_IsPositiveSingleDigit(PyObject* sub) {
+    /*  For a positive single digit int, the value of Py_SIZE(sub) is 0 or 1.
+
+        We perform a fast check using a single comparison by casting from int
+        to uint which casts negative numbers to large positive numbers.
+        For details see Section 14.2 "Bounds Checking" in the Agner Fog
+        optimization manual found at:
+        https://www.agner.org/optimize/optimizing_cpp.pdf
+
+        The function is not affected by -fwrapv, -fno-wrapv and -ftrapv
+        compiler options of GCC and clang
+    */
+    assert(PyLong_CheckExact(sub));
+    Py_ssize_t signed_size = Py_SIZE(sub);
+    return ((size_t)signed_size) <= 1;
+}
+
 #ifdef __cplusplus
 }
 #endif

Lib/asyncio/base_subprocess.py

@@ -215,9 +215,6 @@ def _process_exited(self, returncode):
             # object. On Python 3.6, it is required to avoid a ResourceWarning.
             self._proc.returncode = returncode
         self._call(self._protocol.process_exited)
-        for p in self._pipes.values():
-            if p is not None:
-                p.pipe.close()
 
         self._try_finish()
 

Lib/asyncio/events.py

@@ -836,6 +836,7 @@ def on_fork():
         # Reset the loop and wakeupfd in the forked child process.
         if _event_loop_policy is not None:
             _event_loop_policy._local = BaseDefaultEventLoopPolicy._Local()
+        _set_running_loop(None)
         signal.set_wakeup_fd(-1)
 
     os.register_at_fork(after_in_child=on_fork)