gh-91153: Fix bytearray holding a reference to its internal buffer when calling into potentially mutating __index__ methods #132379
Conversation
…ay allocation during item assignment
|
All commit authors signed the Contributor License Agreement. |
|
Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool. If this change has little impact on Python users, wait for a maintainer to apply the |
|
I've signed the CLA. Let me know if a NEWS entry is required. I'm not sure it is, especially since there was already a previous change that this is a fix to. |
|
Done |
Misc/NEWS.d/next/Core_and_Builtins/2025-05-17-18-22-12.gh-issue-91153.ioA_83.rst
Outdated
Show resolved
Hide resolved
ZeroIntensity
added
needs backport to 3.13
bast0006
force-pushed
the
bast0006/gh-91153
branch
from
3c1de75 to
d3d1974
Compare
Misc/NEWS.d/next/Core_and_Builtins/2025-05-17-20-56-05.gh-issue-91153.afgtG2.rst
Outdated
Show resolved
Hide resolved
| @@ -805,6 +806,7 @@ bytearray_ass_subscript_lock_held(PyObject *op, PyObject *index, PyObject *value | |||
| /* Delete slice */ | |||
| size_t cur; | |||
| Py_ssize_t i; | |||
| char* buf = PyByteArray_AS_STRING(self); | |||
There was a problem hiding this comment.
We can also hold PyByteArray_GET_SIZE(self) temporarily here.
There was a problem hiding this comment.
I agree, but I'm not as familiar with this system or C as I would want to be to feel comfortable making that change. Especially since the relevant code is mostly >10 years old and that change is not immediately required to solve the issue.
Co-authored-by: Bénédikt Tran <[email protected]>
Lib/test/test_bytes.py
Outdated
| @@ -1889,6 +1889,39 @@ def __index__(self): | |||
| with self.assertRaises(IndexError): | |||
| self._testlimitedcapi.sequence_setitem(b, 0, Boom()) | |||
|
|
|||
| def test_mutating_index_inbounds(self): | |||
| # See gh-91153 | |||
There was a problem hiding this comment.
Could you add a brief description of the issue?
| char *buf = PyByteArray_AS_STRING(self); | ||
| // Do not store a reference to the internal buffer since | ||
| // index.__index__() or _getbytevalue() may alter 'self'. | ||
| // See https://github.com/python/cpython/issues/91153. |
There was a problem hiding this comment.
I don't think we need to fully link to it, just the gh- reference should be fine.
| // See https://github.com/python/cpython/issues/91153. | |
| // See gh-91153 |
There was a problem hiding this comment.
picnixz suggested this above, and it's common for the c source to have complete links, so I think it's fine as-is.
There was a problem hiding this comment.
With complete links, I can open them from my IDE. It's much easier that way IMO
bast0006
force-pushed
the
bast0006/gh-91153
branch
from
5853b5c to
50b7998
Compare
bast0006
force-pushed
the
bast0006/gh-91153
branch
from
50b7998 to
4b7ec9c
Compare
bytearray's
__setitem__implementation currently grabs a reference to its internal buffer before calling_getbyvalueto determine the index that needs assignment._getbyvaluecan call into arbitrary python code via__index__dunders, which could alter the internal buffer and leave said reference dangling.A prior fix for this issue ensures that bounds checking occurs after
_getbyvalueis called. However, python code is capable of resizing the bytearray, resulting in limited but still broken behavior.This patch ensures that the reference to the internal buffer is fetched only after
_getbyvalueis called to prevent it from being held while any python code is run.__index__with side-effects #91153