Segmentation fault during interpreter finalization in PyTraceMalloc_Untrack

[rostan-t]

Crash report

What happened?

_PyTraceMalloc_Fini is called before _PyImport_Fini in _Py_Finalize. This can lead to crashes when using tracemalloc if calls to PyTraceMalloc_Untrack are performed during finalization of module (e.g in the destructor of some global variable).

Commit 6b47499 moved the call to TABLES_LOCK before the check tracemalloc_config.tracing so now the issue whether or not tracemalloc is started.

The following extension module reproduces the issue:

#define PY_SSIZE_T_CLEAN
#include <Python.h>
#include <numpy/arrayobject.h>

static PyObject* arr = NULL;

static struct PyModuleDef repromodule = {
    PyModuleDef_HEAD_INIT,
    "repro",
    NULL,
    -1,
    NULL
};

PyMODINIT_FUNC
PyInit_repro(void)
{
    PyObject* m;

    m = PyModule_Create(&repromodule);
    if (!m)
        return NULL;

    import_array();

    arr = PyArray_SimpleNew(1, (npy_intp[]){0}, NPY_DOUBLE);
    if (!arr || PyModule_AddObject(m, "arr", arr) < 0) {
        Py_XDECREF(arr);
        Py_DECREF(m);
        return NULL;
    }

    return m;
}

$ python -c 'import repro'

It declares a global NumPy array and array_dealloc calls PyTraceMalloc_Untrack, creating a segfault.

For context, I initially ran into the issue in a pybind11 extension declaring functions with numpy array as defaults arguments.

CPython versions tested on:

CPython main branch, 3.13

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.13.1+ (heads/3.13:a3797492179, Jan 22 2025, 11:46:15) [GCC 11.4.0]

Linked PRs

• gh-129185: Fix crash during interpreter finalization in PyTraceMalloc_Untrack #129188
• gh-129185: Fix PyTraceMalloc_Untrack() at Python exit #129191
• [3.13] gh-129185: Fix PyTraceMalloc_Untrack() at Python exit (#129191) #129217
• gh-129185: Remove internal TRACE_RAW_MALLOC macro #129218
• [3.12] gh-129185: Fix PyTraceMalloc_Untrack() at Python exit (#129191) (#129217) #129221
• gh-129185: Use PyMutex in tracemalloc #129246
• gh-129185: Simplify PyTraceMalloc_Track() #129256

[picnixz]

Btw, PyModule_AddObject steals a reference, so you might want to incref arr before since you're storing it in a global variable but I don't know whether this would solve the issue. Now, can you actually produce a reproducer that does not depend on numpy so that we can possibly add a regression test?

cc @vstinner

[vstinner]

I wrote a different fix with a test: PR gh-129191.

[vstinner]

Thanks @rostan-t for the bug report and for the initial fix! It's now fixed in 3.13 and main branches, and the 3.12 fix is incoming.