Skip to content

Disable unsafe identical code folding in BOLT #622

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 30, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions cpython-unix/build-cpython.sh
Original file line number Diff line number Diff line change
Expand Up @@ -271,6 +271,12 @@ if [ -n "${PYTHON_MEETS_MINIMUM_VERSION_3_12}" ]; then
# https://github.com/python/cpython/issues/128514
patch -p1 -i ${ROOT}/patch-configure-bolt-apply-flags-128514.patch

# Disable unsafe identical code folding. Objects/typeobject.c
# update_one_slot requires that wrap_binaryfunc != wrap_binaryfunc_l,
# despite the functions being identical.
# https://github.com/python/cpython/pull/134642
patch -p1 -i ${ROOT}/patch-configure-bolt-icf-safe.patch

# Tweak --skip-funcs to work with our toolchain.
patch -p1 -i ${ROOT}/patch-configure-bolt-skip-funcs.patch
fi
Expand Down
84 changes: 84 additions & 0 deletions cpython-unix/patch-configure-bolt-icf-safe.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
From 91fc5ae4a5a66a03931f8cd383abd2aa062bb0e9 Mon Sep 17 00:00:00 2001
From: Geoffrey Thomas <[email protected]>
Date: Sat, 24 May 2025 19:04:09 -0400
Subject: [PATCH 1/1] Use only safe identical code folding with BOLT

"Identical code folding" (ICF) is the feature of an optimizer to find that two
functions have the same code and that they can therefore be deduplicated
in the binary. While this is usually safe, it can cause observable
behavior differences if the program relies on the fact that the two
functions have different addresses.

CPython relies on this in (at least) Objects/typeobject.c, which defines
two functions wrap_binaryfunc() and wrap_binaryfunc_l() with the same
implementation, and stores their addresses in the slotdefs array. If
these two functions have the same address, update_one_slot() in that
file will fill in slots it shouldn't, causing, for instances,
classes defined in Python that inherit from some built-in types to
misbehave.

As of LLVM 20 (llvm/llvm-project#116275), BOLT has a "safe ICF" mode,
where it looks to see if there are any uses of a function symbol outside
function calls (e.g., relocations in data sections) and skips ICF on
such functions. The intent is that this avoids observable behavior
differences but still saves storage as much as possible.

This version is about two months old at the time of writing. To support
older LLVM versions, we have to turn off ICF entirely.

This problem was previously noticed for Windows/MSVC in #53093 (and
again in #24098), where the default behavior of PGO is to enable ICF
(which they expand to "identical COMDAT folding") and we had to turn it
off.
---
configure | 50 +++++++++++++++++++++++++++++++++++++++++++++++++-
configure.ac | 25 ++++++++++++++++++++++++-
2 files changed, 73 insertions(+), 2 deletions(-)

diff --git a/configure.ac b/configure.ac
index 8d939f07505..25737e3f9d6 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2129,6 +2129,29 @@ if test "$Py_BOLT" = 'true' ; then
else
AC_MSG_ERROR([merge-fdata is required for a --enable-bolt build but could not be found.])
fi
+
+ py_bolt_icf_flag="-icf=safe"
+ AC_CACHE_CHECK(
+ [whether ${LLVM_BOLT} supports safe identical code folding],
+ [py_cv_bolt_icf_safe],
+ [
+ saved_cflags="$CFLAGS"
+ saved_ldflags="$LDFLAGS"
+ CFLAGS="$CFLAGS_NODIST"
+ LDFLAGS="$LDFLAGS_NODIST"
+ AC_LINK_IFELSE(
+ [AC_LANG_PROGRAM([[]], [[]])],
+ [py_cv_bolt_icf_safe=no
+ ${LLVM_BOLT} -icf=safe -o conftest.bolt conftest$EXEEXT >&AS_MESSAGE_LOG_FD 2>&1 dnl
+ && py_cv_bolt_icf_safe=yes],
+ [AC_MSG_FAILURE([could not compile empty test program])])
+ CFLAGS="$saved_cflags"
+ LDFLAGS="$saved_ldflags"
+ ]
+ )
+ if test "$py_cv_bolt_icf_safe" = no; then
+ py_bolt_icf_flag=""
+ fi
fi

dnl Enable BOLT of libpython if built.
@@ -2184,7 +2207,7 @@ then
-reorder-blocks=ext-tsp
-reorder-functions=cdsort
-split-functions
-split-strategy=cdsplit
- -icf=1
+ ${py_bolt_icf_flag}
-inline-all
-split-eh
-reorder-functions-use-hot-size
--
2.39.5 (Apple Git-154)