[BUG]: Overwriting CMake PYTHON_MODULE_EXTENSION needs PYBIND11_PYTHON_EXECUTABLE_LAST

[ax3l]

Required prerequisites

• Make sure you've read the documentation. Your issue may be addressed there.
• Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
• Consider asking first in the Gitter chat room or in a Discussion.

Problem description

I realized when trying a work-around for Conda-Forge, which requires me to overwrite PYTHON_MODULE_EXTENSION for PyPy3.7, that I cannot set this variable from the command line without also setting PYBIND11_PYTHON_EXECUTABLE_LAST to the same value as Python_EXECUTABLE (on a fresh build directory).
conda-forge/openpmd-api-feedstock#86

I cannot spot the reason why in pybind11NewTools.cmake or the source of CMake for unset(.. CACHE), yet. But it seems that a passed -DPYTHON_MODULE_EXTENSION=... is unconditionally overwritten.

Reproducible example code

Any build that adds -DPython_EXECUTABLE:FILEPATH=$PYTHON -DPython_INCLUDE_DIR=$(${PYTHON} -c "from sysconfig import get_paths as gp; print(gp()['include'])") -DPYTHON_MODULE_EXTENSION=".something-else.so" as of pybind11 2.8.1 or 2.9.0 (CMake: 3.21.3 & 3.22.1).
I see this with openPMD-api 0.14.4:

• https://github.com/openPMD/openPMD-api
• Rebuild for python310 conda-forge/openpmd-api-feedstock#86

cmake -S . -B build -DopenPMD_USE_PYTHON=ON -DPython_EXECUTABLE:FILEPATH=$(which python3) -DPython_INCLUDE_DIR=$($(which python3) -c "from sysconfig import get_paths as gp; print(gp()['include'])") -DopenPMD_USE_INTERNAL_PYBIND11=OFF -DPYTHON_MODULE_EXTENSION=".something-else.so"
cmake --build build -j 4

# pybind11 2.6.1 + CMake 3.22.1
ls build/lib/python3*/site-packages/openpmd_api/
openpmd_api_cxx.something-else.so

# pybind11 2.8.1 + CMake 3.22.1 or
# pybind11 2.9.0 + CMake 3.21.3 (conda-forge)
ls build/lib/python3*/site-packages/openpmd_api/
openpmd_api_cxx.cpython-39-x86_64-linux-gnu.so

[ax3l]

Hm, not reproducible outside of conda-forge...

[ax3l]

Oh, I can reproduce with pybind11 2.8.1 (locally) and 2.9.0 (conda-forge).

pybind11 2.6.2 does not yet have the problem.

[henryiii]

Is this with the classic discovery or the new discovery? It seems you are setting both Python* and PYTHON* variables. And PYTHON_MODULE_EXTENSION was not supposed to be overridable, it is written to when you do a Python search (classic discovery: tools/FindPythonLibsNew.cmake, new discovery: tools/pybind11NewTools.cmake). Why is it failing to be automatic in the first place? (You'll notice that PYTHON_MODULE_EXTENSION is internal)

[henryiii]

Okay, I followed links and it looks like it's partially due to a bug in pypy37. :(. We could probably make this something that can be user overridable. We could use _PYTHON_MODULE_EXTENSION & PYTHON_MODULE_EXTENSION. The tricky part is we want to be able to change it we rediscover a different Python, but not if it's not being forced by the user.

Actually, we could add a PYBIND11_FORCE_MODULE_EXTENSION, that would likely be better / easier than exposing this setting, which has PYTHON in the name.

[henryiii]

(And 2.6.2 was before #3299 which is where the rediscovery support was added)

[tttapa]

I came across this issue while trying to cross-compile packages for a different architecture than the build Python. This causes PYTHON_MODULE_EXTENSION to be incorrect (it's the extension for build Python, not host Python).

The variable is unset here,

pybind11/tools/pybind11NewTools.cmake

Line 79 in 522c59c

unset(PYTHON_MODULE_EXTENSION CACHE)

which makes it impossible to fix this in e.g. the CMake Toolchain file or from the command line.

The solution I'm using now is to use find_program(python3.x-config) in the sysroot (rather than the build machine), and then query it for the extension suffix. Finally, I override the PYTHON_MODULE_EXTENSION cache variable after importing pybind11.

# pybind11 is header-only, so finding a native version is fine
find_package(pybind11 REQUIRED CONFIG CMAKE_FIND_ROOT_PATH_BOTH)

# Tweak extension suffix when cross-compiling
if (CMAKE_CROSSCOMPILING)
    if (NOT PY_BUILD_EXT_SUFFIX)
        message(STATUS "Determining Python extension suffix")
        # Find the python3.x-config script in the sysroot instead of on the
        # build system:
        find_program(PY_BUILD_Python3_CONFIG
            python${Python3_VERSION_MAJOR}.${Python3_VERSION_MINOR}-config
            ONLY_CMAKE_FIND_ROOT_PATH)
        # Report errors:
        if (NOT PY_BUILD_Python3_CONFIG)
            message(FATAL_ERROR "Unable to find python3-config."
                "\nTry manually setting PY_BUILD_EXT_SUFFIX.")
        else()
            # If we found the python3.x-config script, query it for the
            # extension suffix:
            execute_process(COMMAND ${PY_BUILD_Python3_CONFIG}
                --extension-suffix
                OUTPUT_VARIABLE PY_BUILD_EXT_SUFFIX
                ERROR_VARIABLE PY_BUILD_EXT_SUFFIX_ERR
                OUTPUT_STRIP_TRAILING_WHITESPACE
                RESULT_VARIABLE PY_BUILD_EXT_SUFFIX_RESULT)
            # Report errors:
            if (NOT PY_BUILD_EXT_SUFFIX_RESULT EQUAL 0
                OR NOT PY_BUILD_EXT_SUFFIX)
                message(FATAL_ERROR "Unable to determine extension suffix:"
                    "\n${PY_BUILD_EXT_SUFFIX}"
                    "\n${PY_BUILD_EXT_SUFFIX_ERR}"
                    "\nTry manually setting PY_BUILD_EXT_SUFFIX.")
            endif()
            # Cache the result:
            set(PY_BUILD_EXT_SUFFIX ${PY_BUILD_EXT_SUFFIX} CACHE STRING
                "The extension for Python extension modules")
        endif()
    endif()
    # Override pybind11NewTools.cmake's PYTHON_MODULE_EXTENSION variable:
    message(STATUS "Python extension suffix: ${PY_BUILD_EXT_SUFFIX}")
    set(PYTHON_MODULE_EXTENSION ${PY_BUILD_EXT_SUFFIX}
        CACHE INTERNAL "" FORCE)
endif()

This probably doesn't work when cross-compiling on Windows (because I doubt you can execute the python3.x-config shell script from a Linux sysroot on Windows).

This is also quite intrusive, because it requires patching a project's CMakeLists.txt files before configuring in order to be able to cross-compile it.