test_venv fails from within venv

[hrimov]

Bug report

Bug description:

The test_venv tests were too strict about executable names, causing failures when running the test suite from within a virtual environment created from the built Python. Rather than enforcing specific executable names, it makes more sense to make the tests more flexible to support testing in both contexts - directly from build and from within a venv. What do you think about it?

# build
./configure && make -j

# create venv
./python.exe -m venv .venv
source .venv/bin/activate

# this will fail
python -m test test_venv

Also reproduced on Windows

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-130486: Fix test_venv fails from within venv #130487

[vsajip]

Why is it necessary to support running test_venv in a venv created from a built Python?

[hrimov]

Why is it necessary to support running test_venv in a venv created from a built Python?

I believe it would be logically correct, as test_venv is the only tests module that fails from within a venv. And as I found in the devguide (there might be more places), working/running tests from within a venv is actually usual practice - for example here: https://devguide.python.org/testing/coverage/#install-coverage

[vsajip]

When you say "built Python", do you mean in a source build as opposed to "installed Python"? Does test_venv fail in the latter case?

[hrimov]

When you say "built Python", do you mean in a source build as opposed to "installed Python"? Does test_venv fail in the latter case?

I mean source build. I haven't tested with "installed" Python though (do you mean like pyenv Pythons?).

[vsajip]

No, I mean a Python which you get when you run "make install" after running "make". I'm not sure it's necessary to support a source build in this specific way.

[hrimov]

No, I mean a Python which you get when you run "make install" after running "make". I'm not sure it's necessary to support a source build in this specific way.

I just tested with make install and it fails there too. So this isn't specific to source builds - the test seems too strict about paths and executable names in general.
Would you like me to share the detailed test results or adjust PR to make it working?

[vsajip]

Share the results, please.

[hrimov]

So, without the fix proposed in PR, with installed build it will give such results for from within venv:

make install

/usr/local/bin/python3.14 -m venv .venv

source .venv/bin/activate

python -m test test_venv -v 
== CPython 3.14.0a5+ (heads/main:0ff1611574, Feb 24 2025, 08:21:57) [Clang 16.0.0 (clang-1600.0.26.4)]
== macOS-15.1-arm64-arm-64bit-Mach-O little-endian
== Python build: debug
== cwd: /private/var/folders/ps/655flyt53cd2hdq5ntbr24s00000gn/T/test_python_worker_38660æ
== CPU count: 8
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 3622426512
0:00:00 load avg: 2.82 Run 1 test sequentially in a single process
0:00:00 load avg: 2.82 [1/1] test_venv
test_activate_shell_script_has_no_dos_newlines (test.test_venv.BasicTest.test_activate_shell_script_has_no_dos_newlines)
Test that the `activate` shell script contains no CR LF. ... ok
test_cli_with_scm_ignore_files (test.test_venv.BasicTest.test_cli_with_scm_ignore_files)
Test that default SCM ignore files are created by default via the CLI. ... ok
test_cli_without_scm_ignore_files (test.test_venv.BasicTest.test_cli_without_scm_ignore_files)
Test that ``--without-scm-ignore-files`` doesn't create SCM ignore files. ... ok
test_config_file_command_key (test.test_venv.BasicTest.test_config_file_command_key) ... ok
test_create_scm_ignore_files_empty (test.test_venv.BasicTest.test_create_scm_ignore_files_empty)
Test that no default ignore files are created when ``scm_ignore_files`` ... ok
test_create_scm_ignore_files_multiple (test.test_venv.BasicTest.test_create_scm_ignore_files_multiple)
Test that ``scm_ignore_files`` can work with multiple SCMs. ... ok
test_deactivate_with_strict_bash_opts (test.test_venv.BasicTest.test_deactivate_with_strict_bash_opts) ... ok
test_defaults_with_pathlike (test.test_venv.BasicTest.test_defaults_with_pathlike)
Test the create function with default arguments and a path-like path. ... ok
test_defaults_with_str_path (test.test_venv.BasicTest.test_defaults_with_str_path)
Test the create function with default arguments and a str path. ... ok
test_executable (test.test_venv.BasicTest.test_executable)
Test that the sys.executable value is as expected. ... ok
test_executable_symlinks (test.test_venv.BasicTest.test_executable_symlinks)
Test that the sys.executable value is as expected. ... ok
test_failed_symlink (test.test_venv.BasicTest.test_failed_symlink)
Test handling of failed symlinks on Windows. ... skipped 'symlinks on Windows'
test_isolation (test.test_venv.BasicTest.test_isolation)
Test isolation from system site-packages ... ok
test_macos_env (test.test_venv.BasicTest.test_macos_env) ... ok
test_multiprocessing (test.test_venv.BasicTest.test_multiprocessing)
Test that the multiprocessing is able to spawn. ... ok
test_multiprocessing_recursion (test.test_venv.BasicTest.test_multiprocessing_recursion)
Test that the multiprocessing is able to spawn itself ... ok
test_overwrite_existing (test.test_venv.BasicTest.test_overwrite_existing)
Test creating environment in an existing directory. ... ok
test_pathsep_error (test.test_venv.BasicTest.test_pathsep_error)
Test that venv creation fails when the target directory contains ... ok
test_prefixes (test.test_venv.BasicTest.test_prefixes)
Test that the prefix values are as expected. ... ok
test_prompt (test.test_venv.BasicTest.test_prompt) ... ok
test_scm_ignore_files_git (test.test_venv.BasicTest.test_scm_ignore_files_git)
Test that a .gitignore file is created when "git" is specified. ... ok
test_special_chars_bash (test.test_venv.BasicTest.test_special_chars_bash)
Test that the template strings are quoted properly (bash) ... ok
test_special_chars_csh (test.test_venv.BasicTest.test_special_chars_csh)
Test that the template strings are quoted properly (csh) ... ok
test_special_chars_windows (test.test_venv.BasicTest.test_special_chars_windows)
Test that the template strings are quoted properly on Windows ... skipped 'only relevant on Windows'
test_symlinking (test.test_venv.BasicTest.test_symlinking)
Test symlinking works as expected ... ok
test_sysconfig (test.test_venv.BasicTest.test_sysconfig)
Test that the sysconfig functions work in a virtual environment. ... 
  test_sysconfig (test.test_venv.BasicTest.test_sysconfig) [is_python_build()]
Test that the sysconfig functions work in a virtual environment. ... FAIL
  test_sysconfig (test.test_venv.BasicTest.test_sysconfig) [get_makefile_filename()]
Test that the sysconfig functions work in a virtual environment. ... FAIL
  test_sysconfig (test.test_venv.BasicTest.test_sysconfig) [get_config_h_filename()]
Test that the sysconfig functions work in a virtual environment. ... FAIL
test_sysconfig_symlinks (test.test_venv.BasicTest.test_sysconfig_symlinks)
Test that the sysconfig functions work in a virtual environment. ... 
  test_sysconfig_symlinks (test.test_venv.BasicTest.test_sysconfig_symlinks) [is_python_build()]
Test that the sysconfig functions work in a virtual environment. ... FAIL
  test_sysconfig_symlinks (test.test_venv.BasicTest.test_sysconfig_symlinks) [get_makefile_filename()]
Test that the sysconfig functions work in a virtual environment. ... FAIL
  test_sysconfig_symlinks (test.test_venv.BasicTest.test_sysconfig_symlinks) [get_config_h_filename()]
Test that the sysconfig functions work in a virtual environment. ... FAIL
test_unicode_in_batch_file (test.test_venv.BasicTest.test_unicode_in_batch_file)
Test handling of Unicode paths ... skipped 'only relevant on Windows'
test_unoverwritable_fails (test.test_venv.BasicTest.test_unoverwritable_fails) ... ok
test_upgrade (test.test_venv.BasicTest.test_upgrade)
Test upgrading an existing environment directory. ... ok
test_upgrade_dependencies (test.test_venv.BasicTest.test_upgrade_dependencies) ... FAIL
test_venv_same_path (test.test_venv.BasicTest.test_venv_same_path) ... ok
test_venvwlauncher (test.test_venv.BasicTest.test_venvwlauncher)
Test that the GUI launcher runs the GUI python. ... skipped 'only relevant on Windows'
test_zippath_from_non_installed_posix (test.test_venv.BasicTest.test_zippath_from_non_installed_posix)
Test that when create venv from non-installed python, the zip path ... ERROR
test_devnull (test.test_venv.EnsurePipTest.test_devnull) ... ok
test_explicit_no_pip (test.test_venv.EnsurePipTest.test_explicit_no_pip) ... ok
test_no_pip_by_default (test.test_venv.EnsurePipTest.test_no_pip_by_default) ... ok
test_with_pip (test.test_venv.EnsurePipTest.test_with_pip) ... skipped "resource 'cpu' is not enabled"

======================================================================
ERROR: test_zippath_from_non_installed_posix (test.test_venv.BasicTest.test_zippath_from_non_installed_posix)
Test that when create venv from non-installed python, the zip path
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.14/test/test_venv.py", line 748, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env=child_env)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/subprocess.py", line 414, in check_call
    retcode = call(*popenargs, **kwargs)
  File "/usr/local/lib/python3.14/subprocess.py", line 395, in call
    with Popen(*popenargs, **kwargs) as p:
         ~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/subprocess.py", line 1035, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                        pass_fds, cwd, env,
                        ^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
                        gid, gids, uid, umask,
                        ^^^^^^^^^^^^^^^^^^^^^^
                        start_new_session, process_group)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/subprocess.py", line 1964, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/private/var/folders/ps/655flyt53cd2hdq5ntbr24s00000gn/T/tmprektutbc/bin/python.exe'

======================================================================
FAIL: test_sysconfig (test.test_venv.BasicTest.test_sysconfig) [is_python_build()]
Test that the sysconfig functions work in a virtual environment.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.14/test/test_venv.py", line 302, in test_sysconfig
    self.assertEqual(out.strip(), expected, err)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'True' != 'False'
- True
+ False
 : 

======================================================================
FAIL: test_sysconfig (test.test_venv.BasicTest.test_sysconfig) [get_makefile_filename()]
Test that the sysconfig functions work in a virtual environment.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.14/test/test_venv.py", line 302, in test_sysconfig
    self.assertEqual(out.strip(), expected, err)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '/Users/andrew/Desktop/pets/cpython/Makefile' != '/usr/local/lib/python3.14/config-3.14d-darwin/Makefile'
- /Users/andrew/Desktop/pets/cpython/Makefile
+ /usr/local/lib/python3.14/config-3.14d-darwin/Makefile
 : 

======================================================================
FAIL: test_sysconfig (test.test_venv.BasicTest.test_sysconfig) [get_config_h_filename()]
Test that the sysconfig functions work in a virtual environment.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.14/test/test_venv.py", line 302, in test_sysconfig
    self.assertEqual(out.strip(), expected, err)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '/Users/andrew/Desktop/pets/cpython/pyconfig.h' != '/usr/local/include/python3.14d/pyconfig.h'
- /Users/andrew/Desktop/pets/cpython/pyconfig.h
+ /usr/local/include/python3.14d/pyconfig.h
 : 

======================================================================
FAIL: test_sysconfig_symlinks (test.test_venv.BasicTest.test_sysconfig_symlinks) [is_python_build()]
Test that the sysconfig functions work in a virtual environment.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.14/test/test_venv.py", line 336, in test_sysconfig_symlinks
    self.assertEqual(out.strip(), expected, err)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'True' != 'False'
- True
+ False
 : 

======================================================================
FAIL: test_sysconfig_symlinks (test.test_venv.BasicTest.test_sysconfig_symlinks) [get_makefile_filename()]
Test that the sysconfig functions work in a virtual environment.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.14/test/test_venv.py", line 336, in test_sysconfig_symlinks
    self.assertEqual(out.strip(), expected, err)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '/Users/andrew/Desktop/pets/cpython/Makefile' != '/usr/local/lib/python3.14/config-3.14d-darwin/Makefile'
- /Users/andrew/Desktop/pets/cpython/Makefile
+ /usr/local/lib/python3.14/config-3.14d-darwin/Makefile
 : 

======================================================================
FAIL: test_sysconfig_symlinks (test.test_venv.BasicTest.test_sysconfig_symlinks) [get_config_h_filename()]
Test that the sysconfig functions work in a virtual environment.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.14/test/test_venv.py", line 336, in test_sysconfig_symlinks
    self.assertEqual(out.strip(), expected, err)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '/Users/andrew/Desktop/pets/cpython/pyconfig.h' != '/usr/local/include/python3.14d/pyconfig.h'
- /Users/andrew/Desktop/pets/cpython/pyconfig.h
+ /usr/local/include/python3.14d/pyconfig.h
 : 

======================================================================
FAIL: test_upgrade_dependencies (test.test_venv.BasicTest.test_upgrade_dependencies)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.14/test/test_venv.py", line 260, in test_upgrade_dependencies
    builder.upgrade_dependencies(fake_context)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/venv/__init__.py", line 601, in upgrade_dependencies
    self._call_new_python(context, '-m', 'pip', 'install', '--upgrade',
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                          *CORE_VENV_DEPS)
                          ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/venv/__init__.py", line 454, in _call_new_python
    subprocess.check_output(args, **kwargs)
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/test/test_venv.py", line 246, in pip_cmd_checker
    self.assertEqual(
    ~~~~~~~~~~~~~~~~^
        cmd,
        ^^^^
    ...<7 lines>...
        ]
        ^
    )
    ^
AssertionError: Lists differ: ['/va[55 chars]yk/bin/python.exe', '-m', 'pip', 'install', '--upgrade', 'pip'] != ['/va[55 chars]yk/bin/python', '-m', 'pip', 'install', '--upgrade', 'pip']

First differing element 0:
'/var/folders/ps/655flyt53cd2hdq5ntbr24s00000gn/T/tmpk50okyyk/bin/python.exe'
'/var/folders/ps/655flyt53cd2hdq5ntbr24s00000gn/T/tmpk50okyyk/bin/python'

- ['/var/folders/ps/655flyt53cd2hdq5ntbr24s00000gn/T/tmpk50okyyk/bin/python.exe',
?                                                                          ----

+ ['/var/folders/ps/655flyt53cd2hdq5ntbr24s00000gn/T/tmpk50okyyk/bin/python',
   '-m',
   'pip',
   'install',
   '--upgrade',
   'pip']

----------------------------------------------------------------------
Ran 38 tests in 6.465s

FAILED (failures=7, errors=1, skipped=5)
test test_venv failed
0:00:06 load avg: 2.77 [1/1/1] test_venv failed (1 error, 7 failures)

== Tests result: FAILURE ==

1 test failed:
    test_venv

Total duration: 6.6 sec
Total tests: run=38 failures=7 skipped=5
Total test files: run=1/1 failed=1
Result: FAILURE

But I found out, that attached PR actually make these tests also passing (I run make install from the fix-branch, created venv and its okay)

[sobolevn]

My use-case to have .venv built with source ./python.exe is that I use pyperf and other tools (brurb, cherry-picker) during development.

It would be nice if all tests can pass with this mode, in my opinion.
But, if this does not complicate tests for all other cases.