pyenv virtualenv of miniconda environment switches underlying python version, wanted something based on 3.12, got 3.11 & 3.13

[manuel-koch]

Description

• Platform information (e.g. Ubuntu Linux 16.04): MacOS 14.7.1 (23H222) on Apple M1 Pro
• OS architecture (e.g. amd64): arm64
• pyenv version: 2.4.17
• Python version: miniconda3-3.12-24.9.2-0
• C Compiler information (e.g. gcc 7.3):

$ gcc --version
Apple clang version 16.0.0 (clang-1600.0.26.3)
Target: arm64-apple-darwin23.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Installing python interpreter using

# I want something based on Python 3.12
$ pyenv install miniconda3-3.12-24.9.2-0
# look like it installs python 3.11

$ pyenv virtualenv miniconda3-3.12-24.9.2-0 my_conda-3.12
# looks like it installs python 3.13

$ python3 -c 'import sys; import platform; print(f"Python {platform.python_version()}: {sys.executable}")'
Python 3.13.0: /Users/manuelkoch/.pyenv/versions/my_conda-3.12/bin/python3

Detailed output:

# CWD in ~/tmp/my_conda
$ pyenv install miniconda3-3.12-24.9.2-0
Installing Miniconda3-py312_24.9.2-0-MacOSX-arm64.sh...
Channels:
 - defaults
Platform: osx-arm64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /Users/manuelkoch/.pyenv/versions/miniconda3-latest

  added / updated specs:
    - conda=24.9.2
    - pip


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    certifi-2024.8.30          |  py311hca03da5_0         165 KB
    conda-24.9.2               |  py311hca03da5_0         1.2 MB
    distro-1.9.0               |  py311hca03da5_0          38 KB
    frozendict-2.4.2           |  py311hca03da5_0          38 KB
    libmamba-1.5.8             |       haeffa04_2         1.3 MB
    libmambapy-1.5.8           |  py311h1c5506f_2         282 KB
    menuinst-2.2.0             |  py311hca03da5_0         256 KB
    pip-24.2                   |  py311hca03da5_0         2.8 MB
    platformdirs-3.10.0        |  py311hca03da5_0          39 KB
    ------------------------------------------------------------
                                           Total:         6.1 MB

The following NEW packages will be INSTALLED:

  archspec           pkgs/main/noarch::archspec-0.2.3-pyhd3eb1b0_0
  distro             pkgs/main/osx-arm64::distro-1.9.0-py311hca03da5_0
  frozendict         pkgs/main/osx-arm64::frozendict-2.4.2-py311hca03da5_0
  menuinst           pkgs/main/osx-arm64::menuinst-2.2.0-py311hca03da5_0
  platformdirs       pkgs/main/osx-arm64::platformdirs-3.10.0-py311hca03da5_0

The following packages will be UPDATED:

  ca-certificates                     2023.08.22-hca03da5_0 --> 2024.9.24-hca03da5_0
  certifi                         2023.7.22-py311hca03da5_0 --> 2024.8.30-py311hca03da5_0
  conda                              23.9.0-py311hca03da5_0 --> 24.9.2-py311hca03da5_0
  conda-libmamba-so~ pkgs/main/osx-arm64::conda-libmamba-s~ --> pkgs/main/noarch::conda-libmamba-solver-24.9.0-pyhd3eb1b0_0
  libmamba                                 1.5.1-h15e39b3_0 --> 1.5.8-haeffa04_2
  libmambapy                          1.5.1-py311h1c5506f_0 --> 1.5.8-py311h1c5506f_2
  openssl                                 3.0.11-h1a28f6b_2 --> 3.0.15-h80987f9_0
  pip                                23.2.1-py311hca03da5_0 --> 24.2-py311hca03da5_0
  yaml-cpp                                 0.7.0-hc377ac9_1 --> 0.8.0-h313beb8_1



Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installed Miniconda3-py312_24.9.2-0-MacOSX-arm64.sh to /Users/manuelkoch/.pyenv/versions/miniconda3-3.12-24.9.2-0

# CWD in ~/tmp/my_conda
$ pyenv virtualenv miniconda3-3.12-24.9.2-0 my_conda-3.12
Channels:
 - defaults
Platform: osx-arm64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /Users/manuelkoch/.pyenv/versions/miniconda3-3.12-24.9.2-0/envs/my_conda-3.12

  added / updated specs:
    - python


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    libmpdec-4.0.0             |       h80987f9_0          69 KB
    pip-24.2                   |  py313hca03da5_0         2.4 MB
    python-3.13.0              |h4862095_100_cp313        13.7 MB
    python_abi-3.13            |          0_cp313           7 KB
    setuptools-75.1.0          |  py313hca03da5_0         1.9 MB
    wheel-0.44.0               |  py313hca03da5_0         139 KB
    ------------------------------------------------------------
                                           Total:        18.3 MB

The following NEW packages will be INSTALLED:

  bzip2              pkgs/main/osx-arm64::bzip2-1.0.8-h80987f9_6
  ca-certificates    pkgs/main/osx-arm64::ca-certificates-2024.9.24-hca03da5_0
  expat              pkgs/main/osx-arm64::expat-2.6.3-h313beb8_0
  libcxx             pkgs/main/osx-arm64::libcxx-14.0.6-h848a8c0_0
  libffi             pkgs/main/osx-arm64::libffi-3.4.4-hca03da5_1
  libmpdec           pkgs/main/osx-arm64::libmpdec-4.0.0-h80987f9_0
  ncurses            pkgs/main/osx-arm64::ncurses-6.4-h313beb8_0
  openssl            pkgs/main/osx-arm64::openssl-3.0.15-h80987f9_0
  pip                pkgs/main/osx-arm64::pip-24.2-py313hca03da5_0
  python             pkgs/main/osx-arm64::python-3.13.0-h4862095_100_cp313
  python_abi         pkgs/main/osx-arm64::python_abi-3.13-0_cp313
  readline           pkgs/main/osx-arm64::readline-8.2-h1a28f6b_0
  setuptools         pkgs/main/osx-arm64::setuptools-75.1.0-py313hca03da5_0
  sqlite             pkgs/main/osx-arm64::sqlite-3.45.3-h80987f9_0
  tk                 pkgs/main/osx-arm64::tk-8.6.14-h6ba3021_0
  tzdata             pkgs/main/noarch::tzdata-2024b-h04d1e81_0
  wheel              pkgs/main/osx-arm64::wheel-0.44.0-py313hca03da5_0
  xz                 pkgs/main/osx-arm64::xz-5.4.6-h80987f9_1
  zlib               pkgs/main/osx-arm64::zlib-1.2.13-h18a0788_1



Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate my_conda-3.12
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Looking in links: /var/folders/4l/pq2frs651d1ccv5rtxxmwjww0000gn/T/tmpvxan3kkn
Requirement already satisfied: pip in /Users/manuelkoch/.pyenv/versions/miniconda3-3.12-24.9.2-0/envs/my_conda-3.12/lib/python3.13/site-packages (24.2)

# CWD in ~/tmp/my_conda
$ python3 -c 'import sys; import platform; print(f"Python {platform.python_version()}: {sys.executable}")'
Python 3.13.0: /Users/manuelkoch/.pyenv/versions/my_conda-3.12/bin/python3

[manuel-koch]

Maybe my expectation of pyenv virtualenv with miniconda is plain wrong and it is expected to not work ?
My naive assumption is that I can create a virtualenv of any installed python interpreter.

Looks like the virtualenv is using some other previously installed miniconda ( miniconda3-latest in my case ).

# CWD in ~/tmp/my_conda
$ conda info

/Users/manuelkoch/.pyenv/versions/miniconda3-latest/lib/python3.11/site-packages/conda/base/context.py:198: FutureWarning: Adding 'defaults' to channel list implicitly is deprecated and will be removed in 25.3.

To remove this warning, please choose a default channel explicitly with conda's regular configuration system, e.g. by adding 'defaults' to the list of channels:

  conda config --add channels defaults

For more information see https://docs.conda.io/projects/conda/en/stable/user-guide/configuration/use-condarc.html

  deprecated.topic(

     active environment : /Users/manuelkoch/.pyenv/versions/miniconda3-3.12-24.9.2-0/envs/my_conda-3.12
    active env location : /Users/manuelkoch/.pyenv/versions/miniconda3-3.12-24.9.2-0/envs/my_conda-3.12
            shell level : 0
       user config file : /Users/manuelkoch/.condarc
 populated config files :
          conda version : 24.9.2
    conda-build version : not installed
         python version : 3.11.5.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=m1
                          __conda=24.9.2=0
                          __osx=14.7.1=0
                          __unix=0=0
       base environment : /Users/manuelkoch/.pyenv/versions/miniconda3-latest  (writable)
      conda av data dir : /Users/manuelkoch/.pyenv/versions/miniconda3-latest/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/osx-arm64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-arm64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /Users/manuelkoch/.pyenv/versions/miniconda3-latest/pkgs
                          /Users/manuelkoch/.conda/pkgs
       envs directories : /Users/manuelkoch/.pyenv/versions/miniconda3-latest/envs
                          /Users/manuelkoch/.conda/envs
               platform : osx-arm64
             user-agent : conda/24.9.2 requests/2.31.0 CPython/3.11.5 Darwin/23.6.0 OSX/14.7.1 solver/libmamba conda-libmamba-solver/24.9.0 libmambapy/1.5.8
                UID:GID : 501:20
             netrc file : None
           offline mode : False

[native-api]

It's more like conda uses some metadata cache in a shared location.

The immediate reason is the conda command pyenv-virtualenv runs: conda create <...> --yes <..> python
Which makes it use the latest Python in its metadata.

We should probably read the version of the Python package in the base environment and instruct it to install that.