Skip to content

[doc] Refresh the venv introduction documentation, and correct the statement about VIRTUAL_ENV (GH-98350) #98350

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 19, 2022
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
140 changes: 87 additions & 53 deletions Doc/library/venv.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,23 @@

--------------

.. _venv-def:

The :mod:`venv` module provides support for creating lightweight "virtual
environments" with their own site directories, optionally isolated from system
site directories. Each virtual environment has its own Python binary (which
matches the version of the binary that was used to create this environment) and
can have its own independent set of installed Python packages in its site
directories.
environments" with their own independent set of Python packages installed in
their site directories. A virtual environment is created on top of an existing
Python installation, known as the virtual environment's "base" Python, and may
be optionally isolated from the base's site packages, meaning
that packages installed in the base environment will not be accessible from
the virtual environment.

A virtual environment is therefore a powerful and convenient concept which
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps insert a blank line here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure thing. Done in 1ba4797

allows quick, and relatively light-weight, Python environment creation.

When used from within a virtual environment, common installation tools such as
setuptools_ and pip_ will install Python packages
into the virtual environment's site directory without needing to be told to do
so explicitly.

See :pep:`405` for more information about Python virtual environments.

Expand All @@ -36,54 +47,66 @@ Creating virtual environments

.. include:: /using/venv-create.inc


.. _venv-def:

.. note:: A virtual environment is a Python environment such that the Python
interpreter, libraries and scripts installed into it are isolated from those
installed in other virtual environments, and (by default) any libraries
installed in a "system" Python, i.e., one which is installed as part of your
operating system.

A virtual environment is a directory tree which contains Python executable
files and other files which indicate that it is a virtual environment.

Common installation tools such as setuptools_ and pip_ work as
expected with virtual environments. In other words, when a virtual
environment is active, they install Python packages into the virtual
environment without needing to be told to do so explicitly.

When a virtual environment is active (i.e., the virtual environment's Python
interpreter is running), the attributes :attr:`sys.prefix` and
:attr:`sys.exec_prefix` point to the base directory of the virtual
environment, whereas :attr:`sys.base_prefix` and
:attr:`sys.base_exec_prefix` point to the non-virtual environment Python
installation which was used to create the virtual environment. If a virtual
environment is not active, then :attr:`sys.prefix` is the same as
:attr:`sys.base_prefix` and :attr:`sys.exec_prefix` is the same as
:attr:`sys.base_exec_prefix` (they all point to a non-virtual environment
Python installation).

When a virtual environment is active, any options that change the
installation path will be ignored from all ``setuptools`` configuration
files to prevent projects being inadvertently installed outside of the
virtual environment.

When working in a command shell, users can make a virtual environment active
by running an ``activate`` script in the virtual environment's executables
directory (the precise filename and command to use the file is
shell-dependent), which prepends the virtual environment's directory for
executables to the ``PATH`` environment variable for the running shell. There
should be no need in other circumstances to activate a virtual
environment; scripts installed into virtual environments have a "shebang"
line which points to the virtual environment's Python interpreter. This means
that the script will run with that interpreter regardless of the value of
``PATH``. On Windows, "shebang" line processing is supported if you have the
Python Launcher for Windows installed (this was added to Python in 3.3 - see
:pep:`397` for more details). Thus, double-clicking an installed script in a
Windows Explorer window should run the script with the correct interpreter
without there needing to be any reference to its virtual environment in
``PATH``.
Further venv detail
-------------------

When a virtual environment is running, the attributes :attr:`sys.prefix` and
:attr:`sys.exec_prefix` point to the prefix directory of the virtual
environment, whereas :attr:`sys.base_prefix` and
:attr:`sys.base_exec_prefix` point to the non-virtual environment Python
installation which was used to create the virtual environment (known as the
virtual environment's base environment). It is sufficient to check
``sys.prefix == sys.base_prefix`` to determine if the current interpreter is
a virtual environment.

A virtual environment may be "activated" using a script in the virtual
environment's binary directory. This will prepend the virtual environment's
binary directory to your path, so that running "python" will invoke the virtual
environment's Python interpreter and you can run
installed scripts without having to use their full path. The invocation of the
activation script is platform-specific (``<venv>`` must be replaced by the path
of the directory containing the virtual environment):

+-------------+-----------------+-----------------------------------------+
| Platform | Shell | Command to activate virtual environment |
+=============+=================+=========================================+
| POSIX | bash/zsh | $ source <venv>/bin/activate |
+-------------+-----------------+-----------------------------------------+
| | fish | $ source <venv>/bin/activate.fish |
+-------------+-----------------+-----------------------------------------+
| | csh/tcsh | $ source <venv>/bin/activate.csh |
+-------------+-----------------+-----------------------------------------+
| | PowerShell Core | $ <venv>/bin/Activate.ps1 |
+-------------+-----------------+-----------------------------------------+
| Windows | cmd.exe | C:\\> <venv>\\Scripts\\activate.bat |
+-------------+-----------------+-----------------------------------------+
| | PowerShell | PS C:\\> <venv>\\Scripts\\Activate.ps1 |
+-------------+-----------------+-----------------------------------------+

.. versionadded:: 3.4
``fish`` and ``csh`` activation scripts.

.. versionadded:: 3.8
PowerShell activation scripts installed under POSIX for PowerShell Core
support.

You don't specifically *need* to activate an environment, and all scripts
installed in a virtual environment should be runnable without activating it.
In order to achieve this, scripts installed into virtual environments have
a "shebang" line which points to the virtual environment's Python interpreter.
This means that the script will run with that interpreter regardless of the
value of ``PATH``. On Windows, "shebang" line processing is supported if
you have the Python Launcher for Windows installed (this was added to Python
in 3.3 - see :pep:`397` for more details). Thus, double-clicking an installed
script in a Windows Explorer window should run the script with the correct
interpreter without there needing to be any reference to its virtual
environment in :envvar:`PATH`.

When a virtual environment has been activated, the :envvar:`VIRTUAL_ENV`
environment variable is set to the path of the virtual environment. Since it
is not a requirement to explicitly activate a virtual environment in order
to use it, the :envvar:`VIRTUAL_ENV` environment variable cannot be relied
upon to determine whether a Python environment is virtual or not.

.. warning:: Because scripts installed in environments should not expect the
environment to be activated, their shebang lines contain the absolute paths
Expand All @@ -99,6 +122,17 @@ Creating virtual environments
environment in its new location. Otherwise, software installed into the
environment may not work as expected.

You can deactivate a virtual environment by typing "deactivate" in your shell.
The exact mechanism is platform-specific and is an internal implementation
detail (typically a script or shell function will be used).


.. note:: When a virtual environment is active, any options that change the
installation path will be ignored from all :mod:`distutils` configuration
files to prevent projects being inadvertently installed outside of the
virtual environment.


.. _venv-api:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
.. note:: When a virtual environment is active, any options that change the
installation path will be ignored from all :mod:`distutils` configuration
files to prevent projects being inadvertently installed outside of the
virtual environment.
.. _venv-api:
.. _venv-api:

Distutils has been officially deprecated since Python 3.10 (the oldest backportable version) and removed in this version, 3.12 (thus the link should not resolve). Furthermore, use of these config files and use of distutils as an installer directly has been deprecated for perhaps a decade. Therefore, this should just be removed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The one thing that would make me hesitate with this one is that it would be good to get these improvements into older Python docs. The VIRTUAL_ENV environment variable documentation is wrong since Python 3.8 (by a commit that was backported that far 2 years ago).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But that snippet could be left out even in documentation for older Python versions, as it doesn't really add all that much information - for example, it's not clear which options might be meant in

any options that change the installation path

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The one thing that would make me hesitate with this one is that it would be good to get these improvements into older Python docs.

Right, which is why I mentioned it has been deprecated since the oldest backportable version, Python 3.10, and discouraged for perhaps a decade (never mind "distutils configuration files"—I don't know if I've ever run into any modern mention of anything actually using them these days; I'm involved with a fair bit of packaging stuff and I'd never even heard of them until now). Ergo, this is likely to cause more confusion than anything, if even packaging experts like @vsajip aren't even sure what it is intended to mean. And if worst comes to worse, this one change can always be dropped in a backport (though it seems very unnecessary).

The VIRTUAL_ENV environment variable documentation is wrong since Python 3.8 (by a commit that was backported that far 2 years ago).

I'm not entirely sure what you intend to mean here, sorry.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not entirely sure what you intend to mean here, sorry.

Simply stating that there is a factually incorrect statement in the documenation relating to VIRTUAL_ENV environment use (it claims: "This can be used to check if one is running inside a virtual environment.", which is wrong), which it would be great to remove from as many backported versions as possible to avoid incorrect usage in the wild.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, gotcha 👍 I was confused and wasn't sure the connection.


API
Expand Down
42 changes: 0 additions & 42 deletions Doc/using/venv-create.inc
Original file line number Diff line number Diff line change
Expand Up @@ -105,45 +105,3 @@ Multiple paths can be given to ``venv``, in which case an identical virtual
environment will be created, according to the given options, at each provided
path.

Once a virtual environment has been created, it can be "activated" using a
script in the virtual environment's binary directory. The invocation of the
script is platform-specific (`<venv>` must be replaced by the path of the
directory containing the virtual environment):

+-------------+-----------------+-----------------------------------------+
| Platform | Shell | Command to activate virtual environment |
+=============+=================+=========================================+
| POSIX | bash/zsh | $ source <venv>/bin/activate |
+-------------+-----------------+-----------------------------------------+
| | fish | $ source <venv>/bin/activate.fish |
+-------------+-----------------+-----------------------------------------+
| | csh/tcsh | $ source <venv>/bin/activate.csh |
+-------------+-----------------+-----------------------------------------+
| | PowerShell Core | $ <venv>/bin/Activate.ps1 |
+-------------+-----------------+-----------------------------------------+
| Windows | cmd.exe | C:\\> <venv>\\Scripts\\activate.bat |
+-------------+-----------------+-----------------------------------------+
| | PowerShell | PS C:\\> <venv>\\Scripts\\Activate.ps1 |
+-------------+-----------------+-----------------------------------------+

When a virtual environment is active, the :envvar:`VIRTUAL_ENV` environment
variable is set to the path of the virtual environment. This can be used to
check if one is running inside a virtual environment.

You don't specifically *need* to activate an environment; activation just
prepends the virtual environment's binary directory to your path, so that
"python" invokes the virtual environment's Python interpreter and you can run
installed scripts without having to use their full path. However, all scripts
installed in a virtual environment should be runnable without activating it,
and run with the virtual environment's Python automatically.

You can deactivate a virtual environment by typing "deactivate" in your shell.
The exact mechanism is platform-specific and is an internal implementation
detail (typically a script or shell function will be used).

.. versionadded:: 3.4
``fish`` and ``csh`` activation scripts.

.. versionadded:: 3.8
PowerShell activation scripts installed under POSIX for PowerShell Core
support.