From 10afe05cfd47b358e72e4dce78eef37e8fbbcdd9 Mon Sep 17 00:00:00 2001 From: Chad Smith Date: Sun, 19 Jan 2020 14:18:30 -0800 Subject: [PATCH 1/3] simplify virtual environment instructions --- docs/contents.rst.inc | 2 +- docs/dev/pip-virtualenv.rst | 2 +- docs/dev/virtualenvs.rst | 171 ++++++++---------------------------- 3 files changed, 37 insertions(+), 138 deletions(-) diff --git a/docs/contents.rst.inc b/docs/contents.rst.inc index 29b076197..ae12203b6 100644 --- a/docs/contents.rst.inc +++ b/docs/contents.rst.inc @@ -22,7 +22,7 @@ New to Python? Let's properly setup up your Python environment: starting/install/linux -- Using Virtualenvs with Pipenv: +- Package Installation and Project Isolation: .. toctree:: :maxdepth: 2 diff --git a/docs/dev/pip-virtualenv.rst b/docs/dev/pip-virtualenv.rst index ee223078e..95c0bac7c 100644 --- a/docs/dev/pip-virtualenv.rst +++ b/docs/dev/pip-virtualenv.rst @@ -1,6 +1,6 @@ .. _pip-virtualenv: -Further Configuration of pip and Virtualenv +Further Configuration of pip and virtual environments =========================================== .. image:: /_static/photos/34018732105_f0e6758859_k_d.jpg diff --git a/docs/dev/virtualenvs.rst b/docs/dev/virtualenvs.rst index 326b6e288..e0d966967 100644 --- a/docs/dev/virtualenvs.rst +++ b/docs/dev/virtualenvs.rst @@ -1,7 +1,7 @@ .. _virtualenvironments-ref: -Pipenv & Virtual Environments -============================= +Package Installation and Project Isolation +========================================== .. image:: /_static/photos/35294660055_42c02b2316_k_d.jpg @@ -65,6 +65,9 @@ using your OS package manager, you may have to `install pip `_ is a tool to create -isolated Python environments. virtualenv creates a folder which contains all the -necessary executables to use the packages that a Python project would need. +If you choose not to use Pipenv or it does not fit your needs, you can +use the `venv `_ tool directly to create +isolated Python environments. The ``venv`` module is part of Python's standard library, +and was introduced in Python 3.3. It creates a folder which contains all the necessary +executables to use the packages that a Python project would need. It can be used standalone, in place of Pipenv. -Install virtualenv via pip: - -.. code-block:: console - - $ pip install virtualenv - -Test your installation: - -.. code-block:: console - - $ virtualenv --version - Basic Usage ----------- @@ -226,13 +219,12 @@ Basic Usage .. code-block:: console $ cd project_folder - $ virtualenv venv + $ python -m venv venv -``virtualenv venv`` will create a folder in the current directory which will -contain the Python executable files, and a copy of the ``pip`` library which you +``python -m venv venv`` will create a folder in the current directory which will +contain the Python executable files, and a copy of the ``pip`` application which you can use to install other packages. The name of the virtual environment (in this -case, it was ``venv``) can be anything; omitting the name will place the files -in the current directory instead. +case, it was ``venv``) can be anything. .. note:: 'venv' is the general convention used globally. As it is readily available in ignore files (eg: .gitignore') @@ -240,31 +232,34 @@ in the current directory instead. This creates a copy of Python in whichever directory you ran the command in, placing it in a folder named :file:`venv`. -You can also use the Python interpreter of your choice (like -``python2.7``). +You can also use the Python interpreter of your choice (like ``python3.8``). .. code-block:: console - $ virtualenv -p /usr/bin/python2.7 venv + $ python3.8 -m venv venv -or change the interpreter globally with an env variable in ``~/.bashrc``: -.. code-block:: console +2. To begin using the virtual environment, you can either invoke the virtual environment's executables +directly, or activate it. - $ export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python2.7 +To use the virtual environment's Python executable directly, run +``venv/bin/python``; to use its pip executable, ``venv/bin/pip``. -2. To begin using the virtual environment, it needs to be activated: +Alternatively, you can "activate" +the environment so you can just type ``python`` or ``pip`` and it will automatically use the +executables in the virtual environment (in this case, at ``venv/bin``). .. code-block:: console $ source venv/bin/activate -The name of the current virtual environment will now appear on the left of +Now, the name of the current virtual environment will appear on the left of the prompt (e.g. ``(venv)Your-Computer:project_folder UserName$``) to let you know that it's active. From now on, any package that you install using pip will be placed in the ``venv`` folder, isolated from the global Python installation. -For Windows, the same command mentioned in step 1 can be used to create a virtual environment. However, activating the environment requires a slightly different command. +For Windows, the same command mentioned in step 1 can be used to create a virtual environment. +However, activating the environment requires a slightly different command. Assuming that you are in your project directory: @@ -286,25 +281,26 @@ Install packages using the ``pip`` command: $ deactivate This puts you back to the system's default Python interpreter with all its -installed libraries. +installed libraries. This is not necessary if you invoked the executables directly. To delete a virtual environment, just delete its folder. (In this case, it would be ``rm -rf venv``.) After a while, though, you might end up with a lot of virtual environments -littered across your system, and it's possible you'll forget their names or -where they were placed. +littered across your system. It's possible you'll forget their names or +where they were placed, so try to follow a convention across your projects. .. note:: - Python has included venv module from version 3.3. For more details: `venv `_. + The ``venv`` module is part of Python's standard library in Python3.3+. + Older versions of Python can use the + 3rd party package `virtualenv `_. Other Notes ----------- -Running ``virtualenv`` with the option ``--no-site-packages`` will not +Running ``python -m venv`` with the option ``--no-site-packages`` will not include the packages that are installed globally. This can be useful for keeping the package list clean in case it needs to be accessed later. -[This is the default behavior for ``virtualenv`` 1.7 and later.] In order to keep your environment consistent, it's a good idea to "freeze" the current state of the environment packages. To do this, run: @@ -330,103 +326,6 @@ and across developers. Lastly, remember to exclude the virtual environment folder from source control by adding it to the ignore list (see :ref:`Version Control Ignores`). -.. _virtualenvwrapper-ref: - -virtualenvwrapper ------------------ - -`virtualenvwrapper `_ -provides a set of commands which makes working with virtual environments much -more pleasant. It also places all your virtual environments in one place. - -To install (make sure **virtualenv** is already installed): - -.. code-block:: console - - $ pip install virtualenvwrapper - $ export WORKON_HOME=~/Envs - $ source /usr/local/bin/virtualenvwrapper.sh - -(`Full virtualenvwrapper install instructions `_.) - -For Windows, you can use the `virtualenvwrapper-win `_. - -To install (make sure **virtualenv** is already installed): - -.. code-block:: console - - $ pip install virtualenvwrapper-win - -In Windows, the default path for WORKON_HOME is %USERPROFILE%\\Envs - -Basic Usage -~~~~~~~~~~~ - -1. Create a virtual environment: - -.. code-block:: console - - $ mkvirtualenv project_folder - -This creates the :file:`project_folder` folder inside :file:`~/Envs`. - -2. Work on a virtual environment: - -.. code-block:: console - - $ workon project_folder - -Alternatively, you can make a project, which creates the virtual environment, -and also a project directory inside ``$WORKON_HOME``, which is ``cd``-ed into -when you ``workon project_folder``. - -.. code-block:: console - - $ mkproject project_folder - -**virtualenvwrapper** provides tab-completion on environment names. It really -helps when you have a lot of environments and have trouble remembering their -names. - -``workon`` also deactivates whatever environment you are currently in, so you -can quickly switch between environments. - -3. Deactivating is still the same: - -.. code-block:: console - - $ deactivate - -4. To delete: - -.. code-block:: console - - $ rmvirtualenv venv - -Other useful commands -~~~~~~~~~~~~~~~~~~~~~ - -``lsvirtualenv`` - List all of the environments. - -``cdvirtualenv`` - Navigate into the directory of the currently activated virtual environment, - so you can browse its :file:`site-packages`, for example. - -``cdsitepackages`` - Like the above, but directly into :file:`site-packages` directory. - -``lssitepackages`` - Shows contents of :file:`site-packages` directory. - -`Full list of virtualenvwrapper commands `_. - -virtualenv-burrito ------------------- - -With `virtualenv-burrito `_, you -can have a working virtualenv + virtualenvwrapper environment in a single command. - direnv ------- When you ``cd`` into a directory containing a :file:`.env`, `direnv `_ @@ -438,4 +337,4 @@ Install it on Mac OS X using ``brew``: $ brew install direnv -On Linux follow the instructions at `direnv.net ` +On Linux follow the instructions at `direnv.net `_ \ No newline at end of file From 6e200a1cf2e0b8ba53bad31d5a52d1a8518d098e Mon Sep 17 00:00:00 2001 From: Chad Smith Date: Mon, 17 Feb 2020 18:02:39 -0800 Subject: [PATCH 2/3] updates --- docs/dev/virtualenvs.rst | 38 +++++++++++++++++++++++++++++++++++--- 1 file changed, 35 insertions(+), 3 deletions(-) diff --git a/docs/dev/virtualenvs.rst b/docs/dev/virtualenvs.rst index e0d966967..9d3736fb8 100644 --- a/docs/dev/virtualenvs.rst +++ b/docs/dev/virtualenvs.rst @@ -201,7 +201,7 @@ Congratulations, you now know how to install and use Python packages! ✨ 🍰 Project Isolation with Virtual Environments -======================= +=========================================== If you choose not to use Pipenv or it does not fit your needs, you can use the `venv `_ tool directly to create @@ -221,6 +221,12 @@ Basic Usage $ cd project_folder $ python -m venv venv +On Windows use this command: + +.. code-block:: console + + $ py -m venv venv + ``python -m venv venv`` will create a folder in the current directory which will contain the Python executable files, and a copy of the ``pip`` application which you can use to install other packages. The name of the virtual environment (in this @@ -232,13 +238,20 @@ case, it was ``venv``) can be anything. This creates a copy of Python in whichever directory you ran the command in, placing it in a folder named :file:`venv`. -You can also use the Python interpreter of your choice (like ``python3.8``). +You can also use the Python interpreter of your choice, like ``python3.8``. Sometimes ``python`` will still point to a Python 2 interpreter, so you can do this instead to be sure you are using the right Python version. .. code-block:: console $ python3.8 -m venv venv +On Windows use this command: + +.. code-block:: console + + $ py -3.8 -m venv venv + + 2. To begin using the virtual environment, you can either invoke the virtual environment's executables directly, or activate it. @@ -326,8 +339,27 @@ and across developers. Lastly, remember to exclude the virtual environment folder from source control by adding it to the ignore list (see :ref:`Version Control Ignores`). + +Other Tools +----------- + +tox and nox +~~~~~~~~~~~ + +`tox `_ and `nox `_ are widely used command-line tools that automate environment setup and task execution. They are used to run tests across multiple Python versions, among other things. They do this by reading a configuration file, either ``tox.ini`` for tox, or ``noxfile.py`` for nox. + +For example, if you have unit tests that you want to run with Python 3.6, 3.7, and 3.8, you can use one of these tools to automate virtual environment creation and test execution with all three Python versions. You can also run specific tasks like running a lint check, or publishing a new version of your package. + +The main difference between the two tools are ``tox`` uses a custom file format for configuration, while ``nox`` uses a standard Python file for configuration. + + +pipx +~~~~ +`pipx `_ is a tool to install system-wide command line tools, each to their own individual environment. Unlike ``pip`` which installs all packages to the same environment, ``pipx`` isolates tools in their own virtual environment, and exposes the command-line tools to your shell. ``pipx`` is used for installing command-line tools, similar to ``brew`` or ``apt``, but for Python applications. It's not used to intall libraries. + + direnv -------- +~~~~~~ When you ``cd`` into a directory containing a :file:`.env`, `direnv `_ automagically activates the environment. From 7a54a652a169a59a46411c9ad4f1f4150191c685 Mon Sep 17 00:00:00 2001 From: Chad Smith Date: Mon, 17 Feb 2020 20:25:18 -0800 Subject: [PATCH 3/3] add pip-tools; update site-packages info --- docs/dev/virtualenvs.rst | 45 ++++++++++++++++++++++++---------------- 1 file changed, 27 insertions(+), 18 deletions(-) diff --git a/docs/dev/virtualenvs.rst b/docs/dev/virtualenvs.rst index 9d3736fb8..e6d879f99 100644 --- a/docs/dev/virtualenvs.rst +++ b/docs/dev/virtualenvs.rst @@ -311,9 +311,9 @@ where they were placed, so try to follow a convention across your projects. Other Notes ----------- -Running ``python -m venv`` with the option ``--no-site-packages`` will not -include the packages that are installed globally. This can be useful -for keeping the package list clean in case it needs to be accessed later. +Running ``python -m venv`` with the option ``--system-site-packages`` will include +the packages that are installed globally. Usually you do not want to do this so +the package list remains clean in case it needs to be accessed later. In order to keep your environment consistent, it's a good idea to "freeze" the current state of the environment packages. To do this, run: @@ -343,20 +343,7 @@ control by adding it to the ignore list (see :ref:`Version Control Ignores`_ and `nox `_ are widely used command-line tools that automate environment setup and task execution. They are used to run tests across multiple Python versions, among other things. They do this by reading a configuration file, either ``tox.ini`` for tox, or ``noxfile.py`` for nox. - -For example, if you have unit tests that you want to run with Python 3.6, 3.7, and 3.8, you can use one of these tools to automate virtual environment creation and test execution with all three Python versions. You can also run specific tasks like running a lint check, or publishing a new version of your package. - -The main difference between the two tools are ``tox`` uses a custom file format for configuration, while ``nox`` uses a standard Python file for configuration. - - -pipx -~~~~ -`pipx `_ is a tool to install system-wide command line tools, each to their own individual environment. Unlike ``pip`` which installs all packages to the same environment, ``pipx`` isolates tools in their own virtual environment, and exposes the command-line tools to your shell. ``pipx`` is used for installing command-line tools, similar to ``brew`` or ``apt``, but for Python applications. It's not used to intall libraries. - +There are many tools to complement usage of pip and virtual environments. Here are some useful ones we like. direnv ~~~~~~ @@ -369,4 +356,26 @@ Install it on Mac OS X using ``brew``: $ brew install direnv -On Linux follow the instructions at `direnv.net `_ \ No newline at end of file +On Linux follow the instructions at `direnv.net `_ + +pip-tools +~~~~~~~~~ +`pip-tools `_ is a suite of two tools that complement pip and virtual environments. It has similar functionaly to pipenv, and in fact pipenv uses pip-tools in its implementation. However, compared to pipenv, pip-tools lets you have a little more control over how and when operations are performed. + +It does two things: +1.) Generate a complete dependency list (lockfile, or ``requirements.txt`` file) from abstract dependencies. It does this with a full dependency resolver, which pip does not currently have, and can optionally generate the lockfile with hashes. +2.) Synchronize a virtual environment to exactly match a requirements lockfile. + +pipx +~~~~ +`pipx `_ is a tool to install system-wide command line tools, each to their own individual environment. Unlike ``pip`` which installs all packages to the same environment, ``pipx`` isolates tools in their own virtual environment, and exposes the command-line tools to your shell. ``pipx`` is used for installing command-line tools, similar to ``brew`` or ``apt``, but for Python applications. It's not used to install libraries. + + +tox and nox +~~~~~~~~~~~ + +`tox `_ and `nox `_ are widely used command-line tools that automate environment setup and task execution. tox and nox are often used to run tests across multiple Python versions, among other things. They do this by reading a configuration file, either ``tox.ini`` for tox, or ``noxfile.py`` for nox. + +For example, if you have unit tests that you want to run with Python 3.6, 3.7, and 3.8, you can use one of these tools. It will create three virtual environments, one for each Python version, then install necessary dependencies, and finally run tests in each environment. You can also run specific tasks like running a lint check, or publishing a new version of your package. + +The main difference between the two tools are ``tox`` uses a custom file format for configuration, while ``nox`` uses a standard Python file for configuration.