Python Virtual Environment 101

What is a Virtual Environment?

A Python Virtual Environment, or more commonly called "virtualenv", is a tool that is included in the recent Python distributions.
Its purpose is to create a virtual and isolated Python environment, that can be modified at will and then simply discarded.
"Python environment" refers to the Python binaries, the standard Python libraries, as well as any third party library that is later installed.
When installing a Python distribution from https://www.python.org/ or from a package manager, this installation is referred as the "global environment" or "global installation".

Why is a Virtual Environment needed?

Except for the simplest cases, most Python projects require some dependencies to third party libraries - third party in the sense that they are not included in the Python official distribution.
Typically, these libraries are installed in the "site-packages" folder of the global Python installation, so they can be imported in any Python module.
This approach has a few caveats:

• The global Python installation might be installed in a restricted place, which require admin privileges to write into
• When developing several projects simultaneously, it quickly becomes difficult to manage the dependencies of each project. For example, project A depends on libX version 1.0, and project B depends on the same libX, but version 2.0.
• Similarly, running different Python applications simultaneously can be difficult, because of conflicting dependencies.

The problem is that, in the Python platform, dependencies are installed on a global level, and not on a "project" level - unlike in the Java ecosystem for example.

Virtual environments can solve these problems:

• They can be created anywhere in the file system, to a place where write access is granted. A full replica of a Python installation will be created, and the system environment variables will be updated to point to this new environment.
• Each Python project can use its own virtual environment, that contains only the project's required dependencies.
• Similarly, each Python application can use its own environment.

How to set-up a Virtual Environment?

$ virtualenv my-env 

This will create a folder called my-env, containing a fresh Python installation, with a few pre-installed packages, such as pip or setuptools.

To activate the virtual environment, this command can be used:

$ source my_env/bin/activate

Or, if using Windows:

> my_env\Scripts\activate

Activating a virtaulenv simply change a few environment variables, including PATH, to use the Python interpreter in the virtualenv, rather than the globally installed interpreter.

What are the drawbacks? 

When working on multiple projects at the same time, using different virtual environments can require a lot of maintenance and organization, it is quite easy to create environments a bit everywhere, then forget about them, and end-up with virtual environments all over the place.
Also for example, working on five projects, that needs to be supported in four different versions of Python, will lead to the creation of twenty virtual environments! This is quite hard to maintain.
Why is one environment per Python version required? Because the tests need to run (and pass) against each Python version, and the builds must work on each version as well.
Luckily, there are tools in the Python ecosystem that can help on these pain points.

What are the alternatives?

There isn't any other tool doing a similar job as virtualenv, but there are some that can help overcome some of the limitations, or that are just helpful.

• virtualenvwrapper provides a higher level API on top of virtualenv. It keeps all virtual environments in a single place, allows listing of all the environments, and easy switching between them.
• direnv automatically activates an environment, when entering (cd) its directory
• tox is a testing and packaging tool, that eases and standardizes the testing process by creating automatically virtual environments for each version of Python the project needs to be compatible with

Conclusion

Virtual environments are a must-have in any non-trivial Python project. They help to isolate the project dependencies from the global Python installation, and from the dependencies of other projects. They give more confidence to the developer, as he/she is not afraid to screw the Python environment. If anything goes wrong, or just to start with a fresh environment, the folder can just be deleted and recreated.
For larger projects though, additional tooling such as tox or virtualenvwrapper is often helpful, as maintaining too many virtual environments can be a real hassle.

References

• virtualenv official documentation 
• The excellent Hitchhiker's Guide to Python