Setting up a python environment

You python environment is the ecosystem of modules that you have installed. If you have used pip to install modules then pip freeze will give you a list of all the currently installed modules and their versions. If you used conda (anaconda) to install modules then conda list will do the same. If you use a combination of the two then you are likely heading toward the following problem:

If you have ever sent a python script to a collaborator only for them to say that it doesn’t work on their machine, it’s likely that they have a different set of modules installed, or that they have a different version of the modules installed. It may feel like installing all the modules at once would help out. But different software may depend on different versions of the same modules which would require you to uninstall/reinstall different versions of modules depending on what you are working on.

A better solution is to install multiple different python environments – one for each piece of software that you wish to run. To do this we will work with virtual environments.

Creating a new virtual environment

With the introduction of python 3, there is now a new module called venv which handles the creation of new virtual environments. See the python documentation for a full breakdown. The short version is that we can create a new virtual environment simply via:

python3 -m venv [--prompt PROMPT] ENV_DIR 

where PROMPT is the name of the virtual environment, and ENV_DIR is the location in which all the files for this environment will be stored. A nice place to store your ENV_DIR is in your home directory.

Once you have created a virtual environment you can activate it via:

source ENV_DIR/bin/activate

I find it nice to have a descriptive ENV_DIR such as .env-jupyter or .env-py3-myproject (where the ‘.’ indicates a hidden directory in linux) but a short PROMPT such as jpy, or py3-myproj. The reason for this is that once your project is activated, your terminal will have PROMPT added to the start of your command line. Since my PROMPT and ENV_DIR are different I then add an alias in my ~/.bash_aliases file to link them:

alias jpy='source ~/.env-jupyter/bin/activate'

Creating a new virtual environment is easy, and if you get it wrong, you can just delete the ENV_DIR. The other nice thing is that if you are on a Linux or OSX machine your system version of python (which is used by the OS) can remain untouched, and won’t cause your system to become flaky when you totally mess up your python installs.

If you use Anaconda to install/manage your python modules, then you can simply navigate to the ‘environments’ tab, and create a new environment with the click of a button.

Now that we have created a new python environment we need to install some things into it.