Setting up your Python Environment

When working on multiple Python projects it's common to run into issues with Python versioning, and package management. I am going to introduce two projects to help you tackle these common issues. I’m not going to take about the Conda project, simply because in my experience 90% of the time you run into significant issues with Conda and pip resolving package dependency issues. My approach here should work 100% of the time and allow you to control your Python environment fully.

Pyenv

Pyenv a the project you can use to control the Python’s version. More often than not issues can arise when moving to different versions of Python. For example, from 3.7 to 3.9 there are significant features that can be used that do not exist in older versions.

One criticism I have of Pyenv is that it's installed via a script, and in some corporate environments it can be hard to get the code installed. I am going to assume your working environment is Linux or a Mac. For windows please follow the following directions.

Install

Installing Pyenv is typically done via running the following script

curl https://pyenv.run | bash

Shell Setup

Pyenv requires the following code run in the shell prior to usage, this can be run via the .bashrc file at the initialization of the shell.

export PATH="$HOME/.pyenv/bin:$PATH"
eval "$(pyenv init --path)"
eval "$(pyenv virtualenv-init -)"

Install Python

Next, we will run the install argument with a version, this will install on the system the version of Python we requested.

pyenv install 3.9.2
pyenv versions

Activate

Once your environment is created all that is needed is to activate the environment.

pyenv global 3.9.2

IDE integration

Your IDE of choice will be able to access this version of python using the following path:

~/.pyenv/versions/3.9.2/bin/python

Pipenv

Documentation

Pipenv is a project that extends the Pip Python package manager. It's a significant improvement in many areas most of all dependency management. Once you learn how to use Pipenv, it's very easy to accomplish any task.

Activate our Python Environment

Before we do anything we will activate the Python environment that we created in the first section. Pipenv should be able to find that Python interpreter without any configuration.

pyenv global 3.9.2

Install

We will use the pip Python package manager to install pipenv, and then we will stop using pip.

pip install pipenv

Create a Pipenv Environment

Now that pipenv is installed we are going to create an example project folder, and then initialize pipenv in that folder.

mkdir my_project
cd my_project
pipenv --python 3.9.2 install
pipenv lock

Working with a Dev Environment

You can choose to have an isolated dev environment, all you need to do is add the following to the pip file above your normal [packages] section

[dev-packages]

then use the following in your CLI call

pipenv install --dev
# or
pipenv shell --dev

Pipfile & lock file

Pipenv uses two main files the Pipefile for specifying the packages for your environment. The Pipe.lock file is generated from the Pipefile with the :

pipenv lock

The lock file lists all dependencies that the project uses including sub dependencies. You should commit both files to your git repo.

The Pipfile is used to generate the lock file and contains configurations like repository and Python version.

[[source]]
#url = "your pypi url for packages"
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"
[packages]
pandas = "==0.24.2"
requests = "*"
[requires]
python_version = "3.9"

Syncing with the Lock File

Once you have a lock file you can now sync your environment with what is defined in the lock file.

pipenv sync 

If you want to manually install a package into your environment you can:

$ pipenv install requests~=1.2

This will update your Pipenv file.

Running Code

Now that our environments are set up, it's trivial to run our code all we need to do is :

pipenv run python3 my_python_app.py

Shell

If you would like to have the terminal set to the environment then you can set it by using :

pipenv shell

Updating packages

If we would like to have a list of outdated packages run the following to see all of the packages needing an upgrade:

pipenv update --outdated

You can choose to blindly update all packages :

pipenv update

You can instead update one package at a time:

pipenv update pandas

Requirements.txt

If you have a legacy requirements.txt file you can specify that file for Pipenv to initially use:

pipenv install -r requirements.txt

If you find that for some reason you need to generate a requirements.txt file then you can run :

pip freeze > requirements.txt

We have now seen how we can manged both our Python interpreter and our Python packages. It's important for any project to make sure you are controlling your environment, this increases repeatability, and reduces time wasted troubleshooting odd issues.