Handling project dependencies with pip

The most convenient tool to install and manage Python packages is pip. Rather than installing the packages one by one, it is possible to define a list of packages that you want to install as the contents of a text file. We can pass the text file into the pip tool, which will then handle installation of all packages in the list automatically. An added benefit to this approach is that the package list can be stored in version control. If you have gone through the Working with Docker recipe, then you have already seen this.

Generally speaking, it is ideal and often sufficient to have a single requirements file that directly matches your production environment. When changing versions or adding and removing dependencies, this can be done on a development machine and then managed through version control. It can then be as simple as switching branches to go from one set of dependencies (and associated code changes) to another.

In some cases, environments differ enough that you will need to have at least two different instances of your project: the development environment, where you create new features, and the public website environment that is usually called the production environment, in a hosted server. There might be development environments for other developers, or special tools that are needed during development but are unnecessary in production. Also, you may have a testing and staging environment in order to test the project locally and in a public website-like situation.

For good maintainability, you should be able to install the required Python modules for development, testing, staging, and production environments. Some of the modules will be shared and some of them will be specific to a subset of the environments. In this recipe, we will see how to organize the project dependencies for multiple environments and manage them with pip.