GeezerVison

How can you install Python libraries?

Our program depends on a third-party package

Here we have a program called name.py:

import requests
print(requests.get("https://pseudorandom.name").text.strip())

This program doesn't work right now:

$ python3 name.py
Traceback (most recent call last):
  File "/home/trey/name.py", line 1, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'

It doesn't work because it uses the requests module, which isn't included with Python. This is a popular module that's included in the requests package, that you can install from the Python Package Index by using a tool called pip which comes bundled with Python.

Installing packages using pip

We can use pip by typing python3 -m pip and then using the install command to install the requests module:

$ python3 -m pip install requests
Collecting requests
  Using cached requests-2.27.1-py2.py3-none-any.whl (63 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Installing collected packages: certifi, urllib3, idna, charset-normalizer, requests
Successfully installed certifi-2021.10.8 charset-normalizer-2.0.12 idna-3.3 requests-2.27.1 urllib3-1.26.9

Now that pip has installed the requests module, we can run our code and see that it works:

$ python3 name.py
Marie Story

But we've just done something we probably shouldn't have done: we've installed a package globally.

Python installations that came from your operating system (or from Homebrew) usually refuse to do this at all now, and print an externally-managed-environment error instead:

$ python3 -m pip install requests
error: externally-managed-environment

That's your system protecting its own Python from you. If you saw that error, don't worry: the rest of this article is about the thing you should do instead.

Avoid installing packages globally

If we ask pip to list all our packages, we'll see that there's requests as well as urllib3 and a bunch of other packages:

$ python3 -m pip list
Package            Version
------------------ ---------
argcomplete        2.0.0
certifi            2021.10.8
charset-normalizer 2.0.12
click              8.0.4
idna               3.3
packaging          21.3
pip                22.0.4
pipx               1.0.0
pyparsing          3.0.7
requests           2.27.1
setuptools         58.1.0
urllib3            1.26.9
userpath           1.8.0

But we don't know how many of these packages were dependencies of requests, or were packages that came from some other Python package we installed on our machine.

Instead of installing Python packages globally, I usually recommend installing packages into a virtual environment. This way, if we had multiple projects that needed different versions of the same package, we won't have version conflicts.

When using a separate virtual environment for each project, it's also a lot easier to figure out which code in our machine requires which dependencies to run.

Another project with a third-party dependency

Here we have a different program called make_screencast_card.py:

"""Make an image for a Python Morsels screencast."""
from PIL import Image, ImageDraw, ImageFont
import sys

[_, text] = sys.argv
font = ImageFont.truetype("OpenSans-ExtraBold.ttf", size=116)
card = Image.new("RGB", (1920, 1080), "white")
logo = Image.open("logo-horizontal-white.png")
card.paste(logo, (292, 457))
draw = ImageDraw.Draw(card)
draw.text((960, 250), text, fill="black", anchor="mm", font=font)
card.save(f"{text}.jpg")

This program doesn't work either, because it uses a module called PIL which comes bundled in a package called Pillow:

$ python3 make_screencast_card.py "Installing packages"
Traceback (most recent call last):
  File "/home/trey/screencast-cards/make_screencast_card.py", line 2, in <module>
    from PIL import Image, ImageDraw, ImageFont
ModuleNotFoundError: No module named 'PIL'

Creating a new virtual environment

Before we install pillow, we're going to make a virtual environment by using Python's venv module:

$ python3 -m venv venv

Here we're asking the venv module to make a directory called venv, which represents our virtual environment. We'll talk much more about virtual environments later.

But we need to activate our virtual environment in order to actually use it. The command to do that is different on different machines.

This is the command on my machine (it might be different on yours):

$ source venv/bin/activate

Now, I can tell I'm in an activated virtual environment because my prompt will change. It will show the name of my virtual environment just before it:

(venv) $

Installing packages into a virtual environment

Now that we've created a virtual environment and activated that environment, we can finally install pillow, by using pip as before:

(venv) $ python3 -m pip install pillow
Collecting pillow
  Using cached Pillow-9.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.3 MB)
Installing collected packages: pillow
Successfully installed pillow-9.1.0

But now when we install it, we'll be installing it into our virtual environment.

So if we run our code again, it works this time:

(venv) $ python3 make_screencast_card.py "Installing packages"

It generates a JPEG image for us:

(venv) $ ls *.jpg
'Installing packages.jpg'

But if we do a pip list, we'll see that unlike before, we don't see requests or urllib3, or any of those other packages we have installed globally. We see very little outside of just Pillow:

(venv) $ python3 -m pip list
Package    Version
---------- -------
Pillow     9.1.0
pip        22.0.4

This is because our globally-installed Python packages are isolated from the packages in our virtual environment.

Other Python package management tools

While pip and virtual environments are great, they're not the only packages for managing and installing Python packages. They're just the two that come bundled with Python.

There's also pipx, which is a popular one:

$ pipx install countdown-cli
  installed package countdown-cli 1.0.2, installed using Python 3.10.3
  These apps are now globally available
    - countdown
done! ✨ 🌟 ✨

It's used for installing global command-line tools within their own separate environment.

So you'll still have the ability to run whatever program is actually installed by pipx, but you won't clutter up your global Python package list in pip. So when you run pip list, we won't see this countdown-cli program in our list of packages.

Here are some of the other popular Python package management tools:

1. pip (in a virtual environment preferable) with a requirements.txt file (more on those another time)
2. uv manages virtual environments, dependencies, and Python versions itself
3. pipx installs Python command-line tools into globally accessible isolated environments
4. pipenv attempts to combine pip and virtual environments into one tool
5. poetry helps manage dependencies, building, and publishing to PyPI

uv has become the popular choice for whole projects, because it creates the virtual environment, resolves the dependencies, and can even install Python itself. pipx is the popular choice for global dependencies. Pipenv is a tool that attempts to bundle together pip and virtual environments into a single tool. Poetry is also a pretty popular tool for making third-party libraries because it can help you build and publish that third-party library to the Python Package Index.

Use pip and venv when installing Python packages

The tool pip (that comes bundled with Python) is the usual way to install Python packages.

But don't install packages globally if you can avoid it. It's recommended to use a virtual environment when installing Python packages (we'll talk more about those in a future screencast).

Also, if using Anaconda, you should use Conda for all of this instead. Conda is the command within the Anaconda world that replaces pip and virtual environments.