A practical guide to proxies in Python requests: the proxies argument, proxy authentication, HTTPS and SOCKS5 proxies, sessions, environment variables, rotating proxies, sticky sessions and the errors you will meet. Every script in this repository runs as is.

1. Set a proxy for one request
2. Proxy with username and password
3. SOCKS5 proxy
4. Proxy for a whole session
5. Proxy from environment variables
6. Rotating proxies
7. Sticky sessions: keep the same IP
8. Errors and how to fix them

pip install -r requirements.txt

requests takes a proxies dictionary. The key is the scheme of the website you request, the value is the proxy URL:

import requests

proxies = {
    "http": "http://203.0.113.10:8080",   # used for http:// websites
    "https": "http://203.0.113.10:8080",  # used for https:// websites
}
r = requests.get("https://ipinfo.io/json", proxies=proxies, timeout=30)
print(r.json()["ip"])

A common mistake is writing "https": "https://..." because the site is HTTPS. The value describes how you talk to the proxy, and most proxies, including authenticated residential ones, are plain http:// even for HTTPS websites: requests opens a CONNECT tunnel and TLS runs inside it.

Always set timeout. Without it a dead proxy can block your script forever.

Script: 01_basic_proxy.py

Put the credentials in the proxy URL:

proxy = "http://USERNAME:[email protected]:12286"
r = requests.get("https://ipinfo.io/json", proxies={"http": proxy, "https": proxy}, timeout=30)

If the password contains @, :, / or #, URL-encode it, otherwise requests splits the URL in the wrong place:

from urllib.parse import quote

proxy = f"http://{quote(username, safe='')}:{quote(password, safe='')}@resi.fusionproxy.net:12286"

Script: 02_authenticated_proxy.py

SOCKS support needs one extra package (it is in requirements.txt):

pip install "requests[socks]"

proxy = "socks5h://USERNAME:[email protected]:5390"
r = requests.get("https://ipinfo.io/json", proxies={"http": proxy, "https": proxy}, timeout=30)

Use socks5h://, not socks5://. With socks5h the proxy resolves the host name, so the DNS lookup happens in the proxy's country and does not leak from your machine.

Script: 03_socks5_proxy.py

A Session reuses connections and keeps cookies. Set the proxy once:

session = requests.Session()
session.proxies.update({"http": proxy, "https": proxy})
session.get("https://example.com/login")
session.get("https://example.com/account")

Keep in mind that a session reuses the open tunnel. On a rotating proxy, requests that reuse a connection keep the IP of that connection, so use a sticky session (section 7) when the IP must stay the same on purpose, and a new session when it must change.

requests reads HTTP_PROXY, HTTPS_PROXY and NO_PROXY when no proxy is passed:

export HTTPS_PROXY="http://USERNAME:[email protected]:12286"
export HTTP_PROXY="$HTTPS_PROXY"
export NO_PROXY="localhost,127.0.0.1"
python your_script.py

To ignore them in one session, set session.trust_env = False. This also explains the classic surprise "my script uses a proxy I never set": check env | grep -i proxy.

There are two ways to get a new IP on every request.

A rotating gateway. One endpoint, a new residential IP per connection. Nothing to manage in your code:

proxy = "http://USERNAME-country-us:[email protected]:12286"
for _ in range(3):
    print(requests.get("https://ipinfo.io/ip", proxies={"http": proxy, "https": proxy}, timeout=30).text)
# three different US IPs

A list of proxies. Pick one per request and move on when it fails:

import random

def get(url, proxy_list, attempts=5):
    for proxy in random.sample(proxy_list, min(attempts, len(proxy_list))):
        try:
            return requests.get(url, proxies={"http": proxy, "https": proxy}, timeout=10)
        except requests.RequestException:
            continue
    raise RuntimeError("all proxies failed")

Script: 04_rotate_proxy_list.py loads a list (by default the public monosans/proxy-list) and retries through it.

Logins, carts and multi-step forms break when the IP changes in the middle. With a residential gateway you keep one IP by adding a session ID to the username:

session_user = "USERNAME-country-us-sessid-4821937-sessTime-600s"  # same IP for 10 minutes
proxy = f"http://{session_user}:[email protected]:12286"

Same sessid = same IP; a new sessid = a new IP. The format is explained in the endpoint generator guide.

Script: 05_sticky_session.py

To test a proxy without writing code, use the free online proxy checker or the proxy-checker command-line tool.

export FUSION_USER="your_username"   # from the Fusion Proxy dashboard
export FUSION_PASS="your_password"
python 02_authenticated_proxy.py

Don't have proxy credentials? Fusion Proxy residential proxies start at $2.95/GB, and there is a free trial. 01_basic_proxy.py and 04_rotate_proxy_list.py work with any proxy.

• nodejs-proxy: the same in Node.js (fetch and axios)
• curl-proxy: proxies with cURL
• python-requests-retry: retries and backoff in requests
• python-web-scraping: a complete scraper with requests and BeautifulSoup

MIT, see LICENSE.