How to Make HTTP Requests Through Tor with Python

Why Use Tor for HTTP Requests?

Picture this: you’re in the middle of a data scraping project, and suddenly, your IP address is blacklisted. Or perhaps you’re working on a privacy-first application where user anonymity is non-negotiable. Tor (The Onion Router) is the perfect solution for both scenarios. It routes your internet traffic through a decentralized network of servers (nodes), obscuring its origin and making it exceptionally challenging to trace.

Tor is not just a tool for bypassing restrictions; it’s a cornerstone of privacy on the internet. From journalists working in oppressive regimes to developers building secure applications, Tor is widely used for anonymity and bypassing censorship. It allows you to mask your IP address, avoid surveillance, and access region-restricted content.

However, integrating Tor into your Python projects isn’t as straightforward as flipping a switch. It requires careful configuration and a solid understanding of the tools involved. Today, I’ll guide you through two robust methods to make HTTP requests via Tor: using the requests library with a SOCKS5 proxy and leveraging the stem library for advanced control. By the end, you’ll have all the tools you need to bring the power of Tor into your Python workflows.

Getting Tor Up and Running

Before we dive into Python code, we need to ensure that Tor is installed and running on your system. Here’s a quick rundown for different platforms:

• Linux: Install Tor via your package manager, e.g., sudo apt install tor. Start the service with sudo service tor start.
• Mac: Use Homebrew: brew install tor. Then start it with brew services start tor.
• Windows: Download the Tor Expert Bundle from the official Tor Project website, extract it, and run the tor.exe executable.

By default, Tor runs a SOCKS5 proxy on 127.0.0.1:9050. This is the endpoint we’ll leverage to route HTTP requests through the Tor network.

Method 1: Using the requests Library with a SOCKS5 Proxy

The simplest way to integrate Tor into your Python project is by configuring the requests library to use Tor’s SOCKS5 proxy. This approach is lightweight and straightforward but offers limited control over Tor’s features.

Step 1: Install Required Libraries

First, ensure you have the necessary dependencies installed. The requests library needs an additional component for SOCKS support:

pip install requests[socks]

Step 2: Configure a Tor-Enabled Session

Create a reusable function to configure a requests session that routes traffic through Tor:

import requests

def get_tor_session():
    session = requests.Session()
    session.proxies = {
        'http': 'socks5h://127.0.0.1:9050',
        'https': 'socks5h://127.0.0.1:9050'
    }
    return session

The socks5h protocol ensures that DNS lookups are performed through Tor, adding an extra layer of privacy.