Understanding How the Timeout Parameter Is Implemented in the Python Requests Library

In everyday development, making HTTP requests is ubiquitous, and the requests library is one of the most convenient tools for Python. Besides the common HTTP verbs (GET, POST, DELETE, PUT), the timeout parameter is especially useful because it prevents a request from blocking indefinitely.

When a timeout is set, a request such as requests.get('http://google.com', timeout=1) can raise a requests.exceptions.ConnectTimeout if the connection cannot be established within the specified period.

To understand how this works, we need to follow the call chain for a simple GET request. The high‑level requests.get function forwards to requests.api.get , which calls requests.api.request . This creates a Session object and ultimately invokes Session.request . The session then obtains an appropriate HTTPAdapter and calls its send method, which delegates to urllib3’s pool manager.

The pool manager ( urllib3.PoolManager ) maintains a cache of connection pools identified by a pool_key . The key is built from fields such as scheme, host, port, timeout, retries, and several SSL‑related options. If a request’s parameters match an existing key, the cached pool is reused, saving the cost of creating a new connection.

Inside the pool, the HTTPConnectionPool class handles the actual socket operations. Its private methods _get_conn , _new_conn , and _make_request manage acquiring a connection from the pool, creating a new one when necessary, and sending the HTTP request. The timeout object is prepared by self._get_timeout and assigned to the underlying socket before the request is sent.

The socket used by the connection is created via socket.create_connection , which resolves hostnames for both IPv4 and IPv6 and attempts to connect to each address until one succeeds. The timeout argument passed to create_connection becomes the socket’s own timeout, so the subsequent connect and recv operations respect the user‑specified limit.

In summary, the timeout parameter in requests is implemented by propagating the value down to urllib3’s pool manager, constructing or reusing a connection pool, creating a socket with socket.create_connection , and finally setting the socket’s timeout via conn.sock.settimeout . This layered design makes the timeout behavior reliable while keeping the high‑level API simple.

# requests/api.py

def get(url, params=None, **kwargs):
    kwargs.setdefault('allow_redirects', True)
    return request('get', url, params=params, **kwargs)

def request(method, url, **kwargs):
    with sessions.Session() as session:
        return session.request(method=method, url=url, **kwargs)

# requests/sessions.py
class Session(SessionRedirectMixin):
    def send(self, request, **kwargs):
        # Get the appropriate adapter to use
        adapter = self.get_adapter(url=request.url)
        # Send the request
        r = adapter.send(request, **kwargs)
        ...

# urllib3/connectionpool.py
class HTTPConnectionPool(ConnectionPool, RequestMethods):
    ConnectionCls = HTTPConnection
    def _get_conn(self, timeout=None):
        conn = None
        try:
            conn = self.pool.get(block=self.block, timeout=timeout)
        except AttributeError:
            raise ClosedPoolError(self, "Pool is closed.")
        except queue.Empty:
            if self.block:
                raise EmptyPoolError(self, "Pool reached maximum size and no more connections are allowed.")
            pass
        if conn and is_connection_dropped(conn):
            conn.close()
            if getattr(conn, "auto_open", 1) == 0:
                conn = None
        return conn or self._new_conn()

    def _new_conn(self):
        self.num_connections += 1
        conn = self.ConnectionCls(host=self.host, port=self.port,
                                  timeout=self.timeout.connect_timeout,
                                  strict=self.strict, **self.conn_kw)
        return conn

    def _make_request(self, conn, method, url, timeout=_Default, chunked=False, **httplib_request_kw):
        timeout_obj = self._get_timeout(timeout)
        timeout_obj.start_connect()
        conn.timeout = timeout_obj.connect_timeout
        if chunked:
            conn.request_chunked(method, url, **httplib_request_kw)
        else:
            conn.request(method, url, **httplib_request_kw)
        # Set read timeout on the socket
        read_timeout = timeout_obj.read_timeout
        if getattr(conn, "sock", None):
            if read_timeout == 0:
                raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % read_timeout)
            if read_timeout is Timeout.DEFAULT_TIMEOUT:
                conn.sock.settimeout(socket.getdefaulttimeout())
            else:
                conn.sock.settimeout(read_timeout)
        # Receive the response
        try:
            httplib_response = conn.getresponse(buffering=True)
        except (SocketTimeout, BaseSSLError, SocketError) as e:
            self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
            raise
        return httplib_response

# httplib.py (simplified)
class HTTPConnection:
    def __init__(self, host, port=None, strict=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT, source_address=None):
        self.timeout = timeout
        self.source_address = source_address
        self.sock = None
        self.host, self.port = self._get_hostport(host, port)
        self._create_connection = socket.create_connection

    def connect(self):
        self.sock = self._create_connection((self.host, self.port), self.timeout, self.source_address)

    def send(self, data):
        if self.sock is None:
            if self.auto_open:
                self.connect()
            else:
                raise NotConnected()
        self.sock.sendall(data)