Understanding HTTPS: Encryption Methods, Handshake Process, and HttpClient Implementation

Background

HTTP transmits data in clear text, making all traffic vulnerable. To secure communication there are two main ideas: (1) client and server exchange keys and encrypt data themselves, or (2) let the protocol handle encryption.

Encryption Methods

HTTPS solves the security problem by encrypting data at the SSL/TLS layer. Encryption can be symmetric (e.g., DES, AES) or asymmetric (e.g., RSA). Symmetric encryption is fast but requires key distribution (n·(n‑1) keys for n entities). Asymmetric encryption uses a public/private key pair, reduces the number of keys to 2·n, but is slower.

Hybrid encryption combines both: asymmetric encryption is used to exchange a symmetric key, then the symmetric key encrypts the bulk data, which is the approach used by HTTPS.

HTTPS Handshake

Client and server exchange certificates and verify identities (servers rarely verify client certificates).

Negotiate protocol version and cipher suite.

Negotiate a symmetric key using asymmetric encryption.

Encrypt HTTP payload with the negotiated symmetric key.

TCP transports the encrypted data transparently.

HttpClient Support for HTTPS

4.1 Obtaining SSL Connection Factory and Hostname Verifier

HttpClient builds SSL support starting from HttpClientBuilder. The relevant source code is:

public CloseableHttpClient build() {
    // ... omitted code ...
    HttpClientConnectionManager connManagerCopy = this.connManager;
    if (connManagerCopy == null) {
        LayeredConnectionSocketFactory sslSocketFactoryCopy = this.sslSocketFactory;
        if (sslSocketFactoryCopy == null) {
            final String[] supportedProtocols = systemProperties ? split(System.getProperty("https.protocols")) : null;
            final String[] supportedCipherSuites = systemProperties ? split(System.getProperty("https.cipherSuites")) : null;
            HostnameVerifier hostnameVerifierCopy = this.hostnameVerifier;
            if (hostnameVerifierCopy == null) {
                hostnameVerifierCopy = new DefaultHostnameVerifier(publicSuffixMatcherCopy);
            }
            if (sslContext != null) {
                sslSocketFactoryCopy = new SSLConnectionSocketFactory(sslContext, supportedProtocols, supportedCipherSuites, hostnameVerifierCopy);
            } else {
                if (systemProperties) {
                    sslSocketFactoryCopy = new SSLConnectionSocketFactory((SSLSocketFactory) SSLSocketFactory.getDefault(), supportedProtocols, supportedCipherSuites, hostnameVerifierCopy);
                } else {
                    sslSocketFactoryCopy = new SSLConnectionSocketFactory(SSLContexts.createDefault(), hostnameVerifierCopy);
                }
            }
        }
        final PoolingHttpClientConnectionManager poolingmgr = new PoolingHttpClientConnectionManager(
                RegistryBuilder.<ConnectionSocketFactory>create()
                        .register("http", PlainConnectionSocketFactory.getSocketFactory())
                        .register("https", sslSocketFactoryCopy)
                        .build(),
                null, null, dnsResolver, connTimeToLive,
                connTimeToLiveTimeUnit != null ? connTimeToLiveTimeUnit : TimeUnit.MILLISECONDS);
        // ... omitted code ...
    }
}

The code registers two socket factories (http and https) in the connection pool, allowing HttpClient to create SSL connections when needed.

4.2 Obtaining an SSL Connection

When a connection is requested, HttpClient looks up the appropriate ConnectionSocketFactory and creates a socket. The core logic is:

public void connect(final ManagedHttpClientConnection conn, final HttpHost host,
                    final InetSocketAddress localAddress, final int connectTimeout,
                    final SocketConfig socketConfig, final HttpContext context) throws IOException {
    final Lookup<ConnectionSocketFactory> registry = getSocketFactoryRegistry(context);
    final ConnectionSocketFactory sf = registry.lookup(host.getSchemeName());
    if (sf == null) {
        throw new UnsupportedSchemeException(host.getSchemeName() + " protocol is not supported");
    }
    // Resolve addresses, create socket, set TCP options, bind, then:
    sock = sf.connectSocket(connectTimeout, sock, host, remoteAddress, localAddress, context);
    conn.bind(sock);
    // ... omitted code ...
}

If the created socket is an instance of SSLSocket, a TLS handshake is performed and the hostname is verified.

SSLConnectionSocketFactory Implementation

@Override
public Socket connectSocket(final int connectTimeout, final Socket socket,
                         final HttpHost host, final InetSocketAddress remoteAddress,
                         final InetSocketAddress localAddress, final HttpContext context) throws IOException {
    Args.notNull(host, "HTTP host");
    Args.notNull(remoteAddress, "Remote address");
    final Socket sock = socket != null ? socket : createSocket(context);
    if (localAddress != null) {
        sock.bind(localAddress);
    }
    // Set timeout, connect, then if SSL:
    if (sock instanceof SSLSocket) {
        final SSLSocket sslsock = (SSLSocket) sock;
        sslsock.startHandshake();
        verifyHostname(sslsock, host.getHostName());
        return sock;
    } else {
        return createLayeredSocket(sock, host.getHostName(), remoteAddress.getPort(), context);
    }
}

@Override
public Socket createLayeredSocket(final Socket socket, final String target,
                                 final int port, final HttpContext context) throws IOException {
    final SSLSocket sslsock = (SSLSocket) this.socketfactory.createSocket(
            socket, target, port, true);
    // Apply protocols and cipher suites if configured
    if (supportedProtocols != null) {
        sslsock.setEnabledProtocols(supportedProtocols);
    } else {
        // Remove SSLv2/SSLv3
        final String[] allProtocols = sslsock.getEnabledProtocols();
        final List<String> enabled = new ArrayList<>(allProtocols.length);
        for (final String p : allProtocols) {
            if (!p.startsWith("SSL")) {
                enabled.add(p);
            }
        }
        if (!enabled.isEmpty()) {
            sslsock.setEnabledProtocols(enabled.toArray(new String[enabled.size()]));
        }
    }
    if (supportedCipherSuites != null) {
        sslsock.setEnabledCipherSuites(supportedCipherSuites);
    }
    sslsock.startHandshake();
    verifyHostname(sslsock, target);
    return sslsock;
}

This class creates a normal socket, upgrades it to an SSLSocket, applies the configured protocols and cipher suites, performs the TLS handshake, and finally verifies the server certificate against the requested hostname.

Summary

HTTPS is the secure version of HTTP, adding encryption at the transport layer with some CPU overhead.

The handshake uses asymmetric encryption to exchange a symmetric key, after which data is encrypted symmetrically.

Applications may still apply additional encryption or signing on top of HTTPS.

Apache HttpClient registers separate socket factories for http and https during the build process.

HTTPS connections are created by first establishing a plain socket, then performing a TLS handshake and hostname verification.

The actual cryptographic operations are performed by the JDK’s SSL implementation, not by HttpClient itself.