Why HTTPS Needs More Than a Handshake: One‑Way vs Mutual TLS Explained

HTTP vs HTTPS One‑Way vs Mutual TLS

We all know that an HTTP request only needs a TCP three‑way handshake to establish a connection, with data transmitted in clear text and vulnerable to man‑in‑the‑middle attacks.

Modern public websites use HTTPS, which adds end‑to‑end encryption on top of the TCP handshake, solving data‑in‑transit security and site trust issues.

In HTTPS one‑way authentication, the client validates the server’s public‑key certificate using the system’s root CA store; the server does not verify the client.

HTTPS also supports two‑way (mutual) authentication, where the client must present its own certificate—usually issued by a private CA created by the server—so the server can verify the client’s identity. This is common in banking or internal services.

1. HTTP relies on TCP three‑way handshake

1. TCP three‑way handshake:
   1. Client SYN → Server
   2. Server SYN+ACK → Client
   3. Client ACK → Server (connection established)
2. Client sends HTTP request:
   GET /index.html HTTP/1.1
   Host: www.example.com
   ...
3. Server returns HTTP response:
   HTTP/1.1 200 OK
   Content‑Type: text/html
   ...
4. (Optional) Keep connection or close it

2. HTTPS one‑way authentication (server certificate)

The server first creates a private key and a public‑key certificate signing request (CSR), submits the CSR to a CA, which returns a signed certificate. The client validates this certificate against its trusted root CA list and, if valid, proceeds with the encrypted connection.

3. HTTPS mutual authentication (client & server certificates)

Both sides present certificates. The server’s certificate is still issued by a CA, while the client’s certificate is typically issued by the server’s private CA. The handshake includes additional steps for the client to send its certificate and prove possession of the private key.

1. Client → Server: ClientHello
2. Server → Client: ServerHello (includes server certificate)
3. Client validates server certificate
4. Server requests client certificate (CertificateRequest)
5. Client → Server: ClientCertificate
6. Client → Server: PreMasterSecret (encrypted with server public key)
7. Client → Server: CertificateVerify (signs handshake messages)
8. Both compute Master Secret and switch to symmetric encryption
9. Handshake complete, data exchanged securely

How to Build a Private CA (OpenSSL script for macOS/Linux)

#!/bin/bash
# Create private CA key (2048‑bit)
openssl genrsa -out ca.key 2048
# Generate self‑signed root certificate
openssl req -x509 -new -nodes -key ca.key -sha256 -days 3650 -out ca.crt -subj "/C=CN/ST=Zhejiang/L=Hangzhou/O=MyCompany/OU=CA/CN=myca.local"
# Sign client CSR with the private CA
openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out client.crt -days 365 -sha256

echo "Client certificate issued"

Client Certificate Generation

# Generate client private key
openssl genrsa -out client.key 2048
# Create CSR for client certificate
openssl req -new -key client.key -out client.csr -subj "/C=CN/ST=Zhejiang/L=Hangzhou/O=MyCompany/OU=Dev/CN=client.local"

Common Certificate and Key File Extensions

.crt – Certificate (usually PEM, used on Unix/Linux)

.cer – Certificate (PEM or DER, common on Windows/IIS)

.pem – Certificate, public key, private key, or chain (Base64 with -----BEGIN…END-----, most universal)

.der – Certificate in binary DER format (used by Java, Windows)

.key – Private key (typically PEM encoded)

.csr – Certificate signing request (usually PEM)

.p12 / .jks – Certificate containers (PKCS#12 or Java Keystore, bundle private key and certificate)

Network Security Takeaways

Installing an untrusted root certificate enables a man‑in‑the‑middle proxy (e.g., Charles) to decrypt HTTPS traffic, which is why you should never install unknown certificates.

Public Wi‑Fi can hijack DNS, redirecting you to malicious sites that look legitimate. Always use trusted networks, verify server certificates, and avoid installing unknown CA certificates.

When using HTTPS, the data remains encrypted end‑to‑end unless a trusted root certificate is compromised.