Demystifying SSL/TLS: A Complete Guide to Handshake, Keys, and Security

1. SSL/TLS Overview

1.1 Overall Structure

SSL sits between the HTTP protocol and TCP as an optional layer, providing security for data transmission over the Internet.

SSL : Secure Socket Layer, originally developed by Netscape, now at version 3.0, used for authentication and encrypted data transfer between web browsers and servers. It consists of the SSL Record Protocol (provides encapsulation, compression, encryption) and the SSL Handshake Protocol (handles authentication, algorithm negotiation, key exchange).

TLS : Transport Layer Security, the successor to SSL 3.0 (often called SSL 3.1). It also has a Record Protocol and a Handshake Protocol, built on top of a reliable transport such as TCP.

SSL/TLS provides three main services: authenticating users and servers, encrypting data to prevent eavesdropping, and maintaining data integrity.

1.2 Differences Between TLS and SSL

Version numbers differ: TLS 1.0 uses version number SSLv3.1.

MAC algorithms: TLS uses HMAC (RFC‑2104) while SSLv3.0 uses a similar algorithm with different padding handling.

PRF: TLS introduces a pseudo‑random function (PRF) for key expansion.

Alert codes: TLS adds many new alert codes (e.g., decryption_failed, record_overflow, unknown_ca, access_denied).

Cipher suites and client certificates: TLS drops support for Fortezza key exchange and certain client certificate mechanisms.

Certificate_verify and finished messages: minor differences in MD5/SHA‑1 hash inputs, but security is comparable.

Master secret calculation: TLS and SSLv3.0 compute the master secret differently.

Padding: TLS allows any multiple of the block size up to 255 bytes, reducing length‑based attacks.

TLS Major Enhancements

More secure MAC algorithms.

Stricter alert handling.

Clearer definition of “gray‑area” specifications.

TLS Security Improvements

Uses HMAC for message authentication, which is stronger than SSLv3.0’s MAC.

Enhanced PRF using two hash algorithms for resilience.

Improved finished‑message verification based on PRF and HMAC.

Consistent certificate handling.

More specific alert messages.

2. Key Agreement – TLS Handshake

The TLS protocol consists of a Handshake Protocol (for key negotiation) and a Record Protocol (defining data format). Asymmetric encryption is used for the key exchange, after which symmetric encryption secures the data.

SSL/TLS by default authenticates the server; client authentication is optional.

2.1 ClientHello

The client sends the list of supported cipher suites, a random number (used later to derive the master secret), the protocol version, and supported compression methods.

2.2 ServerHello

The server replies with its chosen protocol version, a random number, the selected cipher suite, and its certificate (which contains the server’s public key). It may also send a Server Key Exchange and request a client certificate.

2.3 Client Response (Certificate Verify)

Client Key Exchange

The client encrypts a PreMaster secret with the server’s public key and sends it to the server.

Certificate Verify

The client validates the server’s certificate, then sends the encrypted PreMaster secret, an indication that subsequent messages will be encrypted, and a Finished message containing a hash of all previous handshake data.

ChangeCipherSpec

This single‑byte message tells the server that the client will now use the negotiated cipher suite and session secret for encryption.

2.4 Server Finish

The server decrypts the PreMaster secret, derives the same session secret, sends its own ChangeCipherSpec, and a Finished message to confirm the handshake succeeded.

2.5 Secrets

PreMaster Secret

Generated by the client (using RSA, Diffie‑Hellman, etc.) and encrypted with the server’s public key. It, together with both random numbers, is used to derive the Master secret.

Master Secret

A series of hash values derived from the PreMaster secret and the two random numbers; it forms part of the key material for encryption and MAC.

Session Secret (Key)

Derived from the Master secret; used as the write MAC key and encryption key for both client‑to‑server and server‑to‑client directions.

2.6 Application Data Transfer

After the handshake, each application record is prefixed with a MAC, then encrypted with the appropriate write encryption key. The receiver decrypts with the corresponding key and verifies the MAC.

2.7 Summary

The client initiates a TLS handshake with a ClientHello, the server responds with ServerHello and its certificate, the client encrypts a PreMaster secret and sends it, both sides derive the same session secret, and thereafter all data is encrypted and authenticated using that secret.

3. Appendix: Visual Metaphor

A simplified dialogue illustrates how the client (A) and server (B) exchange supported algorithms, certificates, and an encrypted secret to establish a secure channel.

4. SSL Security

Man‑in‑the‑middle (MITM) attacks can compromise SSL/TLS if weak key‑exchange algorithms (e.g., unauthenticated Diffie‑Hellman) are used or if users ignore certificate warnings. Export‑grade encryption limits in older browsers also pose risks.

5. Proxy Handling

When an HTTPS proxy is configured, the browser sends a CONNECT request to the proxy:

CONNECT server.example.com:443 HTTP/1.1
Host: server.example.com:443

The proxy then establishes a TCP connection to the target server. The TLS tunnel remains end‑to‑end; the proxy cannot read or modify the encrypted traffic unless it performs a MITM attack with a forged certificate.

6. References

Https (SSL/TLS) detailed principles

Keyless SSL: The Nitty Gritty Technical Details

Differences and introduction to SSL and TLS

Overview of SSL/TLS protocol operation

SSL/TLS/WTLS principles

Transport Layer Security (TLS)

Survival guides – TLS/SSL and X.509 certificates