Understanding DNS and HTTP: From Browser to Server Explained

Preface

Let’s explore the basics of HTTP and DNS, essential knowledge for both developers and operations engineers. The process of a user accessing a website can be divided into two parts: DNS (resolving domain names to IP addresses) and HTTP (fetching data from the server after the IP is known).

What is DNS?

The Domain Name System (DNS) is an internet service that maps domain names to IP addresses using a distributed database. DNS operates over TCP and UDP port 53. Each label in a domain name may be up to 63 characters, and the full domain name cannot exceed 253 characters.

Source: Wikipedia.

DNS Principles

DNS converts a domain name into an IP address (A record). The following diagram illustrates the resolution process.

When a user enters http://blog.ansheng.me in a browser, the steps to obtain the server’s IP address are:

The browser checks its local cache for the domain; if found, it uses the cached data, otherwise it proceeds.

The operating system’s hosts file is examined for a matching entry.

The local DNS (often the router) is queried for the domain’s IP address.

The local DNS checks its cache; if missing, it continues.

The local DNS queries a root server ( .). The root server does not have the specific domain but knows the .me TLD server.

The root server asks the .me server for the domain; the .me server does not have the record but knows the ansheng.me server.

The .me server asks the ansheng.me server, which returns the IP for blog.ansheng.me.

The IP address is passed back through the chain: ansheng.me → .me → root → local DNS.

The local DNS stores the record in its cache and returns it to the browser.

The browser now has the IP address and sends an HTTP request to blog.ansheng.me.

The server processes the request and returns the data, which the browser displays.

The diagram below shows the flow when forwarding mode is disabled.

Local DNS queries the root server for the domain record.

The root server replies that it lacks the record for blog.ansheng.me but provides the .me record.

The Local DNS then asks the .me server for the domain.

The .me server does not have it but knows the ansheng.me server, which returns the IP.

The Local DNS finally receives the IP and returns it to the client.

The query from the browser to the local DNS is a recursive query; the interaction between DNS servers is an iterative query.

You can observe the full resolution process with the dig command:

[root@ansheng ~]# dig +trace blog.ansheng.me

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.47.rc1.el6_8.1 <<>> +trace blog.ansheng.me
;; global options: +cmd
.                       4695    IN      NS      a.root-servers.net.
.                       4695    IN      NS      b.root-servers.net.
... (output truncated for brevity) ...
blog.ansheng.me.        600     IN      A       104.224.139.81
;; Received 49 bytes from 140.205.228.13#53(140.205.228.13) in 182 ms

What is HTTP?

HyperText Transfer Protocol (HTTP) is the most widely used network protocol on the internet. It was originally designed to publish and retrieve HTML pages. Resources requested via HTTP or HTTPS are identified by Uniform Resource Identifiers (URIs).

HTTP Messages

There are two types of HTTP messages:

Request Message – sent from client to server.

Response Message – sent from server to client.

Request Message example:

Response Message example:

Header Capture

The following diagram, captured with fiddler, shows request and response headers:

Request headers example:

# Request line
GET https://www.ansheng.me/ HTTP/1.1
# Request headers
Host: www.ansheng.me
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2868.3 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding: gzip, deflate, sdch, br
Accept-Language: zh-CN,zh;q=0.8
Cookie: _gat=1; _ga=GA1.2.1463852464.1476801026
# No request body for GET

Response headers example:

# Status line
HTTP/1.1 200 OK
# Response headers
Server: nginx
Date: Tue, 18 Oct 2016 17:22:35 GMT
Content-Type: text/html; charset=utf-8
Last-Modified: Sun, 02 Oct 2016 05:51:17 GMT
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Encoding
ETag: W/"57f0a055-9fc"
Expires: Fri, 21 Oct 2016 17:22:35 GMT
Cache-Control: max-age=259200
Content-Encoding: gzip
# Empty line follows before body