Demystifying DNS and HTTP: How Browsers Find and Load Websites

Preface

Let's explore HTTP and DNS, the fundamental network knowledge every developer or ops engineer should know. The user‑to‑website flow can be divided into two parts: DNS (resolving domain names to IP addresses) and HTTP (fetching data from the server after the IP is known).

DNS (used to resolve the IP address of a domain)

HTTP (after obtaining the IP address, retrieve data from the server)

Now let's discuss what DNS and HTTP actually are.

What is DNS?

The Domain Name System (DNS) is an Internet service that maps domain names to IP addresses via a distributed database. DNS uses TCP and UDP port 53. Each label in a domain name may be up to 63 characters, and the full name cannot exceed 253 characters.

From Wikipedia.

DNS Principles

DNS essentially converts a domain name into an IP (A record). The following diagram illustrates the resolution process:

When a user enters

http://blog.ansheng.me

in a browser, the steps are:

The browser checks its local cache for the domain; if found, it returns the data, otherwise it proceeds.

The operating system's

hosts

file is consulted for a matching entry.

The local DNS (usually the router) is queried.

The local DNS checks its cache; if missing, it queries the root server

.

.

The root server returns the .me TLD servers.

The .me servers are asked for

blog.ansheng.me

and forward the request to the ansheng.me DNS servers.

The ansheng.me DNS server returns the IP address for

blog.ansheng.me

back through the chain.

The local DNS caches the record and returns it to the browser.

The browser now sends an HTTP request to the resolved IP address.

The web server processes the request and returns the page content.

The following diagram shows the process when forwarding mode is disabled:

Local DNS asks the root server for the record of

blog.ansheng.me

.

The root server replies that it does not have that record but knows the .me TLD servers.

The Local DNS contacts the .me servers, which in turn refer to the ansheng.me servers.

The ansheng.me servers provide the

blog.ansheng.me

record, which is passed back to the Local DNS and finally to the client.

From the browser to the local DNS server is a recursive query; DNS‑to‑DNS communication is an iterative query.

You can observe the full resolution trace with the

dig

command:

<code>[root@ansheng ~]# dig +trace blog.ansheng.me

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.47.rc1.el6_8.1 <<>> +trace blog.ansheng.me
;; global options: +cmd
.                       4695    IN      NS      a.root-servers.net.
... (output truncated for brevity) ...
blog.ansheng.me.        600     IN      A       104.224.139.81
</code>

What is HTTP?

HyperText Transfer Protocol (HTTP) is the most widely used Internet protocol for publishing and retrieving HTML pages. Resources are identified by Uniform Resource Identifiers (URIs) and can be requested via HTTP or HTTPS.

HTTP Messages

There are two types of HTTP messages:

Request Message – sent by the client to the server.

Response Message – sent by the server back to the client.

Example request message diagram:

Example response message diagram:

Header Capture

The following screenshots captured with

fiddler

show request and response headers:

Request headers:

<code># Request line
GET https://www.ansheng.me/ HTTP/1.1
# Request headers
Host: www.ansheng.me
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2868.3 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding: gzip, deflate, sdch, br
Accept-Language: zh-CN,zh;q=0.8
Cookie: _gat=1; _ga=GA1.2.1463852464.1476801026
# No request body for GET
</code>

Response headers:

<code># Status line
HTTP/1.1 200 OK
# Response headers
Server: nginx
Date: Tue, 18 Oct 2016 17:22:35 GMT
Content-Type: text/html; charset=utf-8
Last-Modified: Sun, 02 Oct 2016 05:51:17 GMT
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Encoding
ETag: W/"57f0a055-9fc"
Expires: Fri, 21 Oct 2016 17:22:35 GMT
Cache-Control: max-age=259200
Content-Encoding: gzip
</code>

Article originally published on Ansheng's blog: https://blog.ansheng.me/article/through-the-browser-to-access-a-site-behind-what-has-gone-through/