Mastering HTTP: Understanding Requests, Responses, and Headers

TCP/IP and HTTP Relationship

TCP/IP is a protocol suite divided into four layers: network interface, network, transport, and application. The transport layer includes TCP and UDP, while the application layer contains protocols such as FTP, HTTP, TELNET, SMTP, and DNS. HTTP is an application‑layer protocol used to transfer hypertext from web servers to browsers.

What is HTTP

HTTP (HyperText Transfer Protocol) is a stateless, request/response protocol widely used on the World Wide Web since the 1990s. Browsers and web servers exchange data over HTTP using TCP as the underlying transport.

URL Structure

A URL (Uniform Resource Locator) describes a network resource using the format scheme://host[:port]/path/...?[query-string][#anchor]. The scheme specifies the protocol (e.g., http, https, ftp), the host is the server’s domain or IP, the optional port defaults to 80 for HTTP, the path points to the resource, the query string carries parameters, and the anchor refers to a fragment within the resource.

HTTP Request/Response

Both request and response messages consist of three parts: a start line, header fields, and an optional body separated by a blank line. In a request, the start line includes the method (e.g., GET, POST), the request‑target (path), and the HTTP version. GET requests typically have no body, while POST requests carry data in the body.

Common Request Headers

Accept : media types the client can process (e.g., Accept: text/html or Accept: */*).

Referer : the URL of the page that linked to the requested resource.

Accept-Language : preferred languages (e.g., Accept-Language: en-us).

Content-Type : the MIME type of the request body (e.g., application/x-www-form-urlencoded).

Accept-Encoding : compression algorithms the client supports (e.g., gzip, deflate).

User-Agent : information about the client’s operating system and browser.

Connection : whether to keep the TCP connection alive or close it after the request.

Content-Length : size of the request body in bytes.

Host : required header specifying the target host and optional port.

Pragma : cache control directive (e.g., no-cache).

Cookie : sends stored cookies to the server.

Accept-Charset : character sets the client can handle.

Response Headers

Cache-Control : caching directives such as public, private, or no-cache.

Content-Type : MIME type and charset of the response body (e.g., text/html; charset=utf-8).

Expires : date after which the response is considered stale.

Last-Modified : timestamp of the resource’s last modification.

Server : software information of the HTTP server.

X-AspNet-Version : ASP.NET version when applicable.

X-Powered-By : technology used to build the site.

Connection : same semantics as in the request.

Content-Length : size of the response body in bytes.

Date : timestamp when the response was generated.

GET vs POST

GET appends data to the URL after a ? and is limited by URL length, making it suitable for queries but exposing data in logs and caches. POST sends data in the request body, allowing larger payloads and better security for sensitive information. GET parameters are accessed via Request.QueryString, while POST data is accessed via Request.Form.