Understanding Intelligent Proxies and Their Implementation with Nginx

Not long ago my teammate Yariv introduced the OpenDNS Intelligent Proxy in a blog post, a proxy that lets us go beyond the DNS layer to block malicious HTTP traffic. Since then our team has been working on other projects, such as consolidating one of the oldest parts of our infrastructure, the lander, freeing more than 70 servers, and adding exciting new features.

Today I want to dive deeper into the intelligent proxy and the technology that supports it, namely Nginx.

By convention, a proxy is explicitly configured in the operating system’s network settings or in specific programs (for example the HTTP proxy settings of Chrome or Firefox).

In addition, the protocol can ensure that the proxy server always knows the client’s intended destination at request time. However, as Yariv explained, we adopted an unconventional approach: instead of proxying everything, we selectively reroute requests for suspicious domains to our proxy at the DNS layer. This selectivity reduces latency, load, and impact, but introduces interesting engineering challenges—mainly around identifying the user and determining the original destination.

For example, when a user tries to visit "some-website.net" and that domain is classified as suspicious, the OpenDNS resolver returns the IP address of the nearest Intelligent Proxy server. The client (e.g., Firefox or Chrome) is unaware of this and assumes the IP belongs to the actual host of "some-website.net". For plain HTTP it is easy to determine the original target because HTTP/1.1 requires a Host header in every request, which modern browsers include correctly. The same principle is used by shared‑hosting providers that serve multiple sites from a single IP. HTTPS traffic can be handled via the TLS Server Name Indication (SNI) extension. For other ports and protocols the process becomes more complex or even impossible.

Another important concept is the distinction between “forward” and “reverse” proxies. A forward proxy serves a set of clients, acting as a single access point that queries the original server on behalf of the client. This is the type used when configuring a proxy in the OS or in browsers like Firefox.

A reverse proxy works the opposite way: it acts as a single access point for multiple server components (such as CGI scripts, file servers, or databases). These proxies are also commonly used as load balancers and SSL termination points.

Based on this, when a client request is routed to our service, our intelligent proxy functions as a forward proxy. Internally it also contains some reverse‑proxy capabilities, especially when we add new features and additional data‑inspection layers. As mentioned earlier, we chose Nginx for this purpose, and readers familiar with Nginx know that its design is fundamentally that of a reverse proxy.

In the next article I will discuss more of the unconventional methods we have adopted and the challenges we must solve.

Original article: https://umbrella.cisco.com/blog/2015/09/18/lets-talk-about-proxies/

Full text: http://jiagoushi.pro/lets-talk-about-proxies

Discussion: please join the Knowledge Planet or WeChat group “Chief Architect Circle”.