What Jeff Dean Really Built: From MapReduce to Spanner

“The speed of light in a vacuum used to be about 35 miles per hour, then Jeff Dean spent a weekend optimizing basic physics.” This tongue‑in‑cheek quote illustrates the playful myths surrounding Jeff Dean, a renowned Google engineer.

While many of the “facts” on a Chuck Norris‑style website are jokes, Dean’s actual achievements are profound. He co‑designed the Google File System (GFS) and the MapReduce programming model, which became the foundation for the open‑source Hadoop framework and sparked the big‑data era.

Chuck Norris (罗礼士) is a karate world champion and American actor, known for his legendary martial‑arts roles.

In 2007, engineers created a parody site with jokes such as:

Compilers never warn Jeff Dean; Jeff Dean warns compilers.

Jeff Dean writes binary code, then documents it as source code.

When Jeff Dean thinks about ergonomics, it’s to protect his keyboard.

He once invented an asynchronous API so a function could return before being called.

Understanding these jokes requires a solid computer‑science background. Dean’s real contributions include MapReduce, BigTable, and Spanner—core systems that power Google’s search, advertising, and cloud services.

Dean joined Google in 1999 after early work on Epi Info, a disease‑tracking tool that outperformed contemporary software by 26× and was adopted by the CDC. He later left academia to help build Google News and AdSense, then focused on scalability challenges.

Facing the need to deploy machines faster, Dean and Sanjay Ghemawat created GFS and MapReduce, enabling parallel processing of massive data sets without developers worrying about hardware failures. Their 2004 paper made MapReduce an industry standard, leading to Hadoop and the rise of big‑data applications.

Building on GFS, Dean and Ghemawat developed BigTable, a distributed database handling petabytes of data, and Spanner, a globally consistent database that synchronizes data across data centers as if it were local.

Dean’s work exemplifies the exponential impact a top software engineer can have, often requiring a team of ten to achieve what a single developer might need months for.

He continues to collaborate, sharing coffee with colleagues and discussing ideas that drive innovation. Recently, he partnered with Andrew Ng and graduate Quoc Le on an unsupervised machine‑learning experiment using 16,000 processors to teach a system to recognize cats in YouTube videos, laying groundwork for future AI technologies.

Although not a machine‑learning specialist, Dean applies his expertise in building scalable, highly available systems to advance AI research.

Dean also emphasizes quick, approximate calculations for engineering decisions, maintaining a mental list of key numbers—such as the 150 ms latency for a packet from California to Amsterdam—to evaluate designs efficiently.

Original author: Will Oremus Translator: Lex Lian (伯乐在线) Source: http://blog.jobbole.com/47726/