This article explores the cutting‑edge data pipelines of Netflix, Facebook, Airbnb and Pinterest, detailing the massive event volumes they handle, the core technologies such as Kafka, Spark, Presto and Hadoop, and how these giants design scalable, real‑time analytics infrastructures.
This article explains why data warehouses should be layered, describes the classic ODS‑DW‑APP model, details each layer’s purpose and implementation techniques, presents an improved layering scheme with dimension and temporary tables, and answers common questions about parallel DWS and DWD processing.
This article examines how Baidu Waimai designed and evolved its big data platform, comparing traditional BI to modern 4V‑driven architectures, detailing database choices, OLTP/OLAP trade‑offs, data‑analysis team structures, and the essential steps for data governance and platformization.
This article explains why data platforms are essential for modern SMEs, defines what a data platform is, outlines a four‑step methodology (source definition, analysis theme, ETL processing, and reporting), and shares architectural choices, team structures, common pitfalls, and practical advice for rapid, iterative implementation.
Game server development suffers from generic client‑communication tools and inadequate data stores, leading to duplicated, latency‑heavy code, so a purpose‑built, memory‑resident distributed cache that persists locally and eliminates serialization boiler‑plate is essential for real‑time, low‑latency gameplay.
The article shares a comprehensive recap of AWS re:Invent 2016, highlighting over a thousand new features across compute, storage, networking, security and tools, discussing serverless concepts, the Modern Data Architecture framework, and practical lessons learned by the English‑Fluent‑Talk engineering team.
The article details Umeng’s mobile big‑data platform architecture, describing its Lambda‑style hybrid design, data ingestion pipeline with dual Kafka clusters, offline and real‑time processing using Hadoop, Spark, Storm, and storage layers such as HDFS, HBase, MongoDB and Elasticsearch, while also discussing challenges in data collection, cleaning, computation, security, and value‑added services.
This article reviews a comprehensive big‑data lecture covering traditional databases, Hadoop ecosystems, commercial big‑data platforms, computing models, analysis techniques, visualization, and leading vendors, highlighting how these technologies shape today’s data‑driven enterprises.
This article details Meituan Dianping's data‑driven approach to precise marketing, describing the O2O marketing framework, a layered pyramid data system, profiling techniques, budget monitoring, and two real‑world case studies that together illustrate how big‑data technologies boost marketing efficiency on mobile platforms.
Meituan‑Dianping’s data‑driven precise‑marketing platform combines a layered pyramid architecture—data warehouse, service, and front‑end layers—with real‑time profile services powered by Redis and Elasticsearch, offering tools such as Hoek, Cord, and Cloud/Star to automate audience selection, coupon recommendation, and KPI monitoring, illustrated by food‑delivery user discovery and WeChat red‑packet coupon case studies, and guided by principles of reusable models and SOA decoupling.
This article presents Meituan Dianping's data‑driven precise marketing architecture, detailing a layered pyramid system, user profiling, budget monitoring, and two real‑world cases—potential user mining and a smart coupon engine—demonstrating how big‑data techniques improve marketing efficiency and ROI.
Airbnb’s engineering team outlines the evolution of its big‑data platform, detailing the philosophy behind its architecture, the dual “gold” and “silver” Hive clusters, migration to Mesos, use of Presto, Airpal, Airflow, and the performance and cost gains achieved through these design choices.
This article examines the critical issues of massive data handling, concurrency, file storage, relational design, indexing, distributed processing, AJAX usage, security, clustering, and OpenAPI trends that developers must address when architecting large, high‑interaction web sites.
This article explains how LinkedIn manages real‑time profile updates, news feeds, and social graph data through a three‑phase architecture that combines RDBMS, NoSQL stores, caching, and Lucene indexes to achieve high consistency, availability, and partition tolerance at massive scale.
The Meizu Tech Open Day showcased the company's rapid evolution to a data‑driven mobile internet firm, detailing its DW1.0 and DW2.0 data‑warehouse architectures, recommendation pipelines, Spark adoption, and ELK‑based log analytics, while sharing practical lessons and future challenges.
The article details Taobao's massive data product architecture, describing a five‑layer system that processes billions of daily records using Hadoop, real‑time streams, distributed MySQL and HBase clusters, and a middleware layer called Glider that unifies queries, caching, and front‑end integration.
This article introduces the fundamental MapReduce model and Hadoop framework, explains their roles in large‑scale data processing, and then examines Taobao’s massive‑data product architecture—including its data source, compute, storage, query, and product layers, as well as the MyFOX, Prom, and Glider components and caching strategies.
This article introduces the fundamental MapReduce model and Hadoop framework, explains their components such as HDFS, MapReduce, and HBase, and then examines Taobao’s large‑scale data product architecture—including storage, computation, query, and caching layers—to illustrate practical big‑data processing techniques.
The article outlines a practical, end‑to‑end data platform architecture for internet businesses, covering data collection, storage and analysis, sharing, real‑time processing, task scheduling, and the importance of simplicity and agility in building an agile data warehouse.
This article explores the 4V characteristics of big data, real‑world data growth examples, historical analogies, Google’s GFS‑MapReduce‑BigTable model, Hadoop’s architecture and HDFS processes, HBase components, NoSQL alternatives, and practical big‑data applications at Tencent and beyond.
This article chronicles Suning’s journey from early SAP‑based data warehouses to a modern, open‑source big data platform featuring real‑time collection, Hadoop‑Hive offline processing, Storm‑based streaming, and a visual development environment, highlighting how each layer addresses growing data volume, variety, and business demands.
This article explains why many companies over‑invest in Hadoop, outlines how to evaluate data size, growth, and relevance, and presents practical alternatives such as archiving, data sampling, database sharding, and hiring business‑savvy analysts before committing to a Hadoop deployment.
