BestHub
Sign in
Tagged articles

data platform

386 articles · Page 4 of 4
DataFunTalk
DataFunTalk
Jul 22, 2020 · Big Data

Building a Real-Time Computing Platform with Apache Flink at iQIYI: Architecture, Improvements, and Business Cases

iQIYI’s senior data engineer shares the evolution of its big‑data services from Hadoop to a Flink‑based real‑time computing platform, detailing architecture, monitoring improvements, StreamingSQL capabilities, business use cases like recommendation and deep‑learning data generation, and future plans for unified stream‑batch processing.

Apache FlinkFlinkReal-Time Computing
0 likes · 11 min read
Building a Real-Time Computing Platform with Apache Flink at iQIYI: Architecture, Improvements, and Business Cases
Big Data Technology Architecture
Big Data Technology Architecture
Jul 8, 2020 · Big Data

Key Interview Questions on Data Warehousing, Data Platforms, and Related Technologies

This article compiles a comprehensive set of 32 interview questions covering data warehouse fundamentals, data platform construction, modeling approaches, real‑time architectures, data quality, governance, Hive optimization, and related analytical techniques to help candidates prepare for data engineering roles.

Data ModelingData WarehouseETL
0 likes · 4 min read
Key Interview Questions on Data Warehousing, Data Platforms, and Related Technologies
Architects' Tech Alliance
Architects' Tech Alliance
Jul 1, 2020 · Big Data

Gartner’s Perspective on Building a Data Middle Platform: Strategies and Recommendations

According to Gartner, enterprises should view data middle platforms as strategic, collaborative hubs that balance data collection and connection, promote reusable analytics, integrate with digital platforms, and leverage data fabric, AI-driven tools, and graph analysis to become truly data‑driven while maintaining trust and privacy.

Data FabricGartneranalytics
0 likes · 12 min read
Gartner’s Perspective on Building a Data Middle Platform: Strategies and Recommendations
Big Data and Microservices
Big Data and Microservices
Jun 28, 2020 · Big Data

Data Warehouse vs Data Lake vs Data Platform vs Data Middle Platform: Which Fits Your Business?

This article compares data warehouse, data lake, data platform, and data middle platform, explaining their definitions, architectures, strengths, limitations, and use‑case differences, and provides tables that highlight how each solution handles structured and unstructured data, governance, flexibility, and business value.

Big DataData LakeData Warehouse
0 likes · 12 min read
Data Warehouse vs Data Lake vs Data Platform vs Data Middle Platform: Which Fits Your Business?
JD Retail Technology
JD Retail Technology
Jun 17, 2020 · Operations

How JD’s Data Platforms Scaled for the 618 Mega‑Sale: Operations, Stress‑Testing, and Dual‑Stream Architecture

The article details JD’s data product teams’ systematic preparation for the 618 shopping festival, covering pressure estimation, capacity expansion, stress testing, emergency downgrade strategies, dual‑data‑center isolation, high‑fidelity end‑to‑end testing, and continuous monitoring to ensure stable, real‑time data services during massive traffic spikes.

Big DataJD.comMonitoring
0 likes · 10 min read
How JD’s Data Platforms Scaled for the 618 Mega‑Sale: Operations, Stress‑Testing, and Dual‑Stream Architecture
DataFunTalk
DataFunTalk
Jun 14, 2020 · Big Data

Practical Experience and Optimization of Apache Druid for Real‑Time OLAP at iQIYI

This article describes how iQIYI evaluated various OLAP engines, selected Apache Druid for real‑time analytics, detailed its architecture, identified performance bottlene‑cks in Coordinator, Overlord and indexing, applied configuration and resource‑allocation optimizations, and built a user‑friendly RAP platform to democratize real‑time data analysis.

Apache DruidBig DataReal-time OLAP
0 likes · 15 min read
Practical Experience and Optimization of Apache Druid for Real‑Time OLAP at iQIYI
Cloud Native Technology Community
Cloud Native Technology Community
Jun 2, 2020 · Cloud Native

Building an Enterprise Cloud‑Native Architecture: A Comprehensive Guide

This article presents a step‑by‑step, enterprise‑wide roadmap for constructing a cloud‑native system, covering the five dimensions of architecture, a detailed evolution diagram, three maturity stages, common pitfalls, and practical solutions ranging from service‑oriented redesign to micro‑service governance and data‑driven personalization.

Enterprise ArchitectureMicroservicesdata platform
0 likes · 57 min read
Building an Enterprise Cloud‑Native Architecture: A Comprehensive Guide
Big Data Technology Architecture
Big Data Technology Architecture
May 19, 2020 · Big Data

Design and Implementation of a Unified Data Lake Platform Using HBase, Kafka, and Elasticsearch

This article summarizes the design, architecture, and key modules of a company-wide data lake platform—named “Tianchi”—built on HBase, Kafka, and Elasticsearch, detailing data ingestion, strategy output, metadata management, indexing, monitoring, and offline analysis, and shares lessons learned and future plans.

ElasticsearchHBaseKafka
0 likes · 11 min read
Design and Implementation of a Unified Data Lake Platform Using HBase, Kafka, and Elasticsearch
Big Data Technology & Architecture
Big Data Technology & Architecture
May 18, 2020 · Big Data

Real‑time Data Platform (RTDP): Concepts, Architecture and Design Considerations

This article examines the design of a real‑time data platform, discussing its background concepts, modern data‑warehouse perspective, architectural layers, unified data‑collection, streaming, compute and visualization platforms, and the functional, quality, stability, cost and agility considerations required for building an end‑to‑end real‑time pipeline.

Data DemocratizationReal‑time DataStream Processing
0 likes · 17 min read
Real‑time Data Platform (RTDP): Concepts, Architecture and Design Considerations
Big Data Technology & Architecture
Big Data Technology & Architecture
May 14, 2020 · Big Data

Five-Step Methodology for Building a Data Middle Platform

The article outlines a practical five-step methodology for constructing a data middle platform, covering data resource inventory, application planning, asset construction, detailed design and implementation, and organizational planning to guide enterprises toward effective data-driven transformation.

data assetsdata middle platformdata platform
0 likes · 8 min read
Five-Step Methodology for Building a Data Middle Platform
DevOps
DevOps
Apr 8, 2020 · Operations

Bilibili DevOps Case Study: Culture, Community, User‑Driven Demand Management, High‑Performance Microservices, and Data Operations

This article presents a comprehensive DevOps case study of Bilibili, covering its cultural background, community ecosystem, user‑centric demand management, migration to high‑performance microservices, and the implementation of logging, monitoring, and real‑time data platforms to support rapid, reliable delivery.

BilibiliDevOpsMonitoring
0 likes · 17 min read
Bilibili DevOps Case Study: Culture, Community, User‑Driven Demand Management, High‑Performance Microservices, and Data Operations
Cloud Native Technology Community
Cloud Native Technology Community
Mar 25, 2020 · Industry Insights

How a Top Real Estate Firm Used Cloud‑Native Middle Platforms to Accelerate Digital Transformation

The case study details how a major state‑owned real estate company, with nearly 40 years of experience, built a three‑stage cloud‑native middle‑platform strategy—business, data, and technology platforms—using microservices, containers, and Tencent Cloud TSF to achieve integrated operations, data‑driven decision making, and agile infrastructure across its nationwide projects.

Cloud NativeDevOpsMicroservices
0 likes · 9 min read
How a Top Real Estate Firm Used Cloud‑Native Middle Platforms to Accelerate Digital Transformation
Tencent Cloud Developer
Tencent Cloud Developer
Feb 13, 2020 · Big Data

Data Middle Platform: Vision, Architecture, and Business Value

The Data Middle Platform, described by Shi Kai, is a service‑oriented architecture that transforms raw enterprise data into reusable, real‑time APIs for business applications, bridging the gap between traditional warehouses and front‑end systems, accelerating digital transformation through unified governance, rapid development, and direct business value.

Big Datadata architecturedata middle platform
0 likes · 26 min read
Data Middle Platform: Vision, Architecture, and Business Value
Big Data Technology Architecture
Big Data Technology Architecture
Feb 3, 2020 · Big Data

NetEase Data Foundation Platform Construction – Technical Sharing

This article, originally shared by NetEase’s data expert Jiang Hongxiang on DataFun, outlines the construction of NetEase’s data foundation platform, covering database kernel insights and the implementation of the ad‑hoc query engine Impala with the distributed storage system Kudu, offering valuable big‑data engineering practices.

Data InfrastructureImpalaKudu
0 likes · 4 min read
NetEase Data Foundation Platform Construction – Technical Sharing
Architects' Tech Alliance
Architects' Tech Alliance
Jan 22, 2020 · R&D Management

The Evolution, Practices, and Pitfalls of Mid‑Platform (Zhongtai) Architecture in Large Tech Companies

This article traces the origin of the mid‑platform concept, examines how major Chinese tech giants implement and classify their mid‑platforms, explains the distinction between front‑end, back‑end, and mid‑platform layers, and outlines common pitfalls and practical challenges in building and operating such platforms.

data platformenterprisemid‑platform
0 likes · 16 min read
The Evolution, Practices, and Pitfalls of Mid‑Platform (Zhongtai) Architecture in Large Tech Companies
Bitu Technology
Bitu Technology
Dec 20, 2019 · Big Data

Building a Model‑Driven Data Platform at Tubi: From Data Warehouse to Automated Machine Learning

The article describes how Tubi, North America’s largest free‑streaming service, built a model‑driven data platform using a high‑quality data warehouse, DBT‑based transformations, Kubernetes‑hosted JupyterHub, low‑latency Scala/Akka services, and automated machine‑learning pipelines to accelerate experimentation and decision‑making.

Data Engineeringdata platformdbt
0 likes · 11 min read
Building a Model‑Driven Data Platform at Tubi: From Data Warehouse to Automated Machine Learning
vivo Internet Technology
vivo Internet Technology
Dec 18, 2019 · Big Data

Comprehensive Overview of Big Data Architecture, Lambda/Kappa Models, and End-to-End Data Platform Design

The article surveys modern big‑data architecture, contrasting Lambda and Kappa models, highlights common governance and integration pain points, and proposes an end‑to‑end platform featuring unified metadata, stream‑batch processing, one‑click ingestion, standardized modeling, intelligent query abstraction, and a comprehensive development IDE.

Big DataData ModelingETL
0 likes · 13 min read
Comprehensive Overview of Big Data Architecture, Lambda/Kappa Models, and End-to-End Data Platform Design
Product Technology Team
Product Technology Team
Dec 11, 2019 · Big Data

How a Data Middle Platform Transforms Business: Design, Architecture, and Modeling Insights

This article explains what a data middle platform is, why it matters, its core components—including storage, compute, IDE, workflow, API services, and data asset management—and details the layered architecture of ODS, DWD, DWT, DIM, and DWA, as well as dimensional modeling using Kimball’s methodology.

Big DataData Warehousearchitecture
0 likes · 6 min read
How a Data Middle Platform Transforms Business: Design, Architecture, and Modeling Insights
Big Data Technology & Architecture
Big Data Technology & Architecture
Dec 7, 2019 · Big Data

Understanding Data Middle Platform: Definition, Construction, Product Selection, and Case Studies

This article explains what a data middle platform is, outlines its construction process, discusses how to choose suitable products, and presents enterprise case studies, offering a comprehensive guide to building and leveraging a data middle platform for big‑data initiatives.

Big Data ArchitectureData Governancedata middle platform
0 likes · 5 min read
Understanding Data Middle Platform: Definition, Construction, Product Selection, and Case Studies
Yanxuan Tech Team
Yanxuan Tech Team
Dec 2, 2019 · Big Data

Why Modern Enterprises Need a Data Middle Platform: Lessons from NetEase Yanxuan

Drawing on NetEase Yanxuan’s experience, this article explains what a data middle platform is, why companies are building one for digital transformation and fine‑grained operations, and details its core components—including the data warehouse, data services, and BI platform—illustrated with real‑world diagrams.

BIBig DataData Warehouse
0 likes · 12 min read
Why Modern Enterprises Need a Data Middle Platform: Lessons from NetEase Yanxuan
Youzan Coder
Youzan Coder
Nov 20, 2019 · Big Data

Understanding Youzan's Data Middle Platform: Architecture, Challenges, and Construction

He Fei explains how Youzan built a two‑layer data middle platform—combining a technology stack of offline, online and streaming components with an asset layer for cataloguing, quality, lineage and unified APIs—to tackle diverse business demands, technical complexity, and to enable cost‑optimized, reusable real‑time data services.

Data Engineeringdata platform
0 likes · 15 min read
Understanding Youzan's Data Middle Platform: Architecture, Challenges, and Construction
dbaplus Community
dbaplus Community
Nov 3, 2019 · Databases

Insights from Data Platform Experts: Distributed Transactions, Aurora, and HBase

A recent data platform salon in Beijing gathered five leading experts who shared practical knowledge on data middle platforms, distributed transaction patterns, SQL audit design, Amazon Aurora's architecture, and JD's large‑scale HBase deployment, offering actionable guidance for modern enterprise data engineering.

Cloud databasesDatabasesHBase
0 likes · 6 min read
Insights from Data Platform Experts: Distributed Transactions, Aurora, and HBase
JD Retail Technology
JD Retail Technology
Oct 30, 2019 · Industry Insights

How JD.com’s Component‑Based Platform Is Redefining Omni‑Channel Retail in Southeast Asia

At the 2019 DTBB expo in Bangkok, JD.com showcased its component‑driven, data‑centric technology stack that powers omni‑channel retail, detailing three key trends, a modular architecture that cuts development costs by half, and a Thailand case study that boosted store traffic and reduced delivery expenses.

Component ArchitectureIndustry InsightsOmni-Channel
0 likes · 8 min read
How JD.com’s Component‑Based Platform Is Redefining Omni‑Channel Retail in Southeast Asia
Architects' Tech Alliance
Architects' Tech Alliance
Oct 17, 2019 · Big Data

Understanding Alibaba's Data Middle Platform: Concepts, Architecture, and Differences from Data Warehouses and Data Lakes

The article explains Alibaba's data middle platform—its definition, methodology, organizational structure, key tools, and how it differs from traditional data warehouses and data lakes—while highlighting its role in supporting scalable, business‑centric data services and digital transformation.

AlibabaBig DataData Lake
0 likes · 16 min read
Understanding Alibaba's Data Middle Platform: Concepts, Architecture, and Differences from Data Warehouses and Data Lakes
Snowball Engineer Team
Snowball Engineer Team
Sep 24, 2019 · Big Data

Snowball Data Middle Platform (AIBO): Architecture, Capabilities, and Future Outlook

The article introduces Snowball's AIBO data middle platform, detailing its storage‑compute separation architecture, core capabilities such as data integration, catalog, tagging, analysis tools, micro‑service data APIs, and outlines future enhancements for security, lineage, and continuous business‑driven iteration.

Big DataData AnalysisData Integration
0 likes · 12 min read
Snowball Data Middle Platform (AIBO): Architecture, Capabilities, and Future Outlook
Suning Technology
Suning Technology
Sep 20, 2019 · Big Data

How Suning’s Big Data Engine Powers Smart Retail Transformation

Suning’s big‑data center, built on a 30‑year retail evolution and leveraging technologies like AI, cloud, and IoT, showcases how integrated data platforms and robust security can drive smart retail, improve services for 600 million users, and create a new competitive edge.

AIBig DataCloud Computing
0 likes · 6 min read
How Suning’s Big Data Engine Powers Smart Retail Transformation
Big Data Technology & Architecture
Big Data Technology & Architecture
Sep 11, 2019 · Big Data

Big Data Technology and Architecture: Case Studies of Taobao, Didi, and Meituan

This article reviews the evolution and key components of big data platforms at leading Chinese internet companies—Taobao, Didi, and Meituan—detailing their data sources, synchronization tools, storage layers, processing engines, and scheduling systems to provide practical guidance for building robust big data infrastructures.

Big DataETLHadoop
0 likes · 9 min read
Big Data Technology and Architecture: Case Studies of Taobao, Didi, and Meituan
Tencent Cloud Developer
Tencent Cloud Developer
Jul 18, 2019 · Big Data

Tencent iData Analysis Center: Why We Chose Spark as Our Computing Platform

Tencent’s iData analysis center selected Spark as its new computing platform because, unlike ElasticSearch, TiDB, and other MPP solutions, Spark offers iterative processing, shuffle support, robust SQL and DAG scheduling, and flexible SMP‑style data exchange, enabling efficient OLAP on billions of game‑user records.

Big DataDistributed ComputingMPP
0 likes · 13 min read
Tencent iData Analysis Center: Why We Chose Spark as Our Computing Platform
JD Retail Technology
JD Retail Technology
Jun 20, 2019 · Big Data

JD.com’s 618 Technical Architecture: Componentization, Data Platform, and Elastic Computing at Massive Scale

The article details JD.com’s 618 shopping festival engineering, describing how componentized micro‑services, a unified data platform, and the Archimedes elastic scheduling system enabled billions of requests, real‑time data processing and seamless online‑offline integration without adding new server resources.

ComponentizationMicroservicesdata platform
0 likes · 8 min read
JD.com’s 618 Technical Architecture: Componentization, Data Platform, and Elastic Computing at Massive Scale
Dada Group Technology
Dada Group Technology
Jun 11, 2019 · Big Data

Building and Evolving the Dada‑JD Daojia Big Data Platform: Architecture, Strategies, and Lessons Learned

This article presents a comprehensive case study of the Dada‑JD Daojia big data platform, detailing its evolution from a MySQL‑based warehouse to a multi‑layered One Data, One Platform, One Service, Many Apps architecture, the technical challenges faced, and the strategic approaches adopted to ensure coverage, accuracy, stability, and scalability.

Big DataCase studyData Governance
0 likes · 14 min read
Building and Evolving the Dada‑JD Daojia Big Data Platform: Architecture, Strategies, and Lessons Learned
JD Retail Technology
JD Retail Technology
Jun 10, 2019 · Industry Insights

How JD.com’s Middle Platform Accelerates 618 Promo Preparation

JD.com’s R&D team transformed its 618 and 11.11 promotion preparation from a six‑month sprint to a one‑to‑two‑month process by building a unified middle platform that combines component‑based technology and data services, enabling rapid, scalable, and cost‑effective retail operations across multiple scenarios.

ComponentizationIndustry InsightsJD.com
0 likes · 9 min read
How JD.com’s Middle Platform Accelerates 618 Promo Preparation
Tencent Cloud Developer
Tencent Cloud Developer
Jun 6, 2019 · Big Data

2019 Big Data Industry Summit Highlights and Outcomes

From June 4‑5, 2019, the China‑hosted Big Data Industry Summit gathered more than 4,000 attendees and 60,000 online viewers to present award winners, release multiple whitepapers and standards, and hold six thematic forums and two roundtables that examined data platforms, asset management, security, law, and emerging technologies, outlining current opportunities and future challenges for big‑data growth.

Big DataChinaIndustry Summit
0 likes · 14 min read
2019 Big Data Industry Summit Highlights and Outcomes
Alibaba Cloud Developer
Alibaba Cloud Developer
Jun 6, 2019 · Artificial Intelligence

How SQLFlow Is Making AI as Simple as Writing a SQL Query

Ant Group's Vice‑CTO announced the open‑source SQLFlow tool that merges SQL simplicity with machine‑learning power, aiming to lower AI adoption barriers, while chief architect He Changhua outlines a real‑time big‑data platform that fuses OLTP, OLAP, and AI for universal data intelligence.

AIReal-Time ComputingSQLFlow
0 likes · 11 min read
How SQLFlow Is Making AI as Simple as Writing a SQL Query
DataFunTalk
DataFunTalk
May 17, 2019 · Big Data

Kuaishou Druid Platform Overview and Precise Deduplication Design

This article presents Kuaishou’s adoption of Apache Druid for massive real‑time analytics, explains why precise deduplication is required, details the platform’s architecture, the hashset and dictionary‑plus‑Bitmap deduplication designs, concurrency handling, performance optimizations, and outlines the future roadmap, providing practical insights for big‑data engineers.

Druiddata platformdeduplication
0 likes · 18 min read
Kuaishou Druid Platform Overview and Precise Deduplication Design
Architects' Tech Alliance
Architects' Tech Alliance
Apr 20, 2019 · Industry Insights

Why Data Middle Platforms Are the New Production Lines for Data Products

The article examines how data middle platforms transform raw, fragmented enterprise data into valuable data products through a supply‑chain approach, outlining their origins, core processes, deep‑processing techniques, and the essential capabilities needed for successful implementation.

Data EngineeringData ProductData Supply Chain
0 likes · 13 min read
Why Data Middle Platforms Are the New Production Lines for Data Products
dbaplus Community
dbaplus Community
Mar 21, 2019 · Big Data

How Real-Time Data Platforms Evolve: From Storm to Flink and Kubernetes

This article summarizes Wang Xinchun's 2018 DAMS China Data Asset Management Summit talk, detailing the current state, core services, responsibilities, evolution, architecture, challenges, and future directions of a large‑scale real‑time data platform built on Storm, Spark, Flink, and Kubernetes, including a unified data management approach.

FlinkKubernetesReal-time Streaming
0 likes · 22 min read
How Real-Time Data Platforms Evolve: From Storm to Flink and Kubernetes
AntTech
AntTech
Feb 27, 2019 · Big Data

Ant Financial Data Governance: Practices and Challenges in Data Quality Management

The article details Ant Financial’s comprehensive data quality governance framework, covering its architecture, challenges, implementation strategies, and real‑world case studies, illustrating how the company integrates data monitoring, AI‑driven self‑healing, and rigorous release controls to ensure high‑quality data across its platform.

Ant FinancialBig DataData Governance
0 likes · 17 min read
Ant Financial Data Governance: Practices and Challenges in Data Quality Management
Youzan Coder
Youzan Coder
Feb 13, 2019 · Big Data

Druid OLAP Platform Practice at YouZan: Architecture, Features, and Challenges

YouZan adopted MetaMarket’s Druid OLAP platform—featuring millisecond‑level interactive queries, high availability, horizontal scalability, and rich SQL/API query types—by configuring simple ingestion tasks that automatically manage real‑time and batch data, tiered hot/cold storage, and monitoring, while still facing ingestion limits, lack of joins, and occasional latency spikes.

Apache DruidDruidLambda Architecture
0 likes · 12 min read
Druid OLAP Platform Practice at YouZan: Architecture, Features, and Challenges
DataFunTalk
DataFunTalk
Jan 16, 2019 · Big Data

NetEase Data Infrastructure: Database Technologies and Big Data Platform Overview

This article presents NetEase Hangzhou Research Institute's experience in building a data infrastructure, covering database innovations such as InnoSQL, NTSDB, and InnoRocks, as well as the integration of big‑data components like HDFS, Spark, Impala, and Kudu to enable efficient storage, processing, and real‑time analytics.

DatabaseImpalaInnoSQL
0 likes · 12 min read
NetEase Data Infrastructure: Database Technologies and Big Data Platform Overview
Youzan Coder
Youzan Coder
Jan 9, 2019 · Big Data

How Youzan Scaled 5,000 Daily SparkSQL Jobs: Migration Lessons from Hive

This article details Youzan's transition from Hive to SparkSQL, covering platform architecture, usability and performance enhancements, migration strategies, automated engine selection, and future plans that together reduced resource consumption by up to 67% while handling thousands of daily jobs.

Big DataHive MigrationSparkSQL
0 likes · 13 min read
How Youzan Scaled 5,000 Daily SparkSQL Jobs: Migration Lessons from Hive
Alibaba Cloud Infrastructure
Alibaba Cloud Infrastructure
Nov 20, 2018 · Big Data

A Decade of Alibaba's Big Data Platform Evolution Through Double 11

The article chronicles Alibaba's ten‑year journey of building and scaling its big data platform—from early Oracle clusters and Hadoop‑based Cloud‑Ladder 1 to the self‑developed ODPS/MaxCompute, real‑time Blink engine, and the unified DataWorks ecosystem—highlighting key technical milestones, performance breakthroughs, and operational challenges that powered successive Double 11 shopping festivals.

AlibabaData EngineeringMaxCompute
0 likes · 22 min read
A Decade of Alibaba's Big Data Platform Evolution Through Double 11
Alibaba Cloud Developer
Alibaba Cloud Developer
Nov 15, 2018 · Big Data

How Alibaba Built a World‑Class Big Data Platform Over a Decade

This article chronicles Alibaba's ten‑year journey of building and scaling its big‑data platform—from early Oracle clusters and Hadoop, through the launch of ODPS and MaxCompute, to global cloud expansion and cutting‑edge streaming innovations that now power billions of transactions each Double‑11.

AlibabaMaxComputedata platform
0 likes · 23 min read
How Alibaba Built a World‑Class Big Data Platform Over a Decade
Meitu Technology
Meitu Technology
Aug 14, 2018 · Big Data

Meitu Data Platform Architecture and Practices

Meitu’s data platform, serving dozens of apps with 500 million monthly active users and billions of daily events, combines the Arachnia log‑collection system, Kafka ingestion, multi‑layer storage (HDFS, MongoDB, HBase, Elasticsearch), offline Hive/MapReduce processing and real‑time Storm/Flink/Naix pipelines, supported by data‑workshop tools, staged evolution for scalability, and robust security and query‑validation mechanisms.

Big DataData EngineeringETL
0 likes · 16 min read
Meitu Data Platform Architecture and Practices
Meitu Technology
Meitu Technology
Aug 11, 2018 · Big Data

Meitu Technology Salon: Evolution of the Big Data Platform, Distributed Bitmap (Naix), and Apache Kylin

At Meitu’s Technology Salon, senior big‑data experts detailed the end‑to‑end architecture and stability measures of Meitu’s large‑scale data platform, introduced the high‑performance distributed bitmap solution Naix, showcased the evolution of Meizu’s user‑insight system, and highlighted Apache Kylin’s OLAP capabilities and Superset integration for scalable, real‑time analytics.

Apache KylinBig DataData Analytics
0 likes · 9 min read
Meitu Technology Salon: Evolution of the Big Data Platform, Distributed Bitmap (Naix), and Apache Kylin
iQIYI Technical Product Team
iQIYI Technical Product Team
Aug 10, 2018 · Big Data

Data-Driven Entertainment: iQIYI’s Big Data Platform and AI Applications

iQIYI’s unified “Tongtian Tower” big‑data platform integrates analytics, AI and open APIs to turn viewer behavior and public sentiment into market insights, personalized recommendations, smart casting and churn‑prediction tools, embedding a data‑driven culture that fuels its rapid subscriber growth and revenue surge.

AIBig DataUser Behavior
0 likes · 12 min read
Data-Driven Entertainment: iQIYI’s Big Data Platform and AI Applications
360 Tech Engineering
360 Tech Engineering
Aug 7, 2018 · Big Data

Evolution and Practice of 360 Big Data Center Platform

The article presents a comprehensive overview of 360's Big Data Center evolution, covering business background, platform‑as‑a‑service architecture, data asset management, user‑profile unification, platform milestones, technical architecture, performance optimizations, online query capabilities, future plans, and a Q&A session.

360Case studyData Governance
0 likes · 22 min read
Evolution and Practice of 360 Big Data Center Platform
Architecture Digest
Architecture Digest
Jul 29, 2018 · Artificial Intelligence

Design and Implementation of a Machine Learning Data Platform at Getui

This article describes Getui's end‑to‑end machine‑learning data platform, covering business use cases, the full ML workflow from data ingestion and feature engineering to model training, deployment, monitoring, and the practical tools and solutions adopted to address common challenges in large‑scale AI projects.

AIJupyterModel deployment
0 likes · 11 min read
Design and Implementation of a Machine Learning Data Platform at Getui
Alibaba Cloud Developer
Alibaba Cloud Developer
Jul 23, 2018 · Big Data

How Alibaba’s MaxCompute Became the Backbone of 99% Data Processing

This article reviews Alibaba's MaxCompute evolution from ODPS to a unified, multi‑cluster big‑data platform, detailing its architecture, development tools, large‑scale deployments, performance optimizations, typical workload scenarios, and why it is the preferred choice for enterprise data processing.

Alibaba CloudBig DataData Warehouse
0 likes · 22 min read
How Alibaba’s MaxCompute Became the Backbone of 99% Data Processing
Youzan Coder
Youzan Coder
Jul 20, 2018 · Big Data

How Youzan Built a Scalable Big Data Development Platform (DP)

This article details the design, architecture, and operational experience of Youzan's Data Platform (DP), covering its scheduling, data‑sync, service, and monitoring modules, the custom Airflow‑based task scheduler, current production metrics, supported task types, and future improvement plans.

AirflowBig DataData synchronization
0 likes · 12 min read
How Youzan Built a Scalable Big Data Development Platform (DP)
dbaplus Community
dbaplus Community
Jul 11, 2018 · Big Data

How 360’s Titan Platform Evolved: From Script Templates to Real‑Time DAG‑Based Data Processing

This article outlines the evolution of 360’s Titan big‑data processing platform, describing the challenges of traditional script‑based development, the three architectural stages (pre‑Titan, Titan 1.0, Titan 2.0), the functional modules, the DITTO component framework, and key takeaways for building flexible, self‑service data pipelines.

DAGDITTOTitan
0 likes · 14 min read
How 360’s Titan Platform Evolved: From Script Templates to Real‑Time DAG‑Based Data Processing
ITPUB
ITPUB
Jun 14, 2018 · Big Data

Why Suning.com Sticks with Hadoop: Insights into China’s Big Data Platform Choices

Amid declining Hadoop usage reports, Suning.com’s 2018‑2020 big‑data platform case study reveals why the retailer still relies on Hadoop’s mature ecosystem, how it integrates HDFS, HBase, YARN, Hive, Spark, Flink and emerging tools, and what future resource‑management plans it envisions.

FlinkHadoopSpark
0 likes · 11 min read
Why Suning.com Sticks with Hadoop: Insights into China’s Big Data Platform Choices
Ctrip Technology
Ctrip Technology
Jan 18, 2018 · Artificial Intelligence

AI Algorithm Practices and Data Platform Architecture at Ping An Bank

The article presents Ping An Bank's AI-driven data platform, covering business background, architectural layers, algorithmic applications such as customer segmentation, portrait, business forecasting, and graph analysis, and shares practical insights on platform design, model deployment, and the role of data product managers.

AIBankingCustomer Segmentation
0 likes · 13 min read
AI Algorithm Practices and Data Platform Architecture at Ping An Bank
Alibaba Cloud Developer
Alibaba Cloud Developer
Nov 3, 2017 · Big Data

How Alibaba Built an EB-Scale, Real-Time Big Data Platform

Alibaba’s senior data expert Yao Bin Hui explains how the company constructed a standardized, end-to-end big-data ecosystem—from low-level data collection and AI algorithms to data services and product platforms—enabling petabyte-scale integration and second-level response times that power both internal operations and millions of external users.

AlibabaBig Datadata architecture
0 likes · 10 min read
How Alibaba Built an EB-Scale, Real-Time Big Data Platform
Meituan Technology Team
Meituan Technology Team
Nov 2, 2017 · Big Data

Meituan Dianping Technology Seminar

The Meituan Dianping Technology Seminar presents five expert‑led sessions covering big‑data recommendation strategies for homestays, the construction of Meitu’s data platform, personalized information‑flow implementation, cross‑city video‑conference communication systems, and the latest trends in front‑end technology, each illustrated with practical case studies.

AIPersonalized RecommendationTech Seminar
0 likes · 4 min read
Meituan Dianping Technology Seminar
Huawei Cloud Developer Alliance
Huawei Cloud Developer Alliance
Oct 31, 2017 · Big Data

How to Profit from Big Data in the Post‑Privacy‑Law Era

The article analyzes how recent data‑privacy regulations force the big‑data industry to choose between building costly, comprehensive data platforms or focusing on lean, high‑value applications, and outlines sustainable profit models and strategic trade‑offs for enterprises.

business modeldata platformdata privacy
0 likes · 16 min read
How to Profit from Big Data in the Post‑Privacy‑Law Era
21CTO
21CTO
Sep 25, 2017 · Big Data

How Meitu Scaled Its Billion-User Data Analytics: Architecture Evolution and Lessons

This article explains how Meitu built and evolved a large‑scale data statistics platform to handle billions of users, detailing the challenges of growing data volume, the architectural shifts from simple scripts to Hadoop, and the design of modular components for job management, scheduling, execution, and future expansion.

Big DataHadoopHive
0 likes · 16 min read
How Meitu Scaled Its Billion-User Data Analytics: Architecture Evolution and Lessons
Meituan Technology Team
Meituan Technology Team
Aug 25, 2017 · Big Data

Data Platform Integration and Multi‑Data‑Center Architecture at Meituan‑Dianping

After Meituan merged with Dianping, engineers unified two massive Hadoop ecosystems across Beijing and Shanghai by breaking the project into four phases—unify, copy, switch, fuse—standardizing versions, implementing zone‑aware transfers, cross‑realm Kerberos, and federated metadata to achieve a single, reliable multi‑data‑center platform.

Big DataCluster FusionDistcp
0 likes · 32 min read
Data Platform Integration and Multi‑Data‑Center Architecture at Meituan‑Dianping
Alibaba Cloud Developer
Alibaba Cloud Developer
Jul 17, 2017 · Artificial Intelligence

How Alibaba Turns Big Data into ‘Data New Energy’ with Automated Tagging and Distributed Knowledge Graphs

Alibaba's senior algorithm expert Yang Hongxia explains how the company fuses massive, heterogeneous data sources into a unified platform, builds automated tag‑production pipelines and large‑scale distributed knowledge graphs, and applies these technologies to drive smarter business decisions and AI‑enabled services.

AlibabaBig DataKnowledge Graph
0 likes · 14 min read
How Alibaba Turns Big Data into ‘Data New Energy’ with Automated Tagging and Distributed Knowledge Graphs
Baidu Waimai Technology Team
Baidu Waimai Technology Team
Apr 28, 2017 · Big Data

Recap of Baidu Waimai Tech Team’s “Code Talk” Session on Data Platform Architecture and Big Data Practices

The article summarizes Baidu Waimai’s recent “Code Talk” event, highlighting the speaker’s overview of the company’s big‑data platform evolution, its technical architecture, practical challenges such as data security and accuracy, and a lively Q&A covering storm, high availability, and metric management.

Baidu WaimaiBig DataTech Talk
0 likes · 6 min read
Recap of Baidu Waimai Tech Team’s “Code Talk” Session on Data Platform Architecture and Big Data Practices
Architecture Digest
Architecture Digest
Feb 11, 2017 · Big Data

LeKe Sports Big Data Platform Evolution: From Early ETL Reporting to 2.0 Streaming Architecture

The article describes how LeKe Sports built and continuously upgraded its Hadoop‑based big data platform—from a manual ETL‑to‑Elasticsearch reporting system to a 2.0 architecture featuring Spark Streaming, SQL‑based query layers, Elasticsearch indexing, and cloud‑native storage and backup solutions—to meet rapidly growing PB‑scale data demands.

Big DataETLHadoop
0 likes · 5 min read
LeKe Sports Big Data Platform Evolution: From Early ETL Reporting to 2.0 Streaming Architecture
dbaplus Community
dbaplus Community
Jan 8, 2017 · Big Data

How to Build a Cost‑Effective Data Platform for Small‑to‑Medium Enterprises

This article explains why data platforms are essential for modern SMEs, defines what a data platform is, outlines a four‑step methodology (source definition, analysis theme, ETL processing, and reporting), and shares architectural choices, team structures, common pitfalls, and practical advice for rapid, iterative implementation.

Data WarehouseETLSME
0 likes · 15 min read
How to Build a Cost‑Effective Data Platform for Small‑to‑Medium Enterprises
Meitu Technology
Meitu Technology
Dec 1, 2016 · Big Data

Multi-dimensional Analysis Platform Based on User Portrait Data

Tencent's Glacier multi‑dimensional analysis platform combines massive user‑portrait tags with routine analytical reports, delivering fast, accurate real‑time queries across countless dimensional combinations, enabling analysts and operators to perform targeted operations and insights as product data continuously evolves.

Big DataGlacierMulti-dimensional Analysis
0 likes · 1 min read
Multi-dimensional Analysis Platform Based on User Portrait Data
Meitu Technology
Meitu Technology
Dec 1, 2016 · Big Data

Meitu Internet Technology Salon: Big Data Architecture Evolution and Practice, and Tencent Multi‑Dimensional Analysis Platform

At Meitu’s third Internet Technology Salon in Xiamen on November 26 2016, over 150 senior engineers heard Meitu’s Lu Rongbin detail the company’s progression from simple rsync scripts to a scalable mobile data and open statistical platform, while Tencent’s Zhao Shiyuan showcased the Glacier multi‑dimensional analysis system for fast, tag‑driven queries, underscoring collaborative technical exchange in South China.

Big DataMeituTencent
0 likes · 6 min read
Meitu Internet Technology Salon: Big Data Architecture Evolution and Practice, and Tencent Multi‑Dimensional Analysis Platform
Architecture Digest
Architecture Digest
Nov 6, 2016 · Big Data

Evolution of Taobao’s Big Data Platform: From RAC to MaxCompute

The article chronicles Taobao’s 13‑year evolution of its big data platform, detailing three phases—from a single‑node Oracle setup and the Tianwang scheduler, through a Hadoop‑based “Cloud Ladder 1” architecture with real‑time analytics, to the current MaxCompute/ODPS era with cross‑region projects and advanced data services.

Big DataData WarehouseHadoop
0 likes · 11 min read
Evolution of Taobao’s Big Data Platform: From RAC to MaxCompute
Ctrip Technology
Ctrip Technology
Sep 19, 2016 · Product Management

Fundamentals and Implementation of A/B Testing at Qunar

This article explains the basic principles, practical demo, platform architecture, statistical validation, sample size estimation, and reporting workflow of A/B testing used at Qunar to evaluate advertising strategies and product features, illustrating how data‑driven experiments are designed, executed, and analyzed.

A/B testingdata platformexperiment design
0 likes · 9 min read
Fundamentals and Implementation of A/B Testing at Qunar
Architecture Digest
Architecture Digest
Apr 9, 2016 · Big Data

Practical Experience of Using Spark at Meituan: Platformization, ETL Templates, Feature Platform, Data Mining, and Real‑World Applications

This article describes how Meituan migrated from Hive‑SQL and MapReduce to Spark on YARN, built an interactive Zeppelin‑based development platform, created reusable ETL templates, constructed a Spark‑driven feature and data‑mining platform, and applied Spark to interactive user‑behavior analysis and large‑scale SEM services, highlighting performance gains and operational benefits.

Big DataDistributed ComputingETL
0 likes · 19 min read
Practical Experience of Using Spark at Meituan: Platformization, ETL Templates, Feature Platform, Data Mining, and Real‑World Applications
21CTO
21CTO
Nov 4, 2015 · Big Data

Evolution of Dazhong Dianping’s Data Platform (2012‑2014): Key Lessons for Growing Big Data Teams

This article chronicles the step‑by‑step evolution of Dazhong Dianping’s data platform from 2012 to 2014, detailing changes in data models, storage and compute architecture, scheduling, monitoring, and data‑driven applications, offering practical insights for teams building early‑stage big‑data infrastructures.

Big Data ArchitectureData EngineeringData Warehouse
0 likes · 7 min read
Evolution of Dazhong Dianping’s Data Platform (2012‑2014): Key Lessons for Growing Big Data Teams
Art of Distributed System Architecture Design
Art of Distributed System Architecture Design
Oct 29, 2015 · Big Data

TalkingData’s Journey to Building a Mobile Big Data Platform with Spark and YARN

This article recounts how TalkingData progressively introduced Spark into its Hadoop‑YARN based mobile big‑data platform, detailing early architectures, migration challenges, performance gains, the fully Spark‑centric redesign with Kafka and Spark Streaming, encountered pitfalls, and future plans for further optimization.

HadoopSparkSpark Streaming
0 likes · 16 min read
TalkingData’s Journey to Building a Mobile Big Data Platform with Spark and YARN