This guide lists six open‑source AI skills that help product managers turn vague ideas into concrete designs, write structured PRDs, break requirements into executable plans, set up rigorous A/B experiments, implement analytics tracking, and optimize new‑user onboarding, each with install commands and usage examples.
DataAgent, built on Spring AI Alibaba, tackles the "last mile" of AI data analysis by combining deterministic workflow orchestration with large‑model reasoning, offering human‑in‑the‑loop feedback, dynamic prompt configuration, hybrid retrieval, containerized Python execution, streaming SSE, multi‑model scheduling, multi‑source connectivity, and secure API‑key management to deliver instant, insight‑rich reports for business users.
This article explains the purpose of the DWS (Data Warehouse Service) layer, why wide‑table modeling is crucial for performance and service‑oriented interfaces, and provides ten practical wide‑table designs with core field definitions, CREATE TABLE statements, and sample INSERT queries for common business domains such as products, users, orders, regions, channels, suppliers, services, finance, logistics, and data quality monitoring.
This article details BMW's digital transformation journey using Sensors Data, covering the background of rapid app growth, the cross‑regional data collection challenges, the systematic solution architecture—including mapping, preprocessing, and historical data migration—and the resulting business impact and future AI‑driven roadmap.
This article outlines the challenges of tracking AI‑generated code and measuring AI contribution, reviews earlier ad‑hoc methods, and presents a comprehensive solution featuring a VSCode plugin for unified AI dialogue management and a cloud service that quantifies AI impact across projects, teams, and individual developers.
This article reviews how Paimon and Doris combine to solve unified storage, data visibility, and performance challenges in modern lakehouse architectures, detailing their complementary features, integration capabilities, and real‑world use cases from leading companies.
This article presents a curated list of cutting‑edge topics covering AI‑powered data governance, large‑model applications, intelligent operations, and advanced analytics, offering readers a concise overview of emerging practices and case studies from industry leaders.
This article outlines Kuaishou's Data Platform team's mission to boost data‑driven decision making through advanced compute engines, high‑performance services, and AI‑enhanced BI, detailing its architecture, challenges, solutions, and future outlook for large‑scale intelligent analytics.
This article outlines how Kuaishou’s Data Platform Department enhances decision‑making efficiency by building advanced compute engines and high‑performance services, detailing the platform’s architecture, challenges of intelligent BI, AI‑driven solutions, and the end‑to‑end BI workflow from data ingestion to analysis.
This article presents a curated catalog of cutting‑edge topics covering AI‑powered data governance, large‑model applications, data cleaning, compliance, lakehouse integration, intelligent operations, and generative analytics, inviting readers to explore the latest innovations and download the full e‑book via QR code.
This article presents a comprehensive catalog of cutting‑edge AI and large‑model topics, covering financial data governance, proactive metadata systems, data cleaning compliance, lake‑warehouse integration, intelligent operations, generative analytics, and QR‑code access to the full e‑book.
This article presents a curated collection of cutting‑edge case studies and research on AI‑driven data governance, large‑model data cleaning, intelligent operations, and innovative analytics from industry leaders such as JD, Alibaba Cloud, Ping An, Didi, and Kuaishou.
This article presents a curated list of cutting‑edge resources covering AI‑powered data governance, large‑model driven data cleaning, intelligent operations, and generative analytics techniques from industry leaders such as JD.com, Alibaba Cloud, Ping An Life, Didi, and Kuaishou.
This guide presents a systematic approach to improve data warehouse modeling by enhancing model completeness, increasing reuse through unified dimensions and services, and enforcing naming and governance standards, supported by quantitative goals, practical strategies, and real‑world case studies.
This article examines nine recurring data‑warehouse dilemmas, exploring when modeling is essential, how to evaluate model quality, the boundaries between data warehouses and business systems, the evolution of modern warehouses, career growth for data engineers, and the future role of data R&D in the AI era.
This article dissects popular product management frameworks—from user stories and Maslow's hierarchy to AARRR, growth hacking, and data models—explaining their definitions, practical applications, and common pitfalls, helping practitioners separate genuine value from misleading hype.
This guide outlines a comprehensive data development roadmap, covering infrastructure setup, governance frameworks, automated pipelines, BI and analytics tools, AI/ML integration, cultural adoption, and continuous performance monitoring to enable intelligent business transformation.
This article details the business background, technical challenges, and the step‑by‑step implementation of an intelligent data warehouse on Alibaba Cloud MaxCompute, covering offline data pipelines, metric calculation, data analysis, and future plans for data lake and AI‑driven analytics.
The article explains how data lakes serve as a foundational component for observability by aggregating raw, diverse data for advanced analysis, while also outlining the technical, cost, and scalability challenges that make them unsuitable for every organization.
This article explains how integrating DuckDB as an extension for RDS PostgreSQL creates a unified HTAP solution that dramatically accelerates complex analytical queries through columnar storage, vectorized execution, and advanced optimizer techniques, delivering up to hundreds‑fold performance improvements and superior compression.
This article offers a comprehensive, multi‑angle guide to building and applying data indicator systems, covering core principles, step‑by‑step construction flow, data collection, cleaning, validation, and continuous optimization to empower digital transformation and high‑quality business growth.
This article outlines Douyin Group's approach to handling massive EB‑scale data, describing the challenges of metric quality and efficiency, the Volcano Engine data platform architecture, three‑layer solutions for metric production, management and consumption, and future plans for automation and governance.
The author recounts reviving the dormant GitStats project by migrating it to Python 3, adding CI/CD pipelines, publishing it on PyPI, providing Docker images and an online preview, while outlining future improvements and inviting community contributions.
This article outlines Douyin Group's approach to handling petabyte‑scale data, addressing metric inconsistencies, and improving data product agility through a four‑layer Volcano Engine platform, systematic indicator production‑management‑consumption cycles, organizational design, automation, and future plans for large‑model‑driven metric splitting.
This article presents Kuaishou's metric middle platform, detailing its background, design principles, architecture, metric management, data modeling, unified analysis language OAX, federated query engine OCTO, acceleration strategies, and future directions, illustrating how it improves data quality, development efficiency, and analytical capabilities at scale.
The article recounts Firefly Thinking's early BI system limitations, the decision‑making process that led to adopting Volcano Engine, subsequent operational strategies to unlock tool potential, and a forward‑looking vision of data analysis in the large‑model era.
This article explains the concept of data thinking, its core components of data sensitivity and methodological experience, outlines a step‑by‑step data analysis process, and shows why cultivating this mindset improves decision‑making, communication efficiency, and business opportunity discovery across various domains.
This article explains the fundamentals of event tracking, covering its definition, monitoring types, reporting methods, TypeScript‑based SDK implementation, testing strategies, and how to consume the collected data with SQL queries and funnel analysis to drive data‑driven decisions.
Amazon Web Services transferred its OpenSearch project—a fork of Elasticsearch and Kibana—to the newly formed OpenSearch Software Foundation under the Linux Foundation, gaining vendor‑neutral governance and support from members like AWS, Uber, Canonical, and Aiven, to foster broader community development of search, analytics, and vector database applications.
Pinterest migrated its Partner Insights analytics from Druid to StarRocks, achieving a 50% reduction in p90 latency, a six‑fold cost‑performance improvement, and simplified data ingestion, illustrating the benefits of a modern MPP database for real‑time ad analytics.
Front‑end developers must embed unique page identifiers in URLs, capture incoming source parameters, and propagate them through outbound links so that traffic, source, interaction and conversion metrics can be automatically recorded by the SDK, enabling data‑driven product optimization and business insight.
This article shares a data‑centric perspective on user growth, covering the evolution of information management, distribution and production, the concept of entropy reduction in products, the data‑driven flywheel model, practical AB‑testing case studies, and a Q&A on analytics tools and team collaboration.
This presentation by senior data warehouse engineer Huang Luo details Wing Payment’s end‑to‑end data warehouse build, covering background challenges, governance framework, platform architecture, layered modeling, naming standards, asset management, monitoring, and future plans, illustrating how systematic data governance drives cost reduction, efficiency, and security.
Event‑tracking governance ensures accurate, consistent user‑behavior data by managing the full lifecycle of logging points through defined quality standards, a digitized workflow, and supporting tools such as rule editors, real‑time testing, and compliance monitoring, while the platform’s page‑scene tree model and metrics improve visibility, reduce duplication, and drive business insight.
On June 3, 2024 DuckDB announced its 1.0.0 "Snow Duck" release, highlighting a custom single‑file storage format with full transaction semantics, backward compatibility, massive community adoption, and a roadmap focused on stability, extensions, and continued open‑source development.
Sugar BI, evolving from the internal ShowX platform to versions 2.0‑4.0, now offers a zero‑code, drag‑and‑drop visual editor, support for over 30 data sources, AI‑powered automatic analysis and the Sugar Bot Q&A module that transforms multi‑day data tasks into minutes, delivering containerized SaaS BI with intelligent chart recommendation and rapid, code‑free decision‑making for enterprises.
This article details the evolution of Baidu's Sugar BI platform, highlighting its AI‑driven analytics, extensive data source support, zero‑code visual design, smart chart recommendation, and the conversational Sugar Bot that transforms natural‑language queries into actionable visual insights.
This article walks designers through three progressive stages—Guard, Break, and Leave—to understand why data matters, how to select and apply it effectively, and how to avoid common pitfalls like Simpson's paradox, ultimately turning data into a strategic tool rather than a constraint.
This article presents an in‑depth overview of JD.com’s retail data visualization platform, detailing its product matrix—including EasyBI, a low‑code platform, and JDV large‑screen tool—its architectural layers, key capabilities, business case studies, challenges faced, and future development directions.
This article presents the problems faced by NetEase Cloud Music's data warehouse attribution system and details a comprehensive solution that includes upgrading the event‑tracking framework, redesigning the attribution model, and launching a unified management platform to improve stability, accuracy, and scalability.
This article introduces a new lakehouse analytics paradigm by combining StarRocks and Paimon, covering the evolution of data lake technologies, key integration scenarios, core technical mechanisms such as JNI connectors, materialized views, and future roadmap for enhanced lakehouse capabilities.
This article provides a comprehensive overview of open‑source OLAP technologies, examines various business scenarios and data‑lake architectures, and details StarRocks' core features, performance optimizations, and future development plans within the EMR ecosystem.
This article explains what frontend event tracking (埋点) is, why it is essential for product analytics, when and how to implement it, the different tracking models and reporting methods, as well as practical tips, iteration processes, and common pitfalls for developers and product teams.
This article provides a comprehensive guide to front‑end event tracking, covering its definition, motivations, scenarios, various tracking types, data models, reporting mechanisms, implementation steps, security considerations, and practical tips for ensuring accurate and non‑blocking data collection in web applications.
The article walks through a detailed, real‑world scenario where a business analyst leverages a data middle platform—covering metric analysis, exploratory queries, data development, visualization, and productization—to diagnose a 30% sales decline and implement data‑driven remediation, illustrating core concepts such as OneData and OneService.
This article outlines the evolution of software architecture styles, describing various architectural domains and sub‑domains—from web and mobile applications to integration, data, and analytics architectures—and their typical implementations, illustrated with a detailed classification table.
This article explains what a data lake is, its origins, key characteristics such as collecting all data, enabling diverse user access, and flexible processing, compares it with traditional data warehouses, discusses cost advantages, potential pitfalls like data swamps, and outlines best‑practice considerations for enterprise adoption.
This article details NetEase EasyData's evolution and product matrix, explains why a metric middle platform is needed, describes its core technical architecture—including a unified logical semantic model, a custom metric query language, and engine decoupling—and outlines future development directions.
This article details Bilibili's comprehensive approach to standardizing event tracking (埋点), covering its definition, data pipeline, common business issues, metadata‑driven management strategies, efficiency gains, and future prospects for unified real‑time and batch processing.
Data lakes and data warehouses serve different purposes in big‑data architectures; this article explains their definitions, core attributes, five major distinctions—including data retention, type support, user coverage, adaptability, and insight speed—and offers guidance on selecting or combining the two approaches.
This article explains what event tracking (埋点) is, its role in collecting user behavior data, the real‑time and offline data pipelines, the responsible teams, the end‑to‑end implementation process, common quality issues, detailed test cases, and the Track platform used for verification.
This report details the migration of Dada's unified data‑collection platform across 43 sites, outlining the achieved cost reductions, data‑analysis benefits, migration workflow, architectural design, technical highlights, and the challenges and solutions encountered during the project.
This article explains how to design efficient StarRocks tables with proper field types, partitioning and bucketing, compares update and primary‑key models, presents real‑world cases of memory and tablet issues, provides a complete table‑creation example, and outlines comprehensive monitoring metrics to keep the analytical data warehouse performant and stable.
This article introduces the concept of Data Fabric, explains how Dipu Technology built the FastData real‑time intelligent lakehouse platform on top of it, describes its architecture, core advantages, practical use cases in energy and retail, and outlines the platform’s future roadmap.
The article provides a deep technical overview of StarRocks 3.0’s data‑lake analysis capabilities, its unified Lakehouse architecture, Catalog integration, Trino compatibility, extensive I/O optimizations, materialized view features, resource isolation techniques, real‑world use cases, and future development directions.
This article explains the differences between diagnostic, descriptive, predictive, and prescriptive analytics, shows when each method is useful, and offers guidance on selecting the right analytics solution for modern enterprises dealing with massive data volumes.
The guide explains how to create an analytical data product by first defining the business scenario and KPI, selecting and abstracting an analysis framework into reusable modules, visualizing core metrics across dimensions, and continuously iterating through cold‑start, promotion, and maintenance phases to keep the product aligned with evolving business needs.
This article explains how MariaDB evolved from a MySQL branch into a comprehensive database ecosystem, covering the Community and Enterprise servers, diverse storage engines, MaxScale proxy, ColumnStore analytics, Xpand distributed scaling, and the SkySQL fully managed cloud service.
This article reviews a decade of data‑science growth, defines data science as a multidisciplinary field, outlines its four high‑level and fourteen low‑level work processes, categorises nine distinct data‑science roles, and discusses how these insights should shape the next generation of data‑visualisation and analysis tools.
This presentation introduces Apache Doris, examines changing data analysis demands in the cloud era, explains why SelectDB was created, and details SelectDB’s cloud‑native architecture, performance, unified capabilities, ease of use, cost efficiency, open‑source nature, and its application scenarios for modern data warehousing and log analytics.
The article explains how data‑driven thinking replaces traditional money‑burning growth tactics by establishing logical loops, experimental validation, and concrete case studies in acquisition, activation, and targeting, while outlining the essential collaborative factors needed for successful data‑powered operations.
The article argues that the era of big data is ending, showing that most organizations store only modest amounts of data, that storage costs outweigh benefits, and that modern cloud and analytics tools allow efficient processing without needing massive datasets.
This article introduces time series data concepts, outlines the challenges of storing and analyzing high‑frequency data, and presents best‑practice guidelines for building MongoDB‑based time‑series applications, covering ingestion, read/write workloads, retention, security, and real‑world use cases.
This article summarizes the Kuaishou Data Platform’s metric middle‑platform sharing from the 2022 DataFun forum, detailing its three‑year evolution, key concepts, architectural design, implementation challenges, and practical lessons for building an enterprise‑grade metric platform that unifies data definition, production, and consumption across the company.
NetEase Cloud Music’s open‑source Dawn framework introduces a high‑performance virtual tree (VTree) to deliver a unified, low‑cost, and highly accurate cross‑platform event‑tracking solution, addressing common pain points such as development overhead, precision, model stability, link tracing, and quality control.
Alibaba Cloud’s cloud‑native data lake analysis solution combines fully managed storage (OSS‑HDFS), a one‑stop lake management platform (Data Lake Formation), and multimodal compute capabilities, delivering high performance, massive scalability, and low cost for big‑data and AI workloads across offline, real‑time, and lake‑house scenarios.
This whitepaper explains why effective BI governance is essential for modern enterprises, outlines the key capabilities of data‑governance tools—including data quality, certification, usage statistics, classification, lineage, glossary, and lifecycle management—and shows how BI portals and data catalogs together enable scalable, user‑centric analytics.
This article introduces data visualization as a scientific discipline, explains its evolving concepts and basic ideas, outlines its relationship to related fields, and provides a curated list of free software tools for visualizing complex multidimensional data.
This article traces the evolution of the modern data stack, explaining the shortcomings of traditional data warehouses, the rise of cloud‑native ELT, self‑service analytics, analytics‑as‑software, and enhanced decision intelligence, while highlighting emerging trends and practical implementations.
This article introduces OAX (Open Analysis eXpressions), Kuaishou’s unified open‑analysis language, detailing its design background, guiding principles, five‑layer language model, syntax—including data types, compute capabilities and five analysis elements—its access protocol, runtime architecture, optimization steps, and the benefits it brings to the company’s big‑data analytics ecosystem.
This article details how ZuanZuan leveraged ClickHouse as the core OLAP engine for its massive self‑service analytics platform, covering OLAP engine selection criteria, system architecture, real‑world use cases, performance tuning, operational challenges, and future development plans.
This article traces the origins and evolution of the modern data stack, explains the shortcomings of traditional data warehouses, describes how cloud‑native ELT, modular components and analytics‑as‑software enable self‑service BI and AI, and outlines emerging trends such as DataOps, enhanced analytics and decision intelligence.
This comprehensive guide explains the evolution from traditional data warehouses to modern data lakes, detailing concepts, architectures, differences, implementation steps, and real‑world case studies, while also comparing major cloud providers' solutions and highlighting how data platforms support digital transformation and analytics.
This article introduces OLAP concepts, compares major OLAP engines such as Druid, Kylin, Doris, and ClickHouse, outlines real‑world application scenarios, and provides detailed optimization techniques—including materialized views, caching, tiered storage, and skip‑index configurations—to improve query performance.
The article explains how product and business teams should collaborate with data teams to build and promote data metric systems, emphasizing mutual empowerment, joint methodology, pilot testing, and scaling, while also announcing DataFun's 5‑year anniversary activities and upcoming big‑data and AI publications.
The article explains how wide tables are maintained without excessive resource consumption, outlines criteria for deciding which metrics belong in a wide table, describes their granularity and timeliness considerations, and clarifies that they do not automatically expand when tracking points change.
This article explains the concepts, architecture, and layered design of data warehouses, covering data flow, ETL processes, ODS, DWD, DWM, DWS, ADS layers, their characteristics, differences from databases, and the role of data marts in supporting OLAP and decision‑making.
This article details Xiaomi's multi‑year journey in building a group‑wide Business Intelligence platform, covering its historical evolution, technical challenges in performance, modeling, visualization and permissions, the current four‑layer architecture, and future plans to make the platform more business‑centric and simpler.
This article explains the fundamental concepts of data warehouses and data lakes, compares their architectures and use cases, discusses common misconceptions, highlights real‑world examples such as Facebook, and outlines the challenges and strategic considerations for organizations adopting both technologies.
This article details YiPay's big data BI analysis platform construction, covering its financial data use cases, platform architecture, OLAP engine implementations with ClickHouse, Presto, and Kylin, as well as identified challenges and future development directions.
This article provides an in-depth overview of SQL window (OLAP) functions, covering their classification, syntax, window definition methods, differences from aggregate functions, and includes interview-style queries for user behavior and student score analysis.
This article analyzes the limitations of traditional BI multi‑dimensional analysis that relies on wide tables and complex SQL JOINs, introduces a new DQL language that redefines JOIN operations into three plus one patterns, and demonstrates how DQL simplifies data modeling, reduces errors, and enables truly self‑service analytics.
This guide details how to plan, design, and manage enterprise‑level data tracking projects, covering role responsibilities, initial and iterative construction phases, event and attribute specifications, best‑practice tips, and common pitfalls to ensure accurate, maintainable analytics.
This article explains the differences between diagnostic, descriptive, predictive, and prescriptive analytics, showing when each approach is useful and how businesses can choose the right analytical solution to turn massive data streams into actionable insights.
The article explains how data lakes excel at ingesting massive, varied data, data warehouses optimize storage and query performance, and lake‑house architectures combine both strengths—offering scalable, low‑cost storage with high‑speed analytics—highlighting industry solutions from Snowflake, Databricks, and major cloud providers.
This article analyzes the technical difficulties of building a data‑driven full‑link system and describes how the Kai Shu Storytelling app overcame them by adopting DataFinder for automated event tracking, metric management, and growth analysis, offering practical guidance for enterprises and developers.
This article explains why event‑tracking (埋点) governance is essential, outlines the methodology and practice of full‑link tracking management, and introduces the one‑stop tracking platform with its innovative features such as standardized processes, verification tools, real‑time dashboards, cross‑platform data unification, and future roadmap.
The article presents a comprehensive guide to distributed storage, organizing its design and implementation into seven essential dimensions—replication, storage engine, transaction processing, analytical query execution, multi‑core scaling, compute engine architecture, and compilation techniques—each explained with core concepts, challenges, and practical considerations.
In the era of big data, poor operations data quality undermines analytics, decision‑making and digital transformation, so organizations must adopt a three‑dimensional governance approach—covering organization, processes and technology—to ensure completeness, consistency, accuracy, uniqueness, relevance and timeliness of their operational data.
This article explains why event‑tracking data governance is essential, outlines the methodology and practical steps, and introduces a one‑stop platform that standardizes data collection, validation, monitoring, and visualization to improve data quality, efficiency, and business insight.
MongoDB is evolving from a purely operational document store to a hybrid system that embeds native analytics, cloud‑native features, and SQL access, aiming to boost developer productivity, support real‑time insights, and complement rather than replace traditional data warehouses.
The article examines MongoDB’s evolution toward built‑in analytics, detailing new features like native search, time‑series support, change streams, Atlas analytics nodes, and the upcoming Atlas SQL interface, while arguing that these capabilities aim to empower developers rather than replace dedicated data‑warehouse solutions.
The article presents a comprehensive end‑to‑end traffic data architecture for an e‑commerce platform, detailing hybrid frontend/backend tracking with SPM/SCM/action standards, data‑warehouse construction of fact and dimension tables, UUID i_code unification, real‑time attribution methods, and future automation of warehouse and model layers.
This article presents a comprehensive overview of how Amazon EMR Serverless leverages serverless technology to simplify, scale, and cost‑optimize big data analytics, covering the evolution of serverless services, the intelligent lakehouse architecture, core concepts, key benefits, common use cases, and available documentation.
This article explains the concept of event tracking (埋点) in applications, detailing its purposes such as user conversion analysis and security monitoring, classifying manual, visual, and full tracking methods, and discussing their advantages, disadvantages, and how to choose the appropriate approach.
This article presents OPPO's commercial data system construction practice, covering the data platform strategy, ingestion pipelines, development efficiency toolkits, data validation, visualization aids, UDF principles, warehouse architecture, metric systems, dimensional modeling, ETL optimization, governance metadata, quality management, monitoring, attribution services, analytics reporting, and a Q&A session.
SAS announced native Python support in its analytics platform, offering code examples, cloud‑native AI capabilities, and seamless integration with SAS Studio and other SAS products to empower data scientists to develop and deploy Python models alongside SAS code.
Facing massive data storage and query challenges, 360 upgraded its analytics architecture by adopting StarRocks, achieving multi‑dimensional, high‑concurrency analysis, simplified data pipelines, and significant performance and cost improvements across its radar and user‑portrait platforms.
This article provides a comprehensive overview of distributed storage, covering seven core aspects such as replication, storage engines, transaction processing, analytical query execution, multi‑core scalability, computation models, and compilation techniques, while also highlighting practical challenges and design considerations for modern database systems.
The article introduces Event Collector, a markup‑based web event tracking framework that replaces imperative tracking with data attributes or React components, explains its technical implementation, management tools, monorepo organization, release workflow, and React coding conventions to improve development efficiency and data analysis.
