Tagged articles
DataOps
69 articles · Page 1 of 1
The article analyzes a Quantum Bit report on enterprise AI agent adoption, highlighting that data inconsistency and semantic gaps cause agent failures, and shows how NetEase Shufan's data platform provides trusted context, enabling Hengfeng Bank's T+1 performance insights and Sanhua's 10-minute budget analysis.
AI agentsDataOpsNetEase Shufan
0 likes · 10 min read

This article presents a six-step framework for operating high-quality datasets as continuous data products, covering responsibility assignment, version baselines, quality and AI effect monitoring, feedback-to-candidate pipelines, controlled release strategies, and retirement mechanisms to ensure datasets evolve with business, models, and risk boundaries.
AI data productsAgentOpsDataOps
0 likes · 14 min read

The article outlines how large language models can automate repetitive data‑warehouse tasks—from natural‑language SQL generation and standardized modeling to automated code review, metadata management, multimodal data handling, and self‑service analytics—presenting a three‑phase implementation roadmap for measurable efficiency gains.
AIChatBIDataOps
0 likes · 9 min read

This article details Hisense's journey to create an AI‑ready multimodal data platform, covering the challenges of integrating diverse business systems, the shift from a Hadoop‑based architecture to a cloud‑native data lake, the JuData governance and development platform, and six practical scenarios that demonstrate unified ingestion, metadata management, rule‑based quality control, intelligent asset retrieval, and future AI‑driven DataOps capabilities.
AI PlatformCloud NativeDataOps
0 likes · 23 min read

This article explains how dbt drives automated data modeling and governance, how DataOps practices bring agility and control to data projects, and how StarRocks’ lakehouse architecture enables real‑time and batch analytics, illustrated with concrete workflows, version‑control conventions, and enterprise case studies.
Data ModelingDataOpsELT
0 likes · 14 min read

The article walks through the evolution of data management—from idealistic visions and messy “shit mountains” to the realities of data lakes, metadata layers, governance challenges, trust breakdowns, and finally the promise of DataOps as a hopeful path forward.
DataOpsbig datadata governance
0 likes · 3 min read

This article examines the challenges of traditional banking data, explains how large language models can fuse structured and unstructured information, outlines a new data‑centric infrastructure and governance approach, and describes the DiFY platform’s AI‑agent and DataOps capabilities for agile, non‑intrusive integration with core banking systems.
AI agentsDataOpsbanking data
0 likes · 16 min read

The article details Xiaohongshu’s Dataverse platform evolution into a Data+AI system, highlighting inefficiencies in algorithm and data‑science workflows, the introduction of an interactive notebook, comprehensive data lineage, AI‑coding assistance, and future DataAgent plans to automate data engineering tasks.
AI codingDataOpsdata lineage
0 likes · 21 min read

Alibaba Cloud’s Serverless Elasticsearch service, combined with the SREWorks data‑driven operations platform, offers a cloud‑native, real‑time search and analytics engine that integrates metric and log collection, cost management, and health monitoring to enhance scalability, performance, and operational efficiency for enterprise applications.
Cloud NativeDataOpsElasticsearch
0 likes · 11 min read

The article details how Alibaba’s Tesla SRE platform supports the massive offline and real‑time big‑data ecosystems through a layered, data‑driven operations framework—DataOps—integrating unified portals, configuration, job, workflow, and analytics platforms, enabling automated monitoring, intelligent decision‑making, and self‑healing capabilities across 100,000+ nodes.
AIOpsDataOpsSRE
0 likes · 20 min read

Alibaba's SREWorks platform, now open‑source, combines cloud‑native architecture, DataOps and AIOps to address the growing complexity of big‑data and AI operations, offering a layered SaaS/PaaS/IaaS solution that streamlines delivery, monitoring, management, control, operation, and service for modern enterprises.
AIOpsCloud NativeDataOps
0 likes · 10 min read

This article explains how to transform traditional SRE troubleshooting into a data‑driven process by pre‑collecting operational metrics into a data warehouse, using ETL to create layered data models (ODS, DIM, DWD, DWS) that enable efficient, repeatable analysis while balancing data freshness and storage costs.
DataOpsETLbig data
0 likes · 7 min read

Tracing Alibaba’s journey from manual, script‑based operations through tool‑centric and platform‑driven DevOps to the data‑focused DataOps era and emerging AIOps, the article outlines the shifting responsibilities, architectural challenges, and the multidisciplinary skill set required for modern operations engineers.
AIOpsDataOpsSkill Development
0 likes · 8 min read

The article explains how DataOps and SRE practices enable large‑scale, data‑driven operations in big‑data environments, aiming for fully automated, intelligent, and ultimately unmanned management of complex systems.
AI OpsDataOpsSRE
0 likes · 6 min read

This article examines how Hainan Shuzhao Technology leverages ChatGPT‑4 and other large language models to enhance DataOps, address traditional data management challenges, improve Text2SQL accuracy, and outline future directions for agile, AI‑driven data pipelines.
AIData EngineeringDataOps
0 likes · 23 min read

This article presents Grab's Hugo platform, an automated fault‑detection and self‑healing system for over 4,000 data pipelines that combines multi‑source signal collection, intelligent diagnosis, layered auto‑repair, and a health API to dramatically improve data visibility, reduce manual intervention, and boost operational efficiency across the company.
DataOpsautomationbig data
0 likes · 12 min read

The article identifies four core challenges in data science—data acquisition and management, technical engineering gaps, team collaboration barriers, and maintenance issues—and proposes a DataOps approach across technology, process, and organization to transform data assets into business value.
DataOpsMLOpsdata governance
0 likes · 9 min read

This article analyzes DataOps as a deep fusion of data analytics, lean thinking, agile practices, and DevOps culture, contrasting it with DevOps, detailing its four genetic components with case studies, explaining its emergence from technical, business, and organizational drivers, examining current hiring fragmentation and talent gaps, and predicting explosive job growth driven by policy, tooling maturity, and talent supply reforms.
AgileData EngineeringDataOps
0 likes · 20 min read

This article details a comprehensive DataOps implementation using Alibaba Cloud's DataWorks, Flink, StarRocks, MaxCompute, and Yunxiao pipeline, covering four-layer architecture, standardized development, automated CI/CD with gray releases, full-chain quality monitoring achieving 80% self-healing, cross-team collaboration, cost optimization, and a phased rollout roadmap delivering 30% faster delivery and 30% cost reduction.
Alibaba CloudCI/CDDataOps
0 likes · 18 min read

This article explains the data resourceization stage, outlines seven core data management functions, introduces the DataOps paradigm to overcome traditional inefficiencies, and showcases real‑world case studies and a panoramic data architecture view for modern enterprises.
Data ManagementDataOpsMaster Data
0 likes · 13 min read

This report outlines a comprehensive framework for turning raw data into valuable assets, introducing DataOps and panoramic data architecture, and detailing practical methods for data value assessment, asset circulation, and operational mechanisms to help enterprises build a solid value baseline and expand data asset applications.
DataOpsbig datadata asset management
0 likes · 4 min read

This article analyzes the three types of data silos — technical, business, and management — their hidden costs, and presents a four-layer framework for building a data middle platform, illustrated with case studies from companies like State Grid, SF Express, Taobao, and ICBC, plus pitfalls to avoid and future trends.
CDCCloud NativeDataOps
0 likes · 16 min read

The article outlines how to obtain the DataFun Summit e‑book by following the public account instructions and provides concise English summaries of twelve technical sessions covering data lineage, integration, AI language models, multimodal content, game AI agents, lake‑warehouse governance, big‑data architecture, and cluster management.
AIDataOpsbig data
0 likes · 5 min read

This article presents a comprehensive overview of how DataOps principles combined with large language models such as GPT‑4 enable more agile and intelligent data engineering workflows, focusing on Text2SQL applications, schema‑linking techniques, practical product implementations, and future research challenges.
AIDataOpsSchema Linking
0 likes · 23 min read

The article outlines how data has become a critical production factor, reviews national policies, industry and enterprise challenges, and presents six key trends—including DataOps, DataFabric, and AI‑focused governance—that shape the future of data management and security.
AI data governanceDataFabricDataOps
0 likes · 12 min read

The article examines the evolution of data integration architectures—from traditional ETL through ELT to the emerging EtLT model—highlighting their advantages, disadvantages, industry trends, maturity stages, and practical guidance for enterprises and professionals navigating modern big‑data pipelines.
DataOpsELTETL
0 likes · 31 min read

This article explains how metadata‑driven data governance addresses the challenges of the digital economy by detailing the era background, limitations of traditional methods, the roles of Data Fabric, Data Mesh and DataOps, and presenting real‑world case studies and future directions.
AIData FabricDataOps
0 likes · 14 min read


DevOpsJan 17, 2024 · Operations
This article explains how agile methodologies can be applied to data management, covering the need for agile data practices, core principles, iterative modeling, governance, CI/CD pipelines, tooling, metrics, security, case studies, challenges, and future outlooks in a comprehensive, step‑by‑step guide.
AgileData ManagementData Modeling
0 likes · 13 min read

Senior Tencent product manager He Zhichao shares his experience and methodology for creating a high‑quality data platform, covering the transition from technical roles to product, understanding data users’ needs, the Euler asset‑factory implementation, product‑manager best practices, and solutions to common data‑engineering challenges.
Data EngineeringDataOpsdata governance
0 likes · 16 min read

This article presents NetEase Data's end‑to‑end experience in building a metrics middle platform, covering product evolution, the motivations for a unified metrics layer, core technologies such as a logical semantic model, a custom metric query language, engine‑agnostic execution, and future roadmap plans.
DataOpsMetrics PlatformSQL
0 likes · 12 min read

This article examines the challenges of big‑data operations, explains how DataOps and AIOps complement each other, and details the ABM intelligent operations architecture, platform components, and real‑world use cases such as Flink hotspot detection, ChatOps assistants, and dynamic MaxCompute resource optimization.
AIOpsDataOpsbig data operations
0 likes · 11 min read

In this presentation, Guo Wei, CEO of Baijiang Open Source and seasoned big‑data expert, explores how large‑model AI reshapes DataOps and DataFabric, detailing efficiency gains, intelligent deployment, and future enterprise architectures for big‑data and AI integration.
DataFabricDataOpsartificial intelligence
0 likes · 3 min read

The article introduces Tencent's OLA data‑governance platform and its t‑Metric metric middle‑platform, explains the Headless BI concept, details the configuration‑driven metric production workflow, core capabilities, architecture, unified query service, ecosystem integration, and answers audience questions about real‑time analysis, dimension handling, and trust mechanisms.
DataOpsHeadless BIdata governance
0 likes · 21 min read

This article reveals how Baidu implements end‑to‑end DataOps for its commercial data products, covering challenges of massive report generation, the design of a layered data architecture, platform‑wide automation, serverless deployment, risk control, monitoring, and optimization to achieve scalable, reliable data pipelines.
DataOpsOptimizationbig data
0 likes · 13 min read

Bilibili's SQLScan is a static‑code analysis tool that parses Hive, Spark, Presto and Flink SQL via Antlr4, builds a unified AST, applies engine‑specific metadata plugins for rule enforcement, provides field‑lineage and cost‑analysis services, and has processed hundreds of thousands of daily queries, intercepting thousands of problematic statements to improve data quality and operational efficiency.
BilibiliDataOpsSQL static analysis
0 likes · 11 min read

This article introduces DataOps, outlines its background and the problems it addresses, describes NetEase’s big‑data product ecosystem, and details a six‑stage data production pipeline—including coding, orchestration, testing, code review, release approval, and deployment – plus insights into two pipeline explorations.
DataOpsbig datadata quality
0 likes · 15 min read

The article summarizes ByteDance's DataOps journey, detailing its mid‑platform tool and Data BP model, core performance metrics, quality, hardware and human efficiency challenges, concrete DataOps implementation, productization through DataLeap, best‑practice promotion, and future outlook for data‑driven business value.
ByteDanceDataOpsbig data
0 likes · 17 min read

This article explains the concept of Data Fabric (data weaving), its background, definition, relationship with data middle platforms, data governance and DataOps, compares it with related concepts, and outlines future development directions and challenges.
Data FabricDataOpsknowledge graph
0 likes · 15 min read

This article explores the challenges of data quality in mature big‑data environments and presents intelligent, automated approaches—including assertions, automatic detection, rule recommendation, link checking, and collaborative mechanisms—to embed quality checks throughout the data pipeline, improving efficiency and reliability.
Data ObservabilityDataOpsautomation
0 likes · 18 min read

At the 2023 Big Data Industry Development Conference in Beijing, the China Communications Standards Association unveiled the top ten big‑data keywords, highlighting trends such as lake‑warehouse integration, data assetization, DataOps, intelligent analytics, data ethics, security, public data licensing, and cross‑border data flows.
Data EthicsDataOpsbig data
0 likes · 16 min read

The article recounts NetEase’s presentation at the inaugural DataOps conference, detailing the evolution from DataOps 1.0 pipeline to a 2.0 integrated data development‑governance model, the challenges faced, practical solutions, and strategic advice for data managers.
Data EngineeringData ManagementDataOps
0 likes · 11 min read

In this interview, three data‑governance experts from Tencent, ByteDance, and Alibaba discuss the fundamental processes, core domains such as metadata, data lineage, metric systems, data quality and security, the main challenges they face, and emerging trends like DataOps, AI‑driven automation, and privacy‑preserving technologies.
DataOpsdata governancedata lineage
0 likes · 14 min read

The article examines how China's data middle platform concept is reshaping enterprise data strategy, highlighting a shift toward value‑driven adoption, the intertwined relationship with data governance, and emerging trends such as fine‑grained business governance, full‑link monitoring, integrated platforms, and DataOps.
DataOpsEnterprise Databig data
0 likes · 9 min read

DataOps, rooted in agile and DevOps philosophies, uses automation and collaborative practices to streamline data processing, improve quality, and align analytics with business goals, offering continuous analytics, faster insights, and breaking data silos for better decision‑making across organizations.
Continuous AnalyticsDataOpsDevOps
0 likes · 10 min read

This article traces the evolution of the modern data stack, explaining the shortcomings of traditional data warehouses, the rise of cloud‑native ELT, self‑service analytics, analytics‑as‑software, and enhanced decision intelligence, while highlighting emerging trends and practical implementations.
AnalyticsBICloud Data Warehouse
0 likes · 15 min read

This article traces the origins and evolution of the modern data stack, explains the shortcomings of traditional data warehouses, describes how cloud‑native ELT, modular components and analytics‑as‑software enable self‑service BI and AI, and outlines emerging trends such as DataOps, enhanced analytics and decision intelligence.
AIAnalyticsBI
0 likes · 15 min read

Zhejiang Mobile leveraged DataOps principles to create a unified, micro‑service‑based anti‑fraud platform that integrates multi‑source telecom data, improves data governance and development efficiency, and achieved top‑level evaluation, showcasing a scalable solution for combating sophisticated telecom scams.
DataOpscase studyplatform architecture
0 likes · 7 min read

DataOps, the data‑centric counterpart of DevOps, combines agile principles, standardized tools, and cross‑team collaboration to manage the full data lifecycle—from integration and development to storage, governance, and service—enabling organizations to handle massive, diverse datasets efficiently, reduce silos, and turn data into actionable value.
Data ManagementDataOpsbig data
0 likes · 15 min read

This article examines the four‑stage evolution of data warehouses, highlights the cost‑effective, scalable advantages of cloud‑native warehouses, explores the rapid growth of data‑management infrastructure, and discusses the emerging practices of DataOps and AI integration that are redefining modern data stacks.
AIData ManagementDataOps
0 likes · 15 min read

This article explores DataOps from its historical evolution through the digital 3.0 era, outlines its core technology stacks such as Data Fabric, Data Mesh, and Modern Data Stack, and demonstrates practical applications across finance, manufacturing, telecom, and public services, highlighting its role in agile, cloud‑native data management.
DataOpsbig datadata governance
0 likes · 18 min read

This article details NetEase DataFan's journey in building a full‑stack big‑data platform, explains the design‑first data‑mid‑platform approach, analyzes cost, quality, and security problems encountered, and presents the modern data‑governance framework that integrates development, governance, and consumption into a closed loop.
DataOpsbig datacost management
0 likes · 22 min read

The article presents an in‑depth overview of Tencent's Oula data governance platform, describing its construction goals, core capabilities, DataOps‑driven development workflow, unified metric store, data map services, and practical Q&A on asset health scoring and data lineage, illustrating a comprehensive end‑to‑end big‑data governance solution.
DataOpsTencentbig data platform
0 likes · 17 min read

Tencent's Oula platform, launched in 2019, provides a DataOps‑driven, end‑to‑end data governance solution covering data discovery, asset factory, metric platform, and governance engine, and the talk details its construction goals, data development governance, unified metric system, data map, and Q&A on asset health and lineage.
DataOpsdata platformmetadata
0 likes · 17 min read

This article explains DataOps as a methodology for improving data analysis quality and efficiency, outlines its origins, standards, and maturity model, and presents practical insights and case studies from Chinese enterprises on how DataOps addresses common data engineering challenges and drives digital transformation.
Data ManagementDataOpsbig data
0 likes · 12 min read

The DataFun Summit 2022, held on September 17, gathered leading experts from Baiji Whale Open Source, NetEase, Tapdata, and Alibaba Cloud to share deep technical insights on SeaTunnel V2 architecture, DataOps implementations, and open‑source big‑data studio tools, offering attendees practical guidance for modern data platforms.
ApacheDataOpsSeaTunnel
0 likes · 8 min read

This guide explains how to create a robust GitHub Actions CI/CD workflow that automatically tests Apache Airflow DAGs using pytest, flake8, and Black, then securely deploys them to Amazon Managed Workflows for Apache Airflow (MWAA) with optional Git hooks and fork‑and‑pull collaboration models.
AWSApache AirflowCI/CD
0 likes · 20 min read

ICBC’s software development center outlines its multi‑year journey adopting XOps practices—DevOps, DevSecOps, DataOps, MLOps, AIOps, ChatOps and BizDevOps—to boost development efficiency, enhance security, accelerate data‑driven AI, and cut costs, showcasing measurable improvements in release frequency, defect rates, and operational automation.
AIOpsDataOpsDevOps
0 likes · 13 min read

This article explains DataOps and its role in modern ETL pipelines, outlines the benefits of DataOps for efficiency and reliability, and provides a detailed roadmap and best‑practice guidelines for planning, implementing, and optimizing DataOps‑driven ETL in cloud‑native environments.
Data EngineeringDataOpsETL
0 likes · 13 min read

Xinwang Bank leverages Apache DolphinScheduler to handle over 9,000 daily task instances across real‑time, near‑real‑time, and offline batch scenarios, detailing background, application scenarios, optimizations, workflow improvements, import/export enhancements, alert system upgrades, and future plans to expand data‑ops capabilities.
Apache DolphinSchedulerDataOpsTask Optimization
0 likes · 13 min read

This article details how NetEase's DataFunTalk presentation explores the growing data quality challenges in data development and demonstrates the application of DataOps principles—including pre‑ and post‑control mechanisms, sandbox environments, and automated governance tools—to systematically reduce defects, optimize resources, and ensure reliable data delivery across the company's diverse business lines.
DataOpsbig datadata platform
0 likes · 14 min read

Vivo’s self‑built database and storage platform evolved from a fragmented 2018 system into a two‑layer DaaS solution offering unified management, automated security, high availability, and cost‑effective scaling across MySQL, Redis, TiDB, object storage, and future autonomous services within a hybrid‑cloud architecture.
DaaSDataOpsMySQL
0 likes · 18 min read


21CTOOct 18, 2021 · Operations
The article surveys rapidly growing IT positions—from quantum computing engineers and security‑compliance managers to big‑data, analytics, and DataOps engineers—explaining how these roles combine advanced technologies, regulatory expertise, and operational practices to drive business transformation and meet the evolving demands of digital enterprises.
CloudOpsDataOpsEmerging roles
0 likes · 9 min read


DevOpsFeb 9, 2021 · Operations
The article examines how data teams can select the appropriate Ops framework—DataOps, MLOps, or AIOps—by comparing their origins, principles, responsibilities, and tooling, and stresses that cultural principles outweigh technology choices for efficient delivery of data and machine‑learning products.
AIOpsData EngineeringDataOps
0 likes · 12 min read

The article explains the origins and differences of DevOps, DataOps, MLOps and AIOps, outlines their shared seven principles, and provides practical guidance on adopting the right X‑Ops culture to accelerate data‑driven and machine‑learning‑powered software delivery.
AIOpsDataOpsDevOps
0 likes · 11 min read

DataOps, an Agile‑derived methodology that extends DevOps principles to data analytics, emphasizes automation, collaboration, and continuous delivery to accelerate and improve data processing, quality, and business insight, while outlining its benefits, relationship to Agile/DevOps, and practical steps for adoption.
AgileContinuous AnalyticsDataOps
0 likes · 12 min read

The 2020 GOPS Global Operations Conference in Shenzhen showcased a series of high‑impact DevOps talks, standards releases, enterprise assessments, and collaborative initiatives that highlighted the evolving landscape of cloud, data, and operational excellence across leading Chinese tech organizations.
AIOpsDataOpsDevOps
0 likes · 10 min read

This talk walks through the evolution from manual to automated operations, outlines Alibaba’s integrated operations platform architecture, showcases practical DataOps implementations, and explores AIOps applications such as self‑healing monitoring, hardware remediation, and resource optimization, offering concrete examples and insights for modern ops engineers.
AIOpsDataOpsplatform engineering
0 likes · 16 min read

The article reviews Alibaba's operational evolution, tracing stages from manual and script‑based Ops through tool‑centric and platform‑centric DevOps, to data‑driven DataOps and AI‑enhanced AIOps, while highlighting the skill shifts and challenges faced by modern SRE teams.
AIOpsDataOpsDevOps
0 likes · 7 min read

This article explains how Alibaba's MaxCompute platform tackles capacity bottlenecks by combining data‑driven insights, linear programming, and automated project migration strategies to predict resource needs, optimize cluster allocation, and quantify migration impacts for improved operational efficiency.
DataOpsLinear ProgrammingMaxCompute
0 likes · 13 min read
