BestHub
Sign in
Tagged articles

data quality

318 articles · Page 2 of 4
DataFunSummit
DataFunSummit
Jan 1, 2025 · Big Data

Douyin Group Data Asset Management Platform: Full‑Stack Data Lineage Evolution and Applications

This article introduces Douyin Group’s end‑to‑end data asset management platform, explains the evolution and architecture of its large‑scale data lineage system, presents quality metrics and ecosystem components, and outlines practical applications and future directions for data governance, development, and security.

Data Asset PlatformData GovernanceDouyin
0 likes · 16 min read
Douyin Group Data Asset Management Platform: Full‑Stack Data Lineage Evolution and Applications
Huolala Tech
Huolala Tech
Dec 24, 2024 · Information Security

How Huolala Accelerated Risk‑Control Testing with Automated Tools

This article details Huolala's challenges in risk‑control testing amid rapid business growth, outlines the inefficiencies of manual configuration verification, and explains how a suite of automated tools and a full‑scope interception strategy dramatically improved testing efficiency, data quality assurance, and cross‑team collaboration.

Process OptimizationTesting automationdata quality
0 likes · 21 min read
How Huolala Accelerated Risk‑Control Testing with Automated Tools
Model Perspective
Model Perspective
Dec 23, 2024 · Fundamentals

Mastering Mathematical Modeling: 5 Stages & Common Pitfalls to Avoid

From the excitement of first encountering mathematical modeling to becoming a seasoned practitioner, this guide outlines five progressive stages, reveals typical misconceptions at each level, and offers practical advice to help learners avoid common traps and develop both technical and soft skills.

Common pitfallsModel Evaluationdata quality
0 likes · 8 min read
Mastering Mathematical Modeling: 5 Stages & Common Pitfalls to Avoid
Architects' Tech Alliance
Architects' Tech Alliance
Dec 23, 2024 · Artificial Intelligence

Why High‑Quality, Massive, Diverse Data Fuels AI Breakthroughs

The article explains how breakthroughs in artificial intelligence depend on high‑quality, large‑scale, and diverse training data, outlines the data‑centric AI movement, details a six‑step workflow for building datasets, and surveys the data industry ecosystem supporting large language model development.

AI dataAnnotationLarge Language Models
0 likes · 7 min read
Why High‑Quality, Massive, Diverse Data Fuels AI Breakthroughs
ByteDance Data Platform
ByteDance Data Platform
Nov 6, 2024 · Big Data

How Douyin’s Data Platform Overcomes EB‑Scale Metric Challenges

This article explains how Douyin Group tackles massive data volume, quality, and efficiency issues by building a four‑layer intelligent platform, standardizing metric management, automating metric decomposition, and creating reusable metric services that boost agility, stability, and cross‑team collaboration.

Big DataData Engineeringdata platform
0 likes · 20 min read
How Douyin’s Data Platform Overcomes EB‑Scale Metric Challenges
Baobao Algorithm Notes
Baobao Algorithm Notes
Oct 30, 2024 · Artificial Intelligence

How to Choose High-Quality Instruction Data for LLM Fine‑Tuning: Methods Compared

This article surveys and categorizes instruction data selection techniques for large language model fine‑tuning, explaining metric‑based, trainable‑LLM, powerful‑LLM, and small‑model approaches, detailing representative papers, their pipelines, and empirical findings on data quality and diversity.

AI researchInstruction TuningLLM data selection
0 likes · 15 min read
How to Choose High-Quality Instruction Data for LLM Fine‑Tuning: Methods Compared
DataFunSummit
DataFunSummit
Sep 1, 2024 · Artificial Intelligence

Data Management in Large Language Model Training: Overview, Pre‑training, SFT, and Future Challenges

This article surveys data management for large language model training, covering an overview, pre‑training data composition, scaling‑law‑driven quantity control, quality filtering, deduplication, harmful‑content removal, instruction fine‑tuning strategies, dynamic data selection, and emerging research challenges such as bias mitigation, multimodal data handling, and synthetic‑data filtering.

data qualityinstruction fine-tuningpretraining
0 likes · 18 min read
Data Management in Large Language Model Training: Overview, Pre‑training, SFT, and Future Challenges
Data Integration and Governance
Data Integration and Governance
Aug 28, 2024 · Industry Insights

Finally, Data Governance Explained Clearly

The article defines data governance as a comprehensive management system for data assets, explains why a single tool cannot achieve it, compares Pull and Push strategies, details the Pull strategy’s three-step process, and highlights how systematic governance improves data quality and decision‑making.

Data GovernanceData ManagementPull Strategy
0 likes · 11 min read
Finally, Data Governance Explained Clearly
Data Thinking Notes
Data Thinking Notes
Aug 20, 2024 · Artificial Intelligence

How Large AI Models Transform Data Governance: Strategies and Challenges

This article explores how the rise of massive AI models reshapes data governance, detailing model fundamentals, architectural types, emerging challenges, a five‑domain governance framework, and practical AI‑driven applications for data standards, metadata, quality, and security, while also looking ahead to future trends.

AIData Governancedata quality
0 likes · 14 min read
How Large AI Models Transform Data Governance: Strategies and Challenges
DataFunSummit
DataFunSummit
Aug 7, 2024 · Big Data

Ant Group Real-Time Data Warehouse: Architecture, Solutions, and Data Lake Outlook

This article presents Ant Group's recent explorations and practices in real-time data warehousing, detailing its architecture, data quality assurance, stream‑batch integration, and future data lake implementation, while highlighting the use of Flink, ODPS, and Paimon for scalable, low‑latency analytics.

FlinkReal‑time DataStream Processing
0 likes · 15 min read
Ant Group Real-Time Data Warehouse: Architecture, Solutions, and Data Lake Outlook
Data Thinking Notes
Data Thinking Notes
Jun 27, 2024 · Fundamentals

How to Build Effective Data Standards for Enterprise Governance

This article explains the concept of data standards, outlines the three main categories of data standards, describes a four‑stage implementation process, and provides a real‑world bank case study to illustrate how enterprises can establish and apply data standards for better data quality and value.

Data GovernanceData Modelingdata quality
0 likes · 11 min read
How to Build Effective Data Standards for Enterprise Governance
DataFunTalk
DataFunTalk
Jun 23, 2024 · Big Data

Building Full-Chain Data Lineage for E‑commerce Scenarios

This article explains how to construct a full‑chain data lineage system for e‑commerce, covering the concepts of data lineage, the design of a lineage foundation, quality measurement, application‑level lineage, and practical use cases such as table migration, field‑level tracing, and automated metric decomposition.

data lineagedata qualitye-commerce
0 likes · 12 min read
Building Full-Chain Data Lineage for E‑commerce Scenarios
Baidu Tech Salon
Baidu Tech Salon
Jun 12, 2024 · Big Data

Event Tracking Governance: Concepts, Challenges, and Platform Solutions

Event‑tracking governance ensures accurate, consistent user‑behavior data by managing the full lifecycle of logging points through defined quality standards, a digitized workflow, and supporting tools such as rule editors, real‑time testing, and compliance monitoring, while the platform’s page‑scene tree model and metrics improve visibility, reduce duplication, and drive business insight.

Toolinganalyticsdata quality
0 likes · 13 min read
Event Tracking Governance: Concepts, Challenges, and Platform Solutions
Baidu Geek Talk
Baidu Geek Talk
Jun 12, 2024 · Big Data

Event Tracking Governance and Logging Platform Solutions

The article explains event tracking, its data‑quality challenges, and presents a logging platform that enforces quality standards, an end‑to‑end online workflow, and specialized design, testing, and validation tools—including extended field types—to govern, monitor, and improve tracking point compliance across applications.

data qualityevent trackinggovernance
0 likes · 13 min read
Event Tracking Governance and Logging Platform Solutions
Big Data Technology & Architecture
Big Data Technology & Architecture
Jun 4, 2024 · Big Data

Ant Group's Data Governance Practices: Quality, Storage, and Future Directions

This article presents Ant Group's comprehensive data governance experience, covering data quality management, storage governance, architectural design, operational strategies, case studies, and forward‑looking thoughts on integrated lake‑warehouse governance, data value realization, and AI‑driven automation.

Ant GroupBig DataStorage Management
0 likes · 19 min read
Ant Group's Data Governance Practices: Quality, Storage, and Future Directions
DataFunTalk
DataFunTalk
May 25, 2024 · Big Data

Data Quality Governance: From Compliance to Reasonableness and the Quality Review Tool System

This article explains how to assess and improve data quality by moving from simple compliance checks to deeper reasonableness analysis, using visual dashboards, a comprehensive quality‑review tool suite, intelligent judgement rules, self‑diagnosis utilities, and key technical components such as sample libraries and a three‑layer architecture.

data qualityintelligent detectionvisualization
0 likes · 25 min read
Data Quality Governance: From Compliance to Reasonableness and the Quality Review Tool System
Data Thinking Notes
Data Thinking Notes
May 21, 2024 · Fundamentals

Master Data Management: Building a Unified, High‑Quality Data Backbone for Enterprises

Enterprises facing fragmented systems like OA, HR, CRM, and ERP must adopt a unified master data management framework that defines standards, governance, organizational structures, and integrated platforms to ensure data consistency, accuracy, and real‑time availability, thereby reducing maintenance costs and supporting digital transformation.

Data GovernanceEnterprise Architecturedata quality
0 likes · 16 min read
Master Data Management: Building a Unified, High‑Quality Data Backbone for Enterprises
DataFunSummit
DataFunSummit
May 21, 2024 · Operations

Bilibili Data Governance Operational Framework Practice

This article presents Bilibili's practical data governance operational framework, introducing the DAMA‑Bok methodology, detailing two real‑world cases on storage‑level risk and data‑loss post‑mortem, and outlining the organizational, metadata, and embedded governance mechanisms that drive cost and quality improvements.

DAMA-Bokcost governancedata quality
0 likes · 19 min read
Bilibili Data Governance Operational Framework Practice
Data Thinking Notes
Data Thinking Notes
Apr 25, 2024 · Fundamentals

Mastering Data Governance: Build High‑Quality, Secure, Traceable Business Data

This article explains how a comprehensive data governance framework—covering data quality, metadata, master data, asset management, security, and standards—can ensure high‑quality, safe, and traceable business data while outlining implementation steps, organizational roles, platform features, and assessment methods.

Data Governancedata qualitydata security
0 likes · 12 min read
Mastering Data Governance: Build High‑Quality, Secure, Traceable Business Data
dbaplus Community
dbaplus Community
Apr 4, 2024 · Artificial Intelligence

10 Guiding Principles for Building LLM‑Powered Software Applications

This article outlines ten practical principles for designing applications with large language models, emphasizing a model‑first mindset, precision through interactive disambiguation, clear division of code and model responsibilities, data quality, handling uncertainty, and recognizing the limits of LLMs to build robust, maintainable software.

AI designLLMPrompt Engineering
0 likes · 13 min read
10 Guiding Principles for Building LLM‑Powered Software Applications
Baobao Algorithm Notes
Baobao Algorithm Notes
Mar 21, 2024 · Artificial Intelligence

Can the CaR Method Achieve Better LLM Performance with Only 1.4% of Training Data?

This article explains how the CaR (Clustering and Ranking) approach evaluates data quality with a scoring model and selects diverse samples via PCA‑reduced sentence embeddings and K‑Means clustering, achieving comparable or superior large‑model performance while using just 1.96% of the original dataset.

CaR methodClusteringLLM training
0 likes · 8 min read
Can the CaR Method Achieve Better LLM Performance with Only 1.4% of Training Data?
DataFunSummit
DataFunSummit
Mar 18, 2024 · Big Data

Scenario‑Based Data Governance Practices in the Securities Industry

This article presents a comprehensive, scenario-driven data governance practice at Guoxin Securities, covering the industry's pain points, a three‑layer governance framework, detailed implementations for data standards, metadata, data quality, data modeling, and data security, and outlines future directions for intelligent and measurable governance.

Big DataMetadatadata quality
0 likes · 30 min read
Scenario‑Based Data Governance Practices in the Securities Industry
Bitu Technology
Bitu Technology
Mar 15, 2024 · Artificial Intelligence

Monitoring Quality Issues in Tubi’s Recommendation System

This article explains how Tubi monitors the quality of its recommendation system by identifying potential failure points, tracking key data streams such as model input, final recommendation output, and training data, and designing a scalable, real‑time monitoring solution with clear protocols and extensible metrics.

Monitoringdata qualitymachine learning
0 likes · 11 min read
Monitoring Quality Issues in Tubi’s Recommendation System
Aikesheng Open Source Community
Aikesheng Open Source Community
Mar 14, 2024 · Databases

SQLE vs Yearning: Detailed Feature, Architecture, and Use‑Case Comparison

This article provides an in‑depth comparison of the open‑source SQL quality management platforms SQLE and Yearning, covering their architecture, supported data sources, UI design, SQL workbench capabilities, user management, ticket workflow, system settings, and overall suitability for different database environments.

Database ManagementOpen SourceSQL
0 likes · 10 min read
SQLE vs Yearning: Detailed Feature, Architecture, and Use‑Case Comparison
DataFunTalk
DataFunTalk
Mar 1, 2024 · Fundamentals

Data Quality Governance: Overview, Challenges, and Practices

This presentation by Zhou Jie, a senior data R&D expert at Ant Financial, outlines the scope of data governance, examines the challenges of ensuring high‑quality financial data, and shares practical architectures, solutions, and case studies to help attendees understand data quality risks and mitigation strategies.

data qualityfinancial data
0 likes · 2 min read
Data Quality Governance: Overview, Challenges, and Practices
JavaEdge
JavaEdge
Feb 20, 2024 · Big Data

Designing a Scalable Data Quality Center for Offline Big‑Data Pipelines

This article describes the design and implementation of a platform‑wide Data Quality Center for offline big‑data pipelines, covering research of existing solutions, design goals, system architecture based on DolphinScheduler, rule definition language, binding and execution mechanisms, and future enhancements such as lineage monitoring and real‑time checks.

Apache GriffinBig DataDolphinScheduler
0 likes · 18 min read
Designing a Scalable Data Quality Center for Offline Big‑Data Pipelines
DataFunSummit
DataFunSummit
Feb 19, 2024 · Big Data

Yipay Data Warehouse Construction and Data Governance Practices

This presentation by senior data warehouse engineer Huang Luo details Yipay's end‑to‑end data warehouse build, covering background challenges, governance framework, platform development, layered architecture, naming standards, monitoring, and future plans, offering practical insights for data engineers, architects, and business stakeholders.

Big DataData Warehousedata architecture
0 likes · 14 min read
Yipay Data Warehouse Construction and Data Governance Practices
DataFunSummit
DataFunSummit
Feb 3, 2024 · Artificial Intelligence

Practical Application of Large Language Models in MaShang Consumer Finance: From Model Building to Deployment

This article details how MaShang Consumer Finance leverages large language models for sales, collection, and customer service, covering company background, AI research achievements, model training infrastructure, data‑quality and compliance challenges, prompt engineering, inference acceleration, evaluation methods, and lessons learned from real‑world deployment.

LLMModel deploymentPrompt Engineering
0 likes · 21 min read
Practical Application of Large Language Models in MaShang Consumer Finance: From Model Building to Deployment
Data Thinking Notes
Data Thinking Notes
Jan 2, 2024 · Big Data

How a Three-Dimensional Data Governance Model Breaks Silos and Boosts Efficiency

Enterprise data governance faces challenges like information silos, departmental walls, and unclear responsibilities; adopting a three‑dimensional “business‑technology‑organization” framework—setting standards, optimizing processes, and innovating structures—helps eliminate these obstacles, enhance collaboration, improve data quality, and drive cost‑saving, efficiency, and innovation.

Big DataBusiness CollaborationData Governance
0 likes · 10 min read
How a Three-Dimensional Data Governance Model Breaks Silos and Boosts Efficiency
Weimob Technology Center
Weimob Technology Center
Jan 2, 2024 · Big Data

How to Efficiently Test BI Reports in a Hive‑StarRocks Data Warehouse

This article details practical methods for testing BI reports built on Hive and StarRocks, covering the report creation workflow, testing characteristics, SQL writing techniques, impact analysis, data warehouse simplification, and the application of data quality tools to ensure accurate and efficient reporting.

BI testingData WarehouseHive
0 likes · 9 min read
How to Efficiently Test BI Reports in a Hive‑StarRocks Data Warehouse
Data Thinking Notes
Data Thinking Notes
Nov 26, 2023 · Fundamentals

How a Large Enterprise Overcame Master Data Chaos: A Practical Case Study

This article outlines a real‑world enterprise master data project, detailing the definition of master data, the four critical data‑quality challenges faced, the comprehensive solution framework with executive backing, and the six measurable outcomes that improved data governance, efficiency, and decision‑making across the organization.

Data GovernanceEnterprise Architecturedata quality
0 likes · 10 min read
How a Large Enterprise Overcame Master Data Chaos: A Practical Case Study
DataFunSummit
DataFunSummit
Nov 23, 2023 · Information Security

How DCMM Supports Digital Transformation and Data Governance at XCMG Mining Machinery Co., Ltd.

This article details how XCMG Mining Machinery leveraged the DCMM framework to drive digital transformation, improve data governance, address data quality and security challenges, and establish a sustainable data-driven culture across the organization, highlighting the background, implementation steps, lessons learned, and future outlook.

DCMMIndustrial Manufacturingdata quality
0 likes · 25 min read
How DCMM Supports Digital Transformation and Data Governance at XCMG Mining Machinery Co., Ltd.
DataFunSummit
DataFunSummit
Nov 22, 2023 · Big Data

Bilibili Data Quality Assurance System: Architecture, Practices, and Case Study

This article presents Bilibili's data quality assurance system, detailing its evolution across four stages, the architectural framework, core capabilities such as a quality data warehouse, monitoring, collaborative safeguards, digital-driven optimization, and efficient incident handling, along with practical case studies and future outlooks.

Big DataData Warehousedata quality
0 likes · 22 min read
Bilibili Data Quality Assurance System: Architecture, Practices, and Case Study
Data Thinking Notes
Data Thinking Notes
Nov 19, 2023 · Fundamentals

How to Build an Effective Data Asset Management Framework for Enterprises

This article explains why enterprises need a data asset framework, outlines its key components such as catalog management, policy support, and development trends, and provides a step‑by‑step guide with visual diagrams for constructing and operating a comprehensive data asset management system.

Data Governancedata catalogdata quality
0 likes · 5 min read
How to Build an Effective Data Asset Management Framework for Enterprises
DataFunTalk
DataFunTalk
Nov 19, 2023 · Big Data

Design and Evolution of Zhihu's Event‑Tracking (埋点) System

This article presents a comprehensive overview of Zhihu's event‑tracking system, covering its evolution from early Hadoop‑based pipelines to cloud‑native architectures, detailing toolsets for requirement management, validation, data collection, querying, and service design, and concluding with a practical Q&A on best practices and optimization.

Data Pipelinedata qualityevent tracking
0 likes · 12 min read
Design and Evolution of Zhihu's Event‑Tracking (埋点) System
DaTaobao Tech
DaTaobao Tech
Nov 15, 2023 · Industry Insights

Inside the E‑Commerce Product Domain: Roles, Challenges, and Cutting‑Edge Solutions

This article systematically outlines the e‑commerce product team's responsibilities, the users and consumer pain points it addresses, the core technical challenges such as high‑concurrency reads/writes and AI‑driven automation, and the innovative solutions the team has implemented to keep the product domain healthy, efficient, and intelligent.

AIIndustry InsightsProduct Domain
0 likes · 19 min read
Inside the E‑Commerce Product Domain: Roles, Challenges, and Cutting‑Edge Solutions
HomeTech
HomeTech
Nov 15, 2023 · Industry Insights

How to Build Accurate Data Asset Lineage for Data Warehouse Governance

This article explains the challenges of data asset lineage in large data warehouses, presents a comprehensive approach using business‑level instrumentation, SQL interceptor plugins, and ETL script parsing to generate fine‑grained lineage graphs, and demonstrates measurable improvements in coverage and zombie‑table cleanup.

Data GovernanceData WarehouseETL parsing
0 likes · 18 min read
How to Build Accurate Data Asset Lineage for Data Warehouse Governance
Data Thinking Notes
Data Thinking Notes
Nov 14, 2023 · Big Data

How Financial Institutions Master Data Governance for Digital Transformation

This article examines why data governance has become a critical pillar for Chinese financial institutions, outlining external regulations and internal business drivers, describing a comprehensive governance architecture, and presenting a detailed case study of a securities company's data‑asset inventory, platform implementation, and quality management.

Big DataData GovernanceFinancial Services
0 likes · 16 min read
How Financial Institutions Master Data Governance for Digital Transformation
DataFunSummit
DataFunSummit
Nov 14, 2023 · Big Data

Integrated Business‑Finance Data Governance and Its Role in Driving Financial Digital Transformation

This article explains the concept, framework, common data problems, three-stage integration model, implementation methods, and the strategic significance of business‑finance data governance for improving data quality, managing data assets, and accelerating enterprise financial digital transformation.

Financial Digital Transformationbusiness finance integrationdata quality
0 likes · 14 min read
Integrated Business‑Finance Data Governance and Its Role in Driving Financial Digital Transformation
Architects Research Society
Architects Research Society
Nov 13, 2023 · Fundamentals

Why Ongoing Data Maintenance Is Crucial for an Outcome‑Driven Enterprise Data Strategy

The article explains why continuous, proactive data maintenance is essential for an outcome‑driven enterprise data strategy, outlines the risks of poor data quality, and provides practical steps—including business rules, service agreements, KPIs, and ownership—to establish an always‑on data‑maintenance process.

continuous improvementdata maintenancedata quality
0 likes · 7 min read
Why Ongoing Data Maintenance Is Crucial for an Outcome‑Driven Enterprise Data Strategy
Data Thinking Notes
Data Thinking Notes
Nov 5, 2023 · Fundamentals

Why Poor Data Quality Costs Companies $15M Annually and How to Fix It

Low‑quality data can cost enterprises up to $15 million each year, making data quality management essential for accurate decision‑making, compliance, and operational efficiency, and this article explains its importance, evaluation dimensions, common issues, monitoring metrics, responsible roles, and a three‑phase management framework of prevention, control, and remediation.

Big DataBusiness IntelligenceData Governance
0 likes · 32 min read
Why Poor Data Quality Costs Companies $15M Annually and How to Fix It
Data Thinking Notes
Data Thinking Notes
Nov 2, 2023 · Operations

How Bilibili Built a Scalable Data Quality Assurance System for Its Data Warehouse

This article details Bilibili's data quality assurance framework, covering its evolution across four data platform stages, the architecture of its quality data warehouse, core capabilities such as a complete assurance system, digital‑driven continuous optimization, and efficient incident handling, plus case studies, future plans, and a Q&A session.

Big DataBilibiliData Warehouse
0 likes · 27 min read
How Bilibili Built a Scalable Data Quality Assurance System for Its Data Warehouse
Big Data Technology & Architecture
Big Data Technology & Architecture
Oct 23, 2023 · Big Data

Bilibili Data Quality Assurance: Architecture, Goals, Core Capabilities, and Future Outlook

This article outlines Bilibili's data quality assurance framework, detailing its evolution across four development stages, the current data platform architecture, identified pain points, four key quality objectives, core capabilities such as a quality data warehouse, comprehensive monitoring, digital optimization, fault handling, and future directions.

Big DataData GovernanceData Warehouse
0 likes · 22 min read
Bilibili Data Quality Assurance: Architecture, Goals, Core Capabilities, and Future Outlook
Architects Research Society
Architects Research Society
Oct 21, 2023 · Fundamentals

Information Governance: Roles, Responsibilities, and Key Processes

The article explains information governance as a business‑driven program that ensures data accuracy, completeness, consistency, accessibility, and security, outlines three essential roles, describes the data administrator’s duties, and details the key procedures and their relationship to corporate and IT governance.

Data ManagementEnterprise Architecturedata quality
0 likes · 11 min read
Information Governance: Roles, Responsibilities, and Key Processes
Tencent Tech
Tencent Tech
Sep 20, 2023 · Artificial Intelligence

Why Do Large Language Models Hallucinate and How to Reduce It?

The article explains why large language models generate hallucinations—due to data errors, training conflicts, and inference uncertainty—and outlines data‑cleaning, model‑level feedback, knowledge augmentation, constraint techniques, and post‑processing methods such as the “Truth‑seeking” algorithm to mitigate the issue.

AI safetyKnowledge RetrievalLarge Language Models
0 likes · 8 min read
Why Do Large Language Models Hallucinate and How to Reduce It?
Data Thinking Notes
Data Thinking Notes
Sep 17, 2023 · Big Data

How to Overcome Data Fragmentation: A Practical Guide to Enterprise Data Governance

Large enterprises generate massive, diverse data across departments, leading to fragmented storage and trust issues, so this article outlines a comprehensive data governance implementation plan covering standards, metadata, quality, integration, asset, and security management to unify and secure enterprise data.

Big DataData Governancedata fragmentation
0 likes · 4 min read
How to Overcome Data Fragmentation: A Practical Guide to Enterprise Data Governance
Data Thinking Notes
Data Thinking Notes
Sep 3, 2023 · Big Data

How to Build an Effective Data Governance Framework: Steps & Best Practices

This article outlines a comprehensive data governance framework for Chinese enterprises, covering organizational structures, data asset inventory, six‑stage methodology, and the creation of unified data standards and quality rules to support effective digital transformation and data‑driven decision making.

Big DataData GovernanceData Management
0 likes · 13 min read
How to Build an Effective Data Governance Framework: Steps & Best Practices
Data Thinking Notes
Data Thinking Notes
Aug 30, 2023 · Fundamentals

Mastering Data Governance: A Complete Guide to Metadata, Standards, Quality, and Security

Data governance encompasses a comprehensive framework—including metadata, master data, standards, quality, assets, exchange, security, and lifecycle management—to ensure data’s accuracy, consistency, and value across an organization, offering step‑by‑step guidance, best‑practice models, and visual references for effective implementation.

Data Governancedata lifecycledata quality
0 likes · 19 min read
Mastering Data Governance: A Complete Guide to Metadata, Standards, Quality, and Security
DeWu Technology
DeWu Technology
Aug 28, 2023 · Operations

Real-time Data Warehouse Business-Side Chaos Engineering Practice

The article describes how a real‑time data warehouse supporting ad‑delivery metrics adopts both technical and business‑side chaos‑engineering, using red‑blue team drills to inject faults, monitor indicator anomalies, and refine response procedures, thereby enhancing early risk detection, system resilience, and overall data stability for the advertising platform.

Chaos EngineeringData WarehousingMonitoring
0 likes · 16 min read
Real-time Data Warehouse Business-Side Chaos Engineering Practice
DataFunSummit
DataFunSummit
Aug 28, 2023 · Big Data

Building Data Production Pipelines with DataOps: Concepts, Practices, and a Six‑Stage Workflow

This article introduces DataOps, outlines its background and the problems it addresses, describes NetEase’s big‑data product ecosystem, and details a six‑stage data production pipeline—including coding, orchestration, testing, code review, release approval, and deployment – plus insights into two pipeline explorations.

Big DataDataOpsdata quality
0 likes · 15 min read
Building Data Production Pipelines with DataOps: Concepts, Practices, and a Six‑Stage Workflow
Data Thinking Notes
Data Thinking Notes
Aug 13, 2023 · Big Data

How to Successfully Deliver a Data Governance Project: Step‑by‑Step Guide

This article outlines a comprehensive methodology for delivering a data governance project, covering planning, blueprint design, implementation, and acceptance phases, with detailed guidance on team formation, stakeholder roles, requirement analysis, platform architecture, management processes, and post‑deployment operations.

Big DataData GovernanceProject Management
0 likes · 12 min read
How to Successfully Deliver a Data Governance Project: Step‑by‑Step Guide
dbaplus Community
dbaplus Community
Aug 2, 2023 · Backend Development

How WeChat Built a Scalable Security Data Warehouse for Billions of Requests

This article explains the evolution of WeChat's security data warehouse—from its business background and the need for unified feature storage to the architectural designs, multi‑IDC synchronization, operation system, and data‑quality safeguards that enable reliable, high‑performance security policy development for over a trillion daily feature reads and writes.

Feature Managementdata qualityreal-time processing
0 likes · 12 min read
How WeChat Built a Scalable Security Data Warehouse for Billions of Requests
Data Thinking Notes
Data Thinking Notes
Jul 26, 2023 · Big Data

How to Build an Effective Data Asset Catalog for Enterprise Data Governance

This article explains what data assets are, why a data asset catalog is essential for data governance, and provides a step‑by‑step framework—including identification criteria, value dimensions, construction phases, tool support, and core functional modules—to help enterprises systematically create, manage, and leverage a data asset catalog.

Data AssetData Governancedata catalog
0 likes · 16 min read
How to Build an Effective Data Asset Catalog for Enterprise Data Governance
DataFunSummit
DataFunSummit
Jul 15, 2023 · Big Data

Intelligent and Automated Data Quality Management in Big Data Systems

This article explores the challenges of data quality in mature big‑data environments and presents intelligent, automated approaches—including assertions, automatic detection, rule recommendation, link checking, and collaborative mechanisms—to embed quality checks throughout the data pipeline, improving efficiency and reliability.

AutomationData GovernanceData Observability
0 likes · 18 min read
Intelligent and Automated Data Quality Management in Big Data Systems
AntTech
AntTech
Jul 6, 2023 · Industry Insights

Unlocking AI Value: Data Quality, Privacy, and Blockchain in the Smart Era

The article examines how high‑quality data, robust privacy protection, and blockchain‑enabled trust infrastructure are essential for unlocking the value of AI models, citing market forecasts, examples from smart‑car and fintech firms, and the growing Chinese big‑data market through 2026.

AIBig DataBlockchain
0 likes · 9 min read
Unlocking AI Value: Data Quality, Privacy, and Blockchain in the Smart Era
Data Thinking Notes
Data Thinking Notes
Jul 2, 2023 · Big Data

Mastering Data Governance: A Comprehensive Framework for Enterprise Success

This article outlines a complete data governance framework, detailing the five managerial domains—control, process, governance, technology, and value—along with strategies for data strategy, organizational structure, policies, processes, standards, quality, security, and platform tools, and highlights AI’s pivotal role in enhancing governance efficiency.

Big DataData Governancedata quality
0 likes · 10 min read
Mastering Data Governance: A Comprehensive Framework for Enterprise Success
php Courses
php Courses
Jun 30, 2023 · Artificial Intelligence

Notable Real-World Failures of Data and Machine Learning Algorithms Over the Past Decade

Over the past decade, numerous high‑profile incidents have shown that flawed data and machine‑learning algorithms can cause severe consequences, from legal mishaps with ChatGPT to biased medical diagnoses, inaccurate real‑estate pricing, and discriminatory hiring practices, underscoring the need for rigorous data validation and algorithmic fairness.

AI ethicsalgorithm biascase studies
0 likes · 3 min read
Notable Real-World Failures of Data and Machine Learning Algorithms Over the Past Decade
Tencent Cloud Developer
Tencent Cloud Developer
Jun 29, 2023 · Industry Insights

How WeChat Built a Scalable Security Data Warehouse: Architecture, Evolution, and Data‑Quality Practices

This article examines the origins, architectural evolution, storage choices, unified access layer, multi‑IDC synchronization, operational tooling, and data‑quality mechanisms of WeChat's security data warehouse, illustrating how centralized feature management and rigorous quality checks enable reliable, high‑performance security policy enforcement at massive scale.

Feature Managementarchitecturedata quality
0 likes · 16 min read
How WeChat Built a Scalable Security Data Warehouse: Architecture, Evolution, and Data‑Quality Practices
Architects Research Society
Architects Research Society
Jun 6, 2023 · Fundamentals

Information Governance: Roles, Responsibilities, and Key Processes

This article explains information governance as a program that ensures data accuracy, completeness, consistency, accessibility, and security across an enterprise, outlines the three essential business‑oriented roles—Data Governance Committee, Data Steward, and Data Custodian—describes their duties, and details the key procedures, metrics, and relationships with corporate and IT governance.

data qualitydata stewardshipenterprise data management
0 likes · 11 min read
Information Governance: Roles, Responsibilities, and Key Processes
Data Thinking Notes
Data Thinking Notes
May 14, 2023 · Big Data

Why Data Governance Matters: Boosting Data Quality and Business Value

Data governance, the overarching framework for evaluating, guiding, and supervising an organization’s data lifecycle—from collection to utilization—ensures high data quality, compliance, and security, ultimately maximizing data value and supporting AI-driven initiatives, while distinguishing itself from data management and data control through a strategic, top‑down approach.

Big DataData GovernanceData Management
0 likes · 8 min read
Why Data Governance Matters: Boosting Data Quality and Business Value
DataFunSummit
DataFunSummit
May 13, 2023 · Big Data

Expert Interview on Data Governance: Core Domains, Challenges, and Future Trends

In this interview, three data‑governance experts from Tencent, ByteDance, and Alibaba discuss the fundamental processes, core domains such as metadata, data lineage, metric systems, data quality and security, the main challenges they face, and emerging trends like DataOps, AI‑driven automation, and privacy‑preserving technologies.

Data GovernanceDataOpsdata lineage
0 likes · 14 min read
Expert Interview on Data Governance: Core Domains, Challenges, and Future Trends
DataFunTalk
DataFunTalk
May 7, 2023 · Big Data

Data Standards and Data Quality: Concepts, Frameworks, Tools, and Case Studies

This article presents a comprehensive overview of data standards and data quality, covering core concepts and frameworks, practical tools and techniques, real‑world case studies, and a detailed Q&A that together illustrate how organizations can govern, measure, and improve the reliability of their data assets.

data qualitydata quality toolsdata standards
0 likes · 22 min read
Data Standards and Data Quality: Concepts, Frameworks, Tools, and Case Studies
Data Thinking Notes
Data Thinking Notes
Apr 25, 2023 · Operations

Why Data Quality Matters: A Practical Guide to Governance and Seven‑Dimensional Evaluation

This article explains why data quality is critical for businesses, outlines common data quality problems, their root causes, and presents a comprehensive governance framework—including monitoring rules, alerting, full‑link monitoring, and a seven‑dimensional evaluation model—to ensure high‑quality data delivery.

Big DataData GovernanceOperations
0 likes · 12 min read
Why Data Quality Matters: A Practical Guide to Governance and Seven‑Dimensional Evaluation
DataFunSummit
DataFunSummit
Apr 23, 2023 · Fundamentals

Data Governance Practices and Implementation Path at Dipu Technology

This article presents Dipu Technology's comprehensive data governance methodology, covering construction paths, a typical enterprise digital platform framework, core governance components, practical case studies, and a Q&A session that together illustrate how businesses can design, implement, and sustain effective data governance across the organization.

Data GovernanceData ManagementEnterprise Architecture
0 likes · 19 min read
Data Governance Practices and Implementation Path at Dipu Technology
Big Data Technology & Architecture
Big Data Technology & Architecture
Apr 17, 2023 · Big Data

Comprehensive Guide to Data Governance and Data Asset Management

This article presents a detailed roadmap for enterprise data governance, covering business digitization goals, data governance construction, typical digital platform architecture, core governance actions, implementation pathways, data asset inventory techniques, and real‑world case studies to illustrate practical execution.

Big DataData GovernanceData Modeling
0 likes · 18 min read
Comprehensive Guide to Data Governance and Data Asset Management
Data Thinking Notes
Data Thinking Notes
Apr 16, 2023 · Big Data

Mastering Data Asset Management: From Inventory to Value Realization

This article outlines a complete data asset management lifecycle—starting with data inventory, moving through governance, classification, responsibility, permission, and security, and culminating in value realization via basic services, profiling, and algorithmic models—providing practical guidance for building a robust big‑data platform.

Big DataData Governancedata quality
0 likes · 10 min read
Mastering Data Asset Management: From Inventory to Value Realization
Data Thinking Notes
Data Thinking Notes
Apr 9, 2023 · Big Data

Why Data Quality Is the Hidden Driver of Big Data Success

In the big‑data era, high‑quality data are essential for reliable analytics, and this article explains data‑quality concepts, key dimensions, analysis methods for missing values, outliers, inconsistencies and duplicates, as well as practical management practices to ensure data assets become a competitive advantage.

Big DataData AnalysisData Governance
0 likes · 15 min read
Why Data Quality Is the Hidden Driver of Big Data Success
DataFunTalk
DataFunTalk
Apr 8, 2023 · Fundamentals

Data Governance Practices and Pathways: Insights from DeepEx Technology

This article outlines DeepEx Technology's comprehensive data governance methodology, covering construction paths, digital platform frameworks, core governance components, implementation steps, case studies, and a Q&A that together illustrate how enterprises can build reliable data assets, models, standards, and quality processes to unlock business value.

Data Modelingdata assetsdata quality
0 likes · 21 min read
Data Governance Practices and Pathways: Insights from DeepEx Technology
Data Thinking Notes
Data Thinking Notes
Apr 5, 2023 · Big Data

Mastering Data Governance: From Challenges to End‑to‑End Solutions

This article explores the key problems data governance aims to solve, outlines a comprehensive governance framework, and details practical implementation steps—including tool integration, metadata management, lake‑in and lake‑out processes, and governance policies—to achieve a closed‑loop, value‑driven data ecosystem.

Big DataData GovernanceData Lake
0 likes · 13 min read
Mastering Data Governance: From Challenges to End‑to‑End Solutions
Aikesheng Open Source Community
Aikesheng Open Source Community
Apr 3, 2023 · Databases

SQL Quality Management with Open-Source SQLE: Insights from the 2023 DAMS China Data Intelligence Management Summit

The 2023 DAMS China Data Intelligence Management Summit in Shanghai featured a technical presentation by Zhang Shenbo on an open‑source SQLE solution for SQL quality control, covering multi‑database auditing, automated review workflows, and practical tips to reduce DBA workload and cross‑department communication.

Data GovernanceDatabase ManagementOpen Source
0 likes · 3 min read
SQL Quality Management with Open-Source SQLE: Insights from the 2023 DAMS China Data Intelligence Management Summit
Data Thinking Notes
Data Thinking Notes
Apr 2, 2023 · Fundamentals

Transforming Bank Data: A Practical Guide to Data Governance and Quality Management

This article explains how modern commercial banks can turn massive operational data into a strategic asset by building a comprehensive data governance framework that addresses data standards, quality management, metadata, master data, and security, while outlining a six‑step methodology for continuous improvement.

BankingData Governancedata quality
0 likes · 18 min read
Transforming Bank Data: A Practical Guide to Data Governance and Quality Management
DataFunTalk
DataFunTalk
Mar 31, 2023 · Big Data

Bilibili's Big Data Development Governance Platform: Architecture, Challenges, and Strategies

This article presents an in‑depth overview of Bilibili’s big data development governance platform, detailing its architecture, the pain points of platform construction, data‑driven methodology, product strategy, and practical solutions for data integration, quality, cost, and security governance across large‑scale data operations.

Bilibilicost optimizationdata quality
0 likes · 35 min read
Bilibili's Big Data Development Governance Platform: Architecture, Challenges, and Strategies
Data Thinking Notes
Data Thinking Notes
Mar 26, 2023 · Big Data

Why Data Governance Is the Key to Unlocking Your Data’s True Value

This article explains how effective data governance transforms raw data into a trusted enterprise asset, outlines common pitfalls such as backward and passive governance, and presents a structured, four‑phase approach—including organizational setup, standards, platform selection, and continuous operations—to successfully implement data governance at scale.

Big DataData GovernanceData Management
0 likes · 10 min read
Why Data Governance Is the Key to Unlocking Your Data’s True Value
Volcano Engine Developer Services
Volcano Engine Developer Services
Mar 22, 2023 · Fundamentals

How ByteDance Scales Data Governance: Challenges, Distributed Solutions, and Best Practices

This article examines ByteDance's data governance journey, outlining business, organizational, and cultural challenges, the six-stage evolution framework, real‑world case studies, and the shift from centralized to distributed autonomous governance to improve quality, security, cost, and team efficiency.

Big DataData GovernanceDistributed Architecture
0 likes · 18 min read
How ByteDance Scales Data Governance: Challenges, Distributed Solutions, and Best Practices
Data Thinking Notes
Data Thinking Notes
Mar 19, 2023 · Big Data

Why Data Quality Is the Key to Successful Big Data Initiatives

The article explains that while big data aims to boost organizational insight and innovation, its true value depends on high data quality, outlines industry standards, identifies technical, business, and management causes of poor quality, and proposes a three‑phase strategy of prevention, monitoring, and post‑improvement to ensure reliable data for decision‑making.

Big DataData Governancedata quality
0 likes · 21 min read
Why Data Quality Is the Key to Successful Big Data Initiatives
Data Thinking Notes
Data Thinking Notes
Mar 8, 2023 · Fundamentals

How BI Portals Transform Enterprise Data Governance for Scalable Analytics

This whitepaper explains why effective BI governance is essential for modern enterprises, outlines the key capabilities of data‑governance tools—including data quality, certification, usage statistics, classification, lineage, glossary, and lifecycle management—and shows how BI portals and data catalogs together enable scalable, user‑centric analytics.

BI governanceBI portalData Governance
0 likes · 12 min read
How BI Portals Transform Enterprise Data Governance for Scalable Analytics
Big Data Technology & Architecture
Big Data Technology & Architecture
Mar 3, 2023 · Fundamentals

Understanding Data Management Principles and Governance: Insights from DMBOK

This article explains the core principles, strategies, frameworks, and governance practices of data management based on DAMA's DMBOK, covering data lifecycle, value, leadership responsibilities, strategic planning, governance models, metrics, and implementation guidelines to help organizations derive business value from high‑quality data.

Business ValueDMBOKData Governance
0 likes · 17 min read
Understanding Data Management Principles and Governance: Insights from DMBOK
DataFunSummit
DataFunSummit
Feb 2, 2023 · Big Data

Data Governance Strategies: Concepts, Practices, and Case Studies

The article explains why data is a critical corporate asset, distinguishes narrow and broad data‑governance approaches, outlines strategic principles such as treating governance as a systematic, prioritized effort, and presents eight real‑world case studies from companies like Tencent, SF Tech, Huolala, and NetEase.

case studiesdata qualitymetadata management
0 likes · 7 min read
Data Governance Strategies: Concepts, Practices, and Case Studies
Data Thinking Notes
Data Thinking Notes
Jan 31, 2023 · Fundamentals

Mastering Data Governance: From Metadata to ETL in One Guide

This comprehensive guide walks you through the entire data governance ecosystem, covering metadata fundamentals, classification, maturity models, data standards, modeling, integration, lifecycle management, quality assurance, security, and ETL processes, all illustrated with clear diagrams and practical steps.

Data GovernanceData IntegrationData Modeling
0 likes · 13 min read
Mastering Data Governance: From Metadata to ETL in One Guide
DataFunTalk
DataFunTalk
Jan 25, 2023 · Artificial Intelligence

Between Heaven and Earth: Reflections of an Algorithm Engineer

The article argues that algorithm engineers should move beyond a narrow focus on deep‑learning models, emphasizing the importance of system architecture, data quality, and thoughtful problem framing to break through performance plateaus in advertising and recommendation systems.

AdvertisingAlgorithm EngineeringRecommendation Systems
0 likes · 10 min read
Between Heaven and Earth: Reflections of an Algorithm Engineer
DataFunSummit
DataFunSummit
Jan 24, 2023 · Big Data

Building a Real-Time Data and User Profiling Architecture with Apache Doris at Zhihu

The article details Zhihu's data empowerment team's design and implementation of a low‑cost, high‑response real‑time data platform built on Apache Doris, covering real‑time business metrics, algorithm features, and user profiling, and explains the challenges, architectural choices, tooling, performance gains, and future directions.

Apache DorisData IntegrationLambda Architecture
0 likes · 22 min read
Building a Real-Time Data and User Profiling Architecture with Apache Doris at Zhihu
Huolala Tech
Huolala Tech
Jan 16, 2023 · Big Data

How Leading Logistics Companies Master Data Governance for Cost and Stability

At the 2022 DataFun Summit, data governance experts from Huolala, Zhongtong, and SF Express shared comprehensive practices—including governance drivers, quality monitoring, model management, master data processes, platform architecture, cost control, and stability measures—illustrating how large logistics firms implement end‑to‑end data governance to boost efficiency, compliance, and business value.

Big DataCost ManagementData Governance
0 likes · 13 min read
How Leading Logistics Companies Master Data Governance for Cost and Stability
DataFunTalk
DataFunTalk
Jan 13, 2023 · Big Data

Data Governance Strategies and Practices: Insights from Leading Companies

The article explains the importance of data governance for organizations handling big data, distinguishes narrow and broad governance approaches, outlines strategic principles, and presents case studies from companies like Tencent, SF Tech, Huolala, and NetEase to illustrate effective governance practices.

Case studyMetadatadata quality
0 likes · 8 min read
Data Governance Strategies and Practices: Insights from Leading Companies