Skip to content
Tagged articles

data integration

327 articles · Page 2 of 4
Data Integration and Governance
Data Integration and Governance
Dec 1, 2025 · Fundamentals

Why Data Standards Fail and How to Implement Them in Four Practical Steps

The article explains why data standards often stall, defines the essential elements of a usable standard, and presents a concrete four‑step framework—including team formation, high‑value pilot selection, detailed gap analysis, and joint review—to turn standards into enforceable, business‑driving rules.

Business AnalyticsData Managementdata governance
0 likes · 11 min read
Why Data Standards Fail and How to Implement Them in Four Practical Steps
DataFunSummit
DataFunSummit
Nov 9, 2025 · Artificial Intelligence

How Zilliz Cut an 8‑Minute Sales Lead Process to Seconds with AI‑Powered Dify

This article recounts how Zilliz leveraged the low‑code platform Dify to integrate large‑model AI, private data, and business logic, transforming an eight‑minute, manual sales‑lead workflow into a seconds‑level automated pipeline and illustrating a new human‑AI collaboration paradigm.

AISales Enablementdata integration
0 likes · 14 min read
How Zilliz Cut an 8‑Minute Sales Lead Process to Seconds with AI‑Powered Dify
BirdNest Tech Talk
BirdNest Tech Talk
Oct 11, 2025 · Artificial Intelligence

How to Load Documents into LangChain: From Files to APIs

Learn how to use LangChain's Document Loaders to import data from files, web pages, databases, and APIs, understand the Document object structure, compare load() versus lazy_load(), and follow a step‑by‑step Python example that demonstrates loading, inspecting, and optionally processing documents with an LLM.

Document LoaderLLMLangChain
0 likes · 12 min read
How to Load Documents into LangChain: From Files to APIs
Smart Sea Tide
Smart Sea Tide
Sep 29, 2025 · Databases

Data Warehouse Architecture and Construction Methodology

The article explains why scattered, heterogeneous enterprise data requires a unified, subject‑oriented data warehouse, outlines its historical evolution, compares Inmon and Kimball architectures, describes the core components and modeling stages, and details dimensional modeling techniques and best practices.

Business IntelligenceETLInmon
0 likes · 48 min read
Data Warehouse Architecture and Construction Methodology
DataFunTalk
DataFunTalk
Aug 26, 2025 · Artificial Intelligence

Exploring Cutting-Edge AI & Knowledge Graph Applications: A Curated Resource Guide

This resource guide presents a curated list of cutting‑edge topics—including multimodal GraphRAG, knowledge‑graph‑driven large‑model applications in finance, traditional Chinese medicine, automotive manufacturing, and knowledge‑management trends—offering insights into AI‑powered knowledge services, and invites readers to scan the QR code to download the full e‑book.

AIdata integrationknowledge graph
0 likes · 2 min read
Exploring Cutting-Edge AI & Knowledge Graph Applications: A Curated Resource Guide
360 Tech Engineering
360 Tech Engineering
Aug 12, 2025 · Artificial Intelligence

How Knowledge Graphs Are Reinventing AI Security: Insights from ISC.AI 2025

At the 13th ISC.AI 2025 Knowledge Graphs Reshaping Intelligent Security Summit in Beijing, leading experts from academia and industry highlighted how knowledge graphs enhance AI model accuracy, explainability, and trust, offering comprehensive data integration and risk monitoring to fortify intelligent systems across sectors.

data integrationknowledge graphrisk monitoring
0 likes · 6 min read
How Knowledge Graphs Are Reinventing AI Security: Insights from ISC.AI 2025
360 Zhihui Cloud Developer
360 Zhihui Cloud Developer
Jul 22, 2025 · Big Data

How Apache SeaTunnel Revolutionizes Heterogeneous Data Integration with Decoupled Connectors

This article explores how Apache SeaTunnel addresses modern data integration challenges by providing a high‑performance, distributed, plugin‑based platform that decouples connectors from execution engines, enabling seamless batch and streaming synchronization across heterogeneous sources such as databases, message queues, and data lakes.

Apache SeaTunnelConnector Architecturebatch processing
0 likes · 24 min read
How Apache SeaTunnel Revolutionizes Heterogeneous Data Integration with Decoupled Connectors
Code Ape Tech Column
Code Ape Tech Column
Jul 8, 2025 · Backend Development

Mastering Spring Batch: Real-World Use Cases and Hands‑On Guide

This comprehensive guide explains why batch processing is essential, walks through typical banking, e‑commerce, logging and medical data scenarios, details Spring Batch's core architecture and components, provides step‑by‑step setup and code examples, and presents a production‑grade bank reconciliation case with monitoring and troubleshooting tips.

Performance OptimizationSpring BatchSpring Boot
0 likes · 27 min read
Mastering Spring Batch: Real-World Use Cases and Hands‑On Guide
Alibaba Cloud Developer
Alibaba Cloud Developer
Jun 13, 2025 · Artificial Intelligence

Accelerate Enterprise Data Insights with Alibaba Cloud Hologres and AI Agents

Learn how to rapidly build an intelligent data analysis agent by integrating multi‑source data through Alibaba Cloud Hologres, leveraging Bailei’s AI model service and the serverless Function AI platform, covering architecture, step‑by‑step deployment, verification, and resource cleanup for cost‑effective, real‑time business insights.

AIAlibaba CloudHologres
0 likes · 8 min read
Accelerate Enterprise Data Insights with Alibaba Cloud Hologres and AI Agents
Java Captain
Java Captain
Jun 10, 2025 · Backend Development

Why Spring Batch? Real‑World Scenarios, Core Architecture and Hands‑On Guide

This article explains the necessity of batch processing, presents typical use cases such as daily interest calculation, e‑commerce order archiving, log analysis and medical data migration, then dives deep into Spring Batch's core components, provides step‑by‑step code examples, performance‑tuning tips, production‑grade fault‑tolerance, monitoring solutions and a comprehensive FAQ.

JavaSpring Batchbatch processing
0 likes · 20 min read
Why Spring Batch? Real‑World Scenarios, Core Architecture and Hands‑On Guide
DataFunSummit
DataFunSummit
Jun 2, 2025 · Artificial Intelligence

Enterprise Knowledge Brain Powered by Large Models and Knowledge Graphs

This article explains how the rapid development of large language models and knowledge graph technologies creates new opportunities for enterprise knowledge management, outlines the challenges of massive unstructured data, describes the architecture and core data flow of a corporate knowledge brain, and showcases key technologies and real‑world applications.

AI Architecturedata integrationenterprise AI
0 likes · 13 min read
Enterprise Knowledge Brain Powered by Large Models and Knowledge Graphs
Data Thinking Notes
Data Thinking Notes
May 5, 2025 · Artificial Intelligence

How MCP’s Text2SQL Service Turns Natural Language into Powerful Database Queries

This article explores the MCP platform’s data service capabilities, detailing its core components—Resources, Prompts, and Tools—and demonstrates how its Text2SQL feature enables natural‑language queries to retrieve table schemas, perform data sampling, and execute complex relational analyses across multiple database tables.

AILLMMCP
0 likes · 7 min read
How MCP’s Text2SQL Service Turns Natural Language into Powerful Database Queries
Big Data Tech Team
Big Data Tech Team
Apr 21, 2025 · Industry Insights

8 Practical Ways DeepSeek Boosts Data Quality for Better Governance

This guide outlines eight concrete methods DeepSeek uses to improve data quality—including automated cleaning, validation, classification, monitoring, governance standards, anomaly detection, integration, and intelligent analysis—providing actionable steps for organizations to enhance data accuracy, completeness, consistency, and usability.

DeepSeekdata cleaningdata integration
0 likes · 5 min read
8 Practical Ways DeepSeek Boosts Data Quality for Better Governance
DataFunSummit
DataFunSummit
Apr 1, 2025 · Big Data

Understanding Flink CDC 3.3: Features, Improvements, and Future Plans

This article provides a comprehensive overview of Flink CDC 3.3, detailing its CDC fundamentals, new connectors, Transform module enhancements, asynchronous snapshot splitting, community adoption, and upcoming roadmap for broader ecosystem support and batch‑mode execution.

CDCFlinkbig data
0 likes · 15 min read
Understanding Flink CDC 3.3: Features, Improvements, and Future Plans
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Mar 25, 2025 · Big Data

How to Connect EMR Serverless Spark with Apache Doris for Seamless Data Processing

This guide explains how to integrate EMR Serverless Spark with the high‑performance Apache Doris analytical database, covering prerequisites, connector download, OSS upload, network configuration, table creation, and both SQL‑session and Notebook examples for reading and writing Doris tables.

Apache DorisEMR Serverless SparkSpark Connector
0 likes · 11 min read
How to Connect EMR Serverless Spark with Apache Doris for Seamless Data Processing
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Mar 20, 2025 · Big Data

How to Read and Write StarRocks Data with EMR Serverless Spark

This step‑by‑step guide explains how to use EMR Serverless Spark together with the StarRocks Spark Connector to create a workspace, upload the connector JAR, configure network connections, create databases and tables in StarRocks, and perform read/write operations via SQL sessions, Notebook sessions, or batch Spark jobs, complete with code examples and UI screenshots.

EMR ServerlessSparkStarRocks
0 likes · 14 min read
How to Read and Write StarRocks Data with EMR Serverless Spark
Data Practical Notes
Data Practical Notes
Mar 10, 2025 · Industry Insights

3D Visualization Dashboards: 5 Core Scenarios from Smart Cities to Industrial Monitoring

This article traces the evolution from 2D reports to 3D visualization dashboards, outlines four core values — data integration, intuitive display, real-time monitoring, and collaboration support — and details five high-value application scenarios: smart city management, industrial production monitoring, energy optimization, healthcare monitoring, and smart campus education, along with implementation strategies for data preparation, visualization design, and system integration.

3D visualizationData Dashboarddata integration
0 likes · 13 min read
3D Visualization Dashboards: 5 Core Scenarios from Smart Cities to Industrial Monitoring
Data Practical Notes
Data Practical Notes
Mar 4, 2025 · Industry Insights

What Is 'One Enterprise One Screen'? Core Concept, Pain Points, and the Shift to Data-Driven Management

This article explains the 'One Enterprise One Screen' concept — a customized dashboard aggregating real-time operational data for holistic oversight — detailing why enterprises need it, the pain points of data silos and delayed insights, how integration and low-code tools like FineVis enable implementation, and case studies from Shenzhen Metro and government regulators.

Business IntelligenceData DashboardEnterprise Data Visualization
0 likes · 14 min read
What Is 'One Enterprise One Screen'? Core Concept, Pain Points, and the Shift to Data-Driven Management
Data Practical Notes
Data Practical Notes
Feb 20, 2025 · Big Data

Building a Data Middle Platform from Scratch: Architecture, Implementation Phases, and Pitfall Avoidance

This article provides a comprehensive guide to designing and implementing a data middle platform, covering its core essence as a business-data fusion mechanism, six architectural modules from ingestion to performance optimization, three implementation phases from requirements to applications, and key pitfalls like business-technology misalignment and over-engineering.

Data ModelingFineDataLinkFineVis
0 likes · 15 min read
Building a Data Middle Platform from Scratch: Architecture, Implementation Phases, and Pitfall Avoidance
AI Product Manager Community
AI Product Manager Community
Feb 17, 2025 · Product Management

How AI Can Transform Your Product Roadmap into a Real‑Time Strategic Tool

In today’s fast‑changing market, traditional product planning falls short, so this article explains how AI‑powered data integration, predictive analytics, and dynamic feedback loops can create a real‑time, data‑driven product roadmap, detailing three implementation phases—data unification, intelligent analysis, and continuous adjustment—with practical steps for product managers.

AIdata integrationpredictive analytics
0 likes · 8 min read
How AI Can Transform Your Product Roadmap into a Real‑Time Strategic Tool
Data Practical Notes
Data Practical Notes
Feb 9, 2025 · Big Data

Data Integration Evolution: 7 Shifts from Tool-Centric to Value-Driven in the Data Element Era

This article traces the evolution of data integration from point-to-point and ETL to modern lakehouse architectures, detailing seven key shifts—tool-to-value orientation, downstream-to-upstream integration, manual-to-automated pipelines, professional-to-citizen access, siloed-to-ecosystem scope, batch-to-real-time processing, and isolated-to-unified storage—driven by data elementization trends.

ETLdata elementizationdata governance
0 likes · 12 min read
Data Integration Evolution: 7 Shifts from Tool-Centric to Value-Driven in the Data Element Era
Architecture Practice Methods
Architecture Practice Methods
Feb 5, 2025 · Fundamentals

Data Architecture: Standard Definitions, Challenges, and Optimization Paths

This article defines data architecture per ISO/IEC/IEEE 42010 and DCMM, outlines core components — data model, flow, metadata, integration — identifies four key challenges including siloed management and technology-first approaches, and proposes optimization paths: enterprise-wide architectural governance, layered stable design, and collaborative data governance.

DCMMData Modelingdata architecture
0 likes · 8 min read
Data Architecture: Standard Definitions, Challenges, and Optimization Paths
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Jan 27, 2025 · Big Data

Unlock Real-Time Data Sync with Flink CDC: YAML Integration, Transform & Route Explained

This article summarizes an advanced Flink CDC presentation, covering Flink CDC fundamentals, real‑time Flink integration, CDC‑YAML core capabilities, supported sync links, Transform and Route modules, monitoring metrics, schema‑change strategies, typical use cases, performance optimizations, demo implementations, and future development plans.

CDCFlinkReal-time
0 likes · 20 min read
Unlock Real-Time Data Sync with Flink CDC: YAML Integration, Transform & Route Explained
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Jan 23, 2025 · Big Data

How Alibaba Cloud DataWorks Leverages Flink CDC for Scalable Data Lake Integration

Alibaba Cloud DataWorks’ Data Integration platform, built on Flink CDC, offers a comprehensive, serverless solution for real‑time and batch data lake ingestion, detailing its architecture, elastic scaling, productized use cases, and future roadmap, including AI‑driven diagnostics and expanded source support.

Elastic ScalingFlink CDCbig data
0 likes · 12 min read
How Alibaba Cloud DataWorks Leverages Flink CDC for Scalable Data Lake Integration
Bilibili Tech
Bilibili Tech
Nov 26, 2024 · Big Data

Bilibili’s Iceberg‑Based Streaming‑Batch Integration: Architecture, Optimizations, and Practices

Bilibili migrated its massive user‑behavior, commercial AI training, and database synchronization pipelines from Hive and Kafka to an Iceberg‑based streaming‑batch architecture, using Flink and the Magnus optimizer to achieve minute‑level freshness, reduce CPU and memory usage by about 20‑22 %, save roughly 3.55 M CNY annually, and dramatically improve query latency and join performance.

FlinkIcebergOptimization
0 likes · 20 min read
Bilibili’s Iceberg‑Based Streaming‑Batch Integration: Architecture, Optimizations, and Practices
Data Thinking Notes
Data Thinking Notes
Nov 5, 2024 · Big Data

How a Next‑Gen Data Management Platform Boosts Efficiency and Innovation

This article outlines the motivations, objectives, and architectural design of a next‑generation data management platform, detailing its four‑layer “four‑ization” approach, core services such as data integration, modeling, API provisioning, componentization, as well as governance, security, and operational best practices.

big datadata governancedata integration
0 likes · 20 min read
How a Next‑Gen Data Management Platform Boosts Efficiency and Innovation

How Apache SeaTunnel Redefines Data Integration for Modern Data Platforms

This article reviews the evolution of data‑integration architectures toward EtLT, explains the core capabilities of Apache SeaTunnel, and details how a Chinese data‑platform vendor applied and extended SeaTunnel to simplify batch and streaming ingestion, unify multi‑engine processing, and reduce development and operational costs.

Apache SeaTunnelConnector DevelopmentDistributed Processing
0 likes · 17 min read
How Apache SeaTunnel Redefines Data Integration for Modern Data Platforms
DataFunSummit
DataFunSummit
Nov 1, 2024 · Big Data

DataFun Summit Session Overview and E‑book Access Instructions

The article outlines how to obtain the DataFun Summit e‑book by following the public account instructions and provides concise English summaries of twelve technical sessions covering data lineage, integration, AI language models, multimodal content, game AI agents, lake‑warehouse governance, big‑data architecture, and cluster management.

AIDataOpsbig data
0 likes · 5 min read
DataFun Summit Session Overview and E‑book Access Instructions
DataFunSummit
DataFunSummit
Oct 27, 2024 · Artificial Intelligence

How Siemens Harnesses Generative AI to Build the Enterprise Knowledge Chatbot “XiaoYu”

This article describes Siemens' journey in applying generative AI and Retrieval‑Augmented Generation to create an internal knowledge chatbot, detailing the business challenges, technical architecture, data integration, multi‑modal capabilities, deployment outcomes, and strategic lessons for enterprise AI adoption.

AI chatbotEnterprise Knowledge ManagementGenerative AI
0 likes · 21 min read
How Siemens Harnesses Generative AI to Build the Enterprise Knowledge Chatbot “XiaoYu”
macrozheng
macrozheng
Sep 27, 2024 · Big Data

Master DataX: Efficient Offline Data Sync for Heterogeneous Sources

This guide walks through the challenges of synchronizing massive datasets across heterogeneous databases, introduces Alibaba's open‑source DataX tool, explains its framework‑plugin architecture, and provides step‑by‑step instructions—including environment setup, installation, job configuration, and both full and incremental MySQL synchronization—complete with code examples and performance metrics.

DataXETLIncremental Sync
0 likes · 15 min read
Master DataX: Efficient Offline Data Sync for Heterogeneous Sources
Data Thinking Notes
Data Thinking Notes
Sep 9, 2024 · Fundamentals

Master the 6‑Step Blueprint for Building an Enterprise Data Middle Platform

This guide outlines a practical six‑step methodology—covering overall planning, data integration, model construction, data development, asset management, and data services—to help enterprises build a robust data middle platform that unlocks business value and supports agile digital transformation.

Data Modelingdata governancedata integration
0 likes · 10 min read
Master the 6‑Step Blueprint for Building an Enterprise Data Middle Platform
Data Thinking Notes
Data Thinking Notes
Jul 29, 2024 · Big Data

What Is a Data Middle Platform and How Does It Transform Enterprise Data Management?

This article explains the concept, design principles, and core components of a data middle platform, detailing its overall, functional, layered, logical, and data architectures, as well as the specific platforms for data collection, processing, organization, governance, quality, sharing, and visualization, illustrated with diagrams.

big datadata architecturedata governance
0 likes · 27 min read
What Is a Data Middle Platform and How Does It Transform Enterprise Data Management?
DaTaobao Tech
DaTaobao Tech
Jul 8, 2024 · Big Data

ODPS (MaxCompute) SQL Basics, Data Integration and Hologres Import Guide

This guide provides a comprehensive, beginner‑to‑advanced reference for ODPS (MaxCompute) SQL, covering table creation, DDL/DML commands, query syntax, join hints, MySQL‑to‑ODPS synchronization, one‑click and custom imports into Hologres, and scheduling variables for automated data pipelines.

ETLHologresODPS
0 likes · 37 min read
ODPS (MaxCompute) SQL Basics, Data Integration and Hologres Import Guide
DataFunSummit
DataFunSummit
Jun 14, 2024 · Big Data

JD Logistics One‑Stop Agile BI Solution: Architecture, Challenges, and Product Evolution

This article presents JD Logistics' one‑stop agile BI platform, detailing the complex data sources, rapid business demands, the UData solution architecture, performance and usability improvements, and future upgrade plans that together enable faster data integration, self‑service reporting, and enhanced decision‑making across the organization.

Agile AnalyticsBIPerformance Optimization
0 likes · 25 min read
JD Logistics One‑Stop Agile BI Solution: Architecture, Challenges, and Product Evolution
DataFunTalk
DataFunTalk
May 13, 2024 · Big Data

Data Integration Maturity Model: From ETL to EtLT

The article examines the evolution of data integration architectures—from traditional ETL through ELT to the emerging EtLT model—highlighting their advantages, disadvantages, industry trends, maturity stages, and practical guidance for enterprises and professionals navigating modern big‑data pipelines.

DataOpsELTETL
0 likes · 31 min read
Data Integration Maturity Model: From ETL to EtLT
DataFunTalk
DataFunTalk
May 8, 2024 · Big Data

Risk Control and Data Application in the Bulk Commodity Industry: Challenges, Solutions, and Core Capabilities

The article presents Ant Group's exploration of applying its data‑driven risk control and credit assessment capabilities to the traditional bulk commodity sector, detailing industry background, data pain points, core technical solutions, and the construction of a secure, explainable data‑model platform for digital transformation.

AIBulk IndustryCredit Scoring
0 likes · 13 min read
Risk Control and Data Application in the Bulk Commodity Industry: Challenges, Solutions, and Core Capabilities
21CTO
21CTO
Apr 28, 2024 · Artificial Intelligence

5 Transformative Business Use Cases for Conversational AI

This article explores how conversational AI, powered by large language models, is reshaping enterprise operations across five key scenarios—from customer support assistants and AI‑driven data interfaces to HR bots, unstructured data processing, and multi‑agent digital assistants—highlighting benefits, implementation considerations, and privacy challenges.

Conversational AICustomer SupportHuman Resources
0 likes · 13 min read
5 Transformative Business Use Cases for Conversational AI
Data Thinking Notes
Data Thinking Notes
Apr 9, 2024 · Big Data

What Is a Data Middle Platform and Why It’s Essential for Modern Enterprises

Data middle platforms transform raw enterprise data into reusable assets by integrating collection, storage, processing, governance, and service layers, enabling faster deployment, consistent metrics, improved data quality, and business value across digital transformation, while addressing challenges like siloed data, low efficiency, and inconsistent standards.

big datadata governancedata integration
0 likes · 23 min read
What Is a Data Middle Platform and Why It’s Essential for Modern Enterprises
DataFunSummit
DataFunSummit
Apr 7, 2024 · Big Data

Li Auto’s Flink on Kubernetes Data Integration Practice

This article presents Li Auto’s end‑to‑end data integration journey, detailing the evolution of its data platform, the challenges of heterogeneous sources, and how a unified Flink‑on‑K8s solution with cloud‑native architecture, operator management, monitoring, and checkpointing addresses batch‑stream convergence and future scalability.

FlinkKubernetesbatch processing
0 likes · 12 min read
Li Auto’s Flink on Kubernetes Data Integration Practice
DataFunTalk
DataFunTalk
Mar 1, 2024 · Big Data

Understanding Data Fabric and Data Virtualization: Concepts, Practices, and Real‑World Case Study

This article explains the fundamentals of Data Fabric and data virtualization, highlights the limitations of traditional centralized data warehouses, describes the three‑layer virtualization architecture, and presents a detailed securities‑industry case study that demonstrates cost, efficiency, and compliance benefits.

Data FabricETLLogical Data Warehouse
0 likes · 17 min read
Understanding Data Fabric and Data Virtualization: Concepts, Practices, and Real‑World Case Study
DataFunTalk
DataFunTalk
Feb 23, 2024 · Artificial Intelligence

Challenges and Opportunities in Applying Large‑Model AI to Healthcare

The article analyzes how large‑model medical AI is rapidly adopted yet struggles with implementation due to doctor shortages, behavioral resistance, data silos, safety regulations, and the need for strategic alignment, while contrasting the more supportive innovation ecosystem in the United States.

AI adoptionHealthcare Innovationdata integration
0 likes · 6 min read
Challenges and Opportunities in Applying Large‑Model AI to Healthcare
DataFunSummit
DataFunSummit
Feb 20, 2024 · Big Data

BitSail Open‑Source Data Integration Engine: Architecture, New Features, CDC Solutions and Future Outlook

This article introduces ByteDance's open‑source data integration engine BitSail, covering its background, layered architecture, recent feature enhancements, automated testing framework, CDC‑based full‑library synchronization solutions, and future development plans for connectors and real‑time data consistency.

CDCDistributed SystemsFlink
0 likes · 12 min read
BitSail Open‑Source Data Integration Engine: Architecture, New Features, CDC Solutions and Future Outlook
DataFunTalk
DataFunTalk
Feb 17, 2024 · Big Data

JD Logistics One‑Stop Agile BI Solution: Architecture, Challenges, and Optimization

This article presents JD Logistics' one‑stop agile BI platform, detailing the complex data sources, rapid requirement changes, and Chinese‑style reporting challenges it addresses, while outlining the UData solution, product methodology, performance enhancements, and real‑world case studies that demonstrate significant efficiency gains.

Agile AnalyticsBIPerformance Optimization
0 likes · 26 min read
JD Logistics One‑Stop Agile BI Solution: Architecture, Challenges, and Optimization
DataFunSummit
DataFunSummit
Feb 5, 2024 · Artificial Intelligence

Ant Group's Knowledge Graph: Overview, Construction, Applications, and Integration with Large Models

Ant Group shares its comprehensive knowledge graph initiatives, detailing the fundamentals, construction pipeline, fusion techniques, cognitive representations, diverse business applications, and the emerging synergy between knowledge graphs and large language models, illustrating how graph-based AI enhances accuracy, interpretability, and downstream services.

Graph FusionLarge Language Modelsartificial intelligence
0 likes · 14 min read
Ant Group's Knowledge Graph: Overview, Construction, Applications, and Integration with Large Models
DataFunTalk
DataFunTalk
Jan 29, 2024 · Big Data

Case Study: Deploying RisingWave for Real-Time Stream Processing in a Large-Scale Quantitative Hedge Fund

An ultra‑large hedge fund with over $10 billion AUM replaced ksqlDB and Flink with RisingWave, leveraging its PostgreSQL‑compatible streaming SQL to achieve sub‑10 ms latency, lower learning and operational costs, rich connectors, advanced operators, and comprehensive observability for real‑time trade data processing.

RisingWaveSQLdata integration
0 likes · 10 min read
Case Study: Deploying RisingWave for Real-Time Stream Processing in a Large-Scale Quantitative Hedge Fund
NetEase LeiHuo UX Big Data Technology
NetEase LeiHuo UX Big Data Technology
Jan 9, 2024 · Artificial Intelligence

Accelerating Recommendation System Development with MindsDB

The article explains how the data team adopted the open‑source machine‑learning platform MindsDB to simplify data integration, enable SQL‑based model training and inference, manage model versions, and dramatically shorten recommendation system development cycles, achieving up to 30% efficiency gains.

MindsDBRecommendation SystemSQL
0 likes · 5 min read
Accelerating Recommendation System Development with MindsDB
Alibaba Cloud Native
Alibaba Cloud Native
Dec 28, 2023 · Cloud Computing

How to Set Up No‑Code Data Dump from Alibaba Cloud Kafka to OSS

This guide explains how to use Alibaba Cloud Message Queue Kafka's no‑code, fully managed, serverless dump feature to transfer data to OSS, covering its benefits, typical scenarios, required prerequisites, step‑by‑step configuration, testing, and verification of the resulting objects.

Alibaba CloudKafkaOSS
0 likes · 9 min read
How to Set Up No‑Code Data Dump from Alibaba Cloud Kafka to OSS
Sohu Tech Products
Sohu Tech Products
Dec 27, 2023 · Big Data

Practical Implementation of Data Integration with Flink on Kubernetes at Li Auto

Li Auto built a cloud‑native data‑integration platform by deploying Flink on Kubernetes, unifying batch and streaming workloads with a storage layer (JuiceFS + BOS) and Flink Operator, enabling simple source‑sink pipelines, elastic scaling, automated checkpointing, and centralized monitoring while addressing earlier fragmentation and resource inefficiencies.

Cloud NativeFlinkKubernetes
0 likes · 11 min read
Practical Implementation of Data Integration with Flink on Kubernetes at Li Auto
DataFunTalk
DataFunTalk
Dec 22, 2023 · Big Data

Practical Implementation of Flink on Kubernetes for Data Integration at Li Auto

This article details Li Auto's end‑to‑end data integration practice using Flink on Kubernetes, covering the evolution of their integration platform, architectural design, cloud‑native deployment, operational challenges, and future roadmap, while highlighting unified batch‑stream processing and resource elasticity.

Cloud NativeFlinkKubernetes
0 likes · 12 min read
Practical Implementation of Flink on Kubernetes for Data Integration at Li Auto
Baidu Intelligent Cloud Tech Hub
Baidu Intelligent Cloud Tech Hub
Dec 12, 2023 · Databases

Master Database Migration to Cloud: Challenges & Solutions with Baidu DTS

This article examines the rapid growth of China's database market, the technical hurdles of moving databases to public cloud—including engine selection, lengthy migration processes, efficiency, disaster recovery, and data consistency—and explains how Baidu Intelligent Cloud's DTS platform offers a smooth, reliable, high‑availability, and high‑performance one‑stop solution with real‑world use cases.

Baidu CloudDTSData Consistency
0 likes · 25 min read
Master Database Migration to Cloud: Challenges & Solutions with Baidu DTS
Data Thinking Notes
Data Thinking Notes
Dec 5, 2023 · Big Data

How to Overcome Data Governance Challenges and Unlock Business Value

Enterprises face significant hurdles in data governance and integration, from siloed systems and unclear responsibilities to poor data quality, but by establishing clear rules, fostering user department engagement, and aligning governance with business-driven data applications, they can create a cohesive data asset management framework that drives value.

Financial Servicesbig datadata assets
0 likes · 10 min read
How to Overcome Data Governance Challenges and Unlock Business Value
DataFunSummit
DataFunSummit
Oct 24, 2023 · Big Data

Practices of Data Fabric in Data Integration Scenarios

The presentation by Aloudata Vice President Yu Jun introduces his extensive background in large‑scale internet and big‑data platforms and outlines how Data Fabric and data virtualization can be applied to data integration, highlighting the differences from traditional solutions and the business value of logical data warehouses.

Data FabricLogical Data Warehousebig data
0 likes · 2 min read
Practices of Data Fabric in Data Integration Scenarios
DataFunTalk
DataFunTalk
Sep 30, 2023 · Big Data

Building a Marketing‑Oriented Data Middle Platform: Concepts and Practices

This article outlines how a marketing‑focused data middle platform can be constructed by integrating online and offline behavior data, business data, and third‑party sources, then applying data integration, modeling, processing, and application capabilities to enable data‑driven user journeys and personalized marketing strategies.

Data Modelingbig datadata integration
0 likes · 13 min read
Building a Marketing‑Oriented Data Middle Platform: Concepts and Practices
Java High-Performance Architecture
Java High-Performance Architecture
Sep 28, 2023 · Databases

How to Use Debezium for MySQL CDC in Spring Boot Without Adding Extra Middleware

Learn how to capture MySQL data changes using Debezium's CDC capabilities within a Spring Boot application, avoiding heavyweight message brokers by leveraging binlog monitoring, configuring connectors, handling snapshots, and processing change events for use cases like cache invalidation, data integration, and simplifying monolithic architectures.

CDCDebeziumKafka
0 likes · 24 min read
How to Use Debezium for MySQL CDC in Spring Boot Without Adding Extra Middleware
Architects Research Society
Architects Research Society
Sep 27, 2023 · Fundamentals

What Is the Common Data Model and Why Use It?

The Common Data Model provides a shared, standardized data language and metadata system that simplifies cross‑application data integration, reduces custom development effort, and enables consistent, extensible data structures for business and analytics scenarios across Microsoft Power Platform and Azure services.

Common Data ModelEnterprise DataMicrosoft Power Platform
0 likes · 8 min read
What Is the Common Data Model and Why Use It?
DataFunSummit
DataFunSummit
Sep 8, 2023 · Big Data

Tianqiong OLAP Real‑time Lakehouse Fusion Platform Architecture Practice

This article explains why lake‑warehouse fusion is needed, describes the challenges of integrating real‑time data warehouses with data lakes, introduces a new StarRocks‑based architecture that supports real‑time ingestion, cooling, offline loading, and adaptive hot‑cold query rewriting, and outlines future plans and Q&A.

Real-time OLAPStarRocksbig data
0 likes · 21 min read
Tianqiong OLAP Real‑time Lakehouse Fusion Platform Architecture Practice
DataFunSummit
DataFunSummit
Aug 10, 2023 · Databases

ClickHouse Deployment in Lenovo Manufacturing: Architecture, Data Integration, and Performance Optimization

This article details Lenovo's implementation of ClickHouse in a manufacturing environment, covering the current data landscape, cluster architecture, integration challenges, performance enhancements, and solutions such as Seatunnel and query pre‑aggregation, illustrating how OLAP engines can address real‑time analytics and concurrency issues in production data pipelines.

ClickHouseOLAPSeaTunnel
0 likes · 11 min read
ClickHouse Deployment in Lenovo Manufacturing: Architecture, Data Integration, and Performance Optimization
Data Thinking Notes
Data Thinking Notes
Aug 2, 2023 · Fundamentals

Mastering Enterprise Data: A Practical Guide to Master Data Management

This article explains why fragmented data hampers business insight in large enterprises and provides a comprehensive overview of master data concepts, governance structures, standards, processes, and step‑by‑step implementation practices to achieve consistent, high‑quality enterprise data.

Enterprise DataMDMMaster Data
0 likes · 18 min read
Mastering Enterprise Data: A Practical Guide to Master Data Management
Architects Research Society
Architects Research Society
Aug 2, 2023 · Fundamentals

Data Fabric Architecture: Three Patterns, Core Technical Components, and Inherent Limitations

The article explains data fabric architecture as a promising approach for enabling data exchange across distributed systems, outlines its three design patterns, describes key technical components such as data virtualization, data catalog, and knowledge graphs, and discusses the trade‑offs, costs, and limitations that organizations must consider.

Data Fabricdata catalogdata integration
0 likes · 17 min read
Data Fabric Architecture: Three Patterns, Core Technical Components, and Inherent Limitations
Didi Tech
Didi Tech
Jul 31, 2023 · Big Data

Data Serviceization at Didi: Architecture, Phases, and Standard Metric Service

Didi’s data serviceization converts raw business data into consumable services through a four‑stage pipeline—integration, development, production, and back‑flow—while the Data Dream Factory and Shu‑Chain platform automate synchronization, provide a unified access gateway for thousands of APIs, and introduce a standard metric service that abstracts storage complexities and ensures high‑performance, secure data delivery.

data integrationdata platformdata serviceization
0 likes · 16 min read
Data Serviceization at Didi: Architecture, Phases, and Standard Metric Service
Inke Technology
Inke Technology
Jun 28, 2023 · Big Data

Extending Apache Seatunnel for Trino and Kyuubi Integration: A Practical Guide

This article outlines the challenges of scaling data integration platforms, proposes a comprehensive solution using Apache Seatunnel and Dinky, details the implementation of Trino and Kyuubi JDBC support, and describes the platform's architecture, task publishing workflow, logging, monitoring, resource management, and future enhancements.

Apache SeaTunnelData PipelineKyuubi
0 likes · 16 min read
Extending Apache Seatunnel for Trino and Kyuubi Integration: A Practical Guide
Architects Research Society
Architects Research Society
Jun 21, 2023 · Fundamentals

The Strategic Role of Enterprise Architects: Five Strategic and One Tactical Focus Areas

Enterprise architects align IT strategy with business goals by overseeing application portfolio management, technology and risk, IT operations, security and privacy, integration and data, and finance, defining roadmaps for 1‑3‑5 year plans while balancing strategic and tactical responsibilities in a rapidly changing environment.

Application PortfolioIT OperationsIT strategy
0 likes · 6 min read
The Strategic Role of Enterprise Architects: Five Strategic and One Tactical Focus Areas
21CTO
21CTO
Jun 20, 2023 · Fundamentals

ETL vs ELT: Which Data Integration Method Wins for Your Business?

ETL extracts, transforms, then loads data, while ELT extracts, loads, and transforms later, each offering distinct advantages; the article compares their processes, key differences, and factors such as data volume, complexity, latency, and cost to help businesses choose the optimal integration approach.

ELTdata integrationdata warehousing
0 likes · 12 min read
ETL vs ELT: Which Data Integration Method Wins for Your Business?
StarRocks
StarRocks
May 26, 2023 · Big Data

How SeaTunnel’s StarRocks Connector Enables High‑Performance Data Sync

This article explains SeaTunnel’s architecture and its StarRocks connector, detailing source and sink features such as field projection, predicate push‑down, parallel reading, state recovery, data type mapping, Stream Load writes, CDC support, configuration examples, and future roadmap for exactly‑once semantics.

ConnectorSeaTunnelStarRocks
0 likes · 16 min read
How SeaTunnel’s StarRocks Connector Enables High‑Performance Data Sync
Top Architect
Top Architect
May 4, 2023 · Big Data

Data Middle Platform: General Architecture and Core Components

The article explains the concept, benefits, and detailed modular architecture of a data middle platform, covering data storage, acquisition, processing, governance, security, and operation frameworks, and illustrates how enterprises can build and evolve such platforms to turn data into valuable services.

big datadata architecturedata governance
0 likes · 19 min read
Data Middle Platform: General Architecture and Core Components
ITPUB
ITPUB
Apr 26, 2023 · Databases

Mastering Change Data Capture: Open‑Source Tools and How to Choose the Right One

This article explains the concept of Change Data Capture (CDC), outlines its common use cases, compares the main technical approaches—including timestamps, data diff, triggers, and log‑based methods—and reviews popular open‑source CDC solutions and their database‑specific configuration requirements.

CDCReal-timechange data capture
0 likes · 15 min read
Mastering Change Data Capture: Open‑Source Tools and How to Choose the Right One
ITPUB
ITPUB
Apr 25, 2023 · Big Data

Top 8 Open‑Source ETL Tools for Data Migration and Integration

This article reviews eight widely used ETL and data‑migration tools—including Kettle, DataX, DataPipeline, Talend, DataStage, Sqoop, FineDataLink, and Canal—detailing their core features, architectures, supported data sources, and typical usage scenarios to help practitioners choose the right solution.

Data PipelineETLbig data
0 likes · 13 min read
Top 8 Open‑Source ETL Tools for Data Migration and Integration
DataFunSummit
DataFunSummit
Apr 9, 2023 · Big Data

Expert Interview: Architecture and Trends of Big Data Platforms

This article presents a comprehensive interview with several big‑data platform experts, outlining the core components such as data integration, storage and computation, distributed scheduling, and query analysis, while also highlighting current challenges, best‑practice tools, and future trends in big‑data architecture.

OLAPbig datadata integration
0 likes · 10 min read
Expert Interview: Architecture and Trends of Big Data Platforms
DataFunTalk
DataFunTalk
Apr 4, 2023 · Big Data

Upgrading Hangzhou Bank Consumer Finance Big Data Platform with Apache Doris 1.2: Architecture, Performance Gains, and Integration

This article details how Hangzhou Bank Consumer Finance modernized its big‑data platform by introducing Apache Doris 1.2, replacing the original Greenplum + CDH architecture, unifying data sources via Multi‑Catalog, achieving second‑level query latency, reducing storage and compute costs, and outlining the integration workflow with DolphinScheduler, SeaTunnel, and Spark.

Apache DorisPerformance Optimizationbig data
0 likes · 20 min read
Upgrading Hangzhou Bank Consumer Finance Big Data Platform with Apache Doris 1.2: Architecture, Performance Gains, and Integration
HomeTech
HomeTech
Mar 31, 2023 · Artificial Intelligence

Digital Transformation of Used‑Car Buying: Integrated Data, AI Valuation, and VR Visualization

The article describes how a comprehensive digital platform combines structured, semi‑structured, and panoramic data with machine‑learning valuation models, natural‑language processing, and VR technology to make used‑car condition information transparent, improve estimation accuracy, and enhance user decision‑making in the Chinese second‑hand car market.

AI valuationVR Visualizationbig data
0 likes · 15 min read
Digital Transformation of Used‑Car Buying: Integrated Data, AI Valuation, and VR Visualization
macrozheng
macrozheng
Mar 27, 2023 · Big Data

Top 8 Open-Source ETL Tools for Efficient Data Migration

This guide reviews eight popular ETL and data migration tools—including Kettle, DataX, DataPipeline, Talend, DataStage, Sqoop, FineDataLink, and Canal—detailing their core features, architectures, and use cases to help engineers choose the right solution for reliable data integration.

ETLToolsbig data
0 likes · 14 min read
Top 8 Open-Source ETL Tools for Efficient Data Migration
Su San Talks Tech
Su San Talks Tech
Mar 24, 2023 · Big Data

Top 8 Open-Source ETL Tools You Should Know for Efficient Data Migration

Explore a comprehensive overview of eight popular ETL and data migration tools—including Kettle, DataX, DataPipeline, Talend, DataStage, Sqoop, FineDataLink, and Canal—detailing their features, architectures, and use cases to help you choose the right solution for efficient data integration.

ETLToolsbig data
0 likes · 13 min read
Top 8 Open-Source ETL Tools You Should Know for Efficient Data Migration
Architects Research Society
Architects Research Society
Mar 5, 2023 · Big Data

Best Open‑Source and Commercial ETL Tools: Detailed Comparison

This article introduces the concept of ETL, explains its importance for modern data‑driven applications, and provides a comprehensive comparison of the most popular open‑source and commercial ETL platforms—including their key features, supported data sources, and deployment options—helping readers choose the right tool for their data integration needs.

CommercialETLbig data
0 likes · 19 min read
Best Open‑Source and Commercial ETL Tools: Detailed Comparison
HomeTech
HomeTech
Mar 1, 2023 · Backend Development

Overview of the Wenjie Low-Code Platform: Architecture, Technologies, and Use Cases

The article presents a comprehensive overview of the Wenjie low-code platform, detailing its motivation, front‑end React framework, back‑end Spring Cloud micro‑services architecture, PowerJob scheduler, custom ORM, various data‑modeling and data‑processing scenarios, dashboard visualizations, monitoring and alerting features, as well as future plans and a concluding summary.

PlatformPowerJobReact
0 likes · 11 min read
Overview of the Wenjie Low-Code Platform: Architecture, Technologies, and Use Cases
DataFunTalk
DataFunTalk
Feb 27, 2023 · Big Data

Comprehensive Overview of Data Middle Platform Architecture and Its Core Frameworks

This article provides a detailed overview of data middle platform concepts, describing a decoupled six‑subsystem architecture—including storage, collection, processing, governance, security, and operation frameworks—while illustrating typical enterprise implementations, industry‑specific solutions, and best‑practice considerations for building scalable, secure, and value‑driven data platforms.

big datadata governancedata integration
0 likes · 25 min read
Comprehensive Overview of Data Middle Platform Architecture and Its Core Frameworks
Aikesheng Open Source Community
Aikesheng Open Source Community
Feb 24, 2023 · Databases

SQLE 2.2302.0 Release Notes – New Features, Enhancements, and Bug Fixes

The SQLE 2.2302.0 release introduces data source import from external platforms, operation‑record viewing, improved CloudBeaver integration, manual deployment support, numerous UI optimizations, and a long list of bug fixes, providing a more seamless and secure SQL auditing experience for both enterprise and community users.

CloudBeaverDatabase AuditingSQL
0 likes · 8 min read
SQLE 2.2302.0 Release Notes – New Features, Enhancements, and Bug Fixes
DataFunTalk
DataFunTalk
Feb 2, 2023 · Big Data

SeaTunnel: Design Goals, Current Status, Architecture, and Future Roadmap

This article provides a comprehensive overview of Apache SeaTunnel, covering its design objectives, current capabilities such as multi‑engine support and extensive connector ecosystem, detailed architecture including engine‑independent APIs and execution flows, and outlines the upcoming roadmap to expand connectors, launch a visual web UI, and introduce a dedicated SeaTunnel Engine.

ApacheConnectorSeaTunnel
0 likes · 12 min read
SeaTunnel: Design Goals, Current Status, Architecture, and Future Roadmap
Data Thinking Notes
Data Thinking Notes
Jan 31, 2023 · Fundamentals

Mastering Data Governance: From Metadata to ETL in One Guide

This comprehensive guide walks you through the entire data governance ecosystem, covering metadata fundamentals, classification, maturity models, data standards, modeling, integration, lifecycle management, quality assurance, security, and ETL processes, all illustrated with clear diagrams and practical steps.

Data ModelingETLdata governance
0 likes · 13 min read
Mastering Data Governance: From Metadata to ETL in One Guide
DataFunTalk
DataFunTalk
Jan 31, 2023 · Big Data

SPI Refactoring Practice in Apache InLong Manager to Reduce Maintenance Cost and Enhance Extensibility

This article presents the SPI-based refactoring of Apache InLong Manager, describing the project's background, existing maintenance challenges, the concept of Java Service Provider Interface, the concrete implementation steps, code restructuring, and the resulting benefits such as higher code reuse, easier extension, and reduced DDL changes.

Apache InLongJavaSPI
0 likes · 10 min read
SPI Refactoring Practice in Apache InLong Manager to Reduce Maintenance Cost and Enhance Extensibility