Official account of the DataFun community, dedicated to sharing big data and AI industry summit news and speaker talks, with regular downloadable resource packs.
This article details Tencent Search's exploration of large‑model‑driven ranking, covering the evolution from traditional keyword retrieval to RAG‑based AI search, the multi‑stage AI ranking architecture (L0‑L5), model training pipelines, distillation, synthetic data generation, and future research directions.
The article explains how Kuaishou modernized its data lake by partnering with Apache Hudi to address latency, storage cost, and consistency issues in both AI and BI pipelines, detailing architectural changes, new ingestion tools, partitioning strategies, compaction mechanisms, performance gains and future plans.
This article explains Apache Flink’s stream processing fundamentals, introduces the open‑source Flink Agents framework for building event‑driven AI agents, details Alibaba Cloud’s Flink AI Function for real‑time LLM inference, and showcases demos, architecture, integration patterns, and practical use cases such as VOC analysis, live‑stream analytics, and intelligent operations.
In a deep‑dive roundtable, three data‑engineering veterans discuss the rise of AI agents, the importance of data context, memory mechanisms, workflow versus agent trade‑offs, and the future of database intelligence, offering practical strategies and architectural philosophies for building smarter data pipelines.
In the era of massive AI model training, this article presents a step‑by‑step technical guide—covering the full data‑collection pipeline, three acquisition modes, IP‑type choices, bandwidth savings, path and mixed‑request optimizations, and business‑level cost controls—to reduce web‑scraping expenses by more than 60% while maintaining data quality.
This article examines the shortcomings of traditional big‑data engines for AI workloads, presents a Ray‑based heterogeneous fusion architecture that unifies CPU/GPU scheduling, Python ecosystems, and streaming‑batch processing, and details fault‑tolerance, checkpointing, compute‑storage separation, resource‑utilization, scalability, and observability improvements that enable thousands of nodes and dramatically higher GPU efficiency.
This article outlines the evolution from traditional AI to embodied intelligence, explains the Vision‑Language‑Action (VLA) paradigm, highlights data‑collection bottlenecks, introduces the UnrealZoo simulation platform built on Unreal Engine, and showcases real‑world case studies and future challenges for embodied AI research.
Healthpeak’s commercial‑real‑estate workflow, plagued by data silos and manual meter‑reading, was transformed by deploying Palantir’s AI Platform, which introduced an ontology‑based four‑layer architecture that automates billing, detects anomalies, and enables mobile‑first, AI‑driven decision making.
This article explores the challenges of international logistics and presents the WOL‑APL‑EVAL three‑layer architecture—workflow governance, adaptive planning, and continuous evaluation—demonstrating how AI agents, rule engines, and dynamic planning can automate customs clearance, reduce manual effort, and improve compliance and efficiency.
This article examines how Healthpeak, a large healthcare REIT, replaced manual spreadsheet‑based processes with Palantir’s AI Platform (AIP), using an ontology‑driven architecture to automate billing, detect anomalies, and orchestrate workflows, delivering faster operations, higher accuracy, and scalable growth.
