Author

Big Data Technology & Architecture

Wang Zhiwu, a big data expert, dedicated to sharing big data technology.

1.0k

Articles

Likes

Views

Comments

Latest from Big Data Technology & Architecture

100 recent articles max

Big Data Technology & Architecture

Jul 1, 2025 · Big Data

What’s New in Apache Hive 4.0? Key Features and Industry Outlook

After a weekend dive into Apache Hive’s official Wiki and GitHub, this article highlights Hive’s declining visibility compared to Spark and Flink, examines its 4.0 release’s major features—including Iceberg integration, enhanced ACID, cost‑based optimizer upgrades, and Ozone support—while reflecting on its role in modern data ecosystems.

Apache HiveBig DataData Warehouse

0 likes · 4 min read

What’s New in Apache Hive 4.0? Key Features and Industry Outlook

Big Data Technology & Architecture

Jun 24, 2025 · Big Data

How Leading Companies Leverage Apache Paimon for Real‑Time Lakehouse Success

This article summarizes how major tech firms such as Vivo, Shopee, Alibaba, and TikTok adopt Apache Paimon to unify batch and streaming data pipelines, improve latency, reduce costs, and optimize storage, highlighting key challenges, architectural solutions, and real‑world performance gains.

Batch ProcessingPaimonlakehouse

0 likes · 8 min read

How Leading Companies Leverage Apache Paimon for Real‑Time Lakehouse Success

Big Data Technology & Architecture

Jun 20, 2025 · Big Data

2025 Mid-Year Data Job Market Trends: Skills, Hiring Preferences, and Career Advice

This semi‑annual report analyzes over 120 interview experiences to reveal current hiring trends, required skills, and strategic advice for data professionals targeting mid‑to‑senior positions in large‑scale companies during the first half of 2025.

2025 trendscareer advicedata engineering

0 likes · 9 min read

2025 Mid-Year Data Job Market Trends: Skills, Hiring Preferences, and Career Advice

Big Data Technology & Architecture

Jun 11, 2025 · Big Data

How to Solve Common Paimon Performance Issues in Flink: Small Files, OOM, and More

This article compiles frequent problems encountered when using Paimon with Flink—such as small‑file generation, write‑performance bottlenecks, OOM/GC issues, file‑deletion conflicts, dimension‑table join slowness, and snapshot expiration—and provides practical configuration and optimization solutions.

Big DataFlinkOptimization

0 likes · 9 min read

How to Solve Common Paimon Performance Issues in Flink: Small Files, OOM, and More

Big Data Technology & Architecture

Jun 10, 2025 · Big Data

Transforming Real‑Time Analytics: Incremental Computing with Lakehouse Architecture

This article examines how Xiaohongshu replaced its costly Lambda architecture with a real‑time lakehouse built on Iceberg, Paimon, Spark, and StarRocks, achieving minute‑level latency, higher data quality, lower resource consumption, and dramatically faster query performance.

Big Data ArchitectureIcebergPaimon

0 likes · 7 min read

Transforming Real‑Time Analytics: Incremental Computing with Lakehouse Architecture

Big Data Technology & Architecture

Jun 9, 2025 · Databases

Why Data Warebase Could Be the Next Game‑Changer for AI Workloads

The article examines how emerging data‑infrastructure trends, multi‑modal databases like Neon, Supabase, and ClickHouse, and the convergence of OLTP, OLAP, and vector search are reshaping AI workloads, introducing the Data Warebase concept that unifies warehouse and database capabilities to meet modern AI workflow demands.

AIHTAPReal-time analytics

0 likes · 32 min read

Why Data Warebase Could Be the Next Game‑Changer for AI Workloads

Big Data Technology & Architecture

Jun 6, 2025 · Big Data

How to Ace Big Data Interviews: A Complete 3‑Month Prep Guide

This guide outlines a step‑by‑step three‑month preparation plan—including resume building, project showcase, interview mindset, mock sessions, and offer negotiation—to help candidates secure high‑paying big‑data positions at top companies.

Big DataInterview preparationoffer negotiation

0 likes · 7 min read

How to Ace Big Data Interviews: A Complete 3‑Month Prep Guide

Big Data Technology & Architecture

Jun 5, 2025 · Big Data

Flink Web UI Monitoring and End‑to‑End Latency Implementation Guide

This article explains the key monitoring items of the Flink Web UI, details task topology, operator and system metrics, checkpoint and log inspection, and provides two practical solutions—custom metrics and distributed tracing—to measure and visualize full‑chain latency in Flink jobs.

Big DataDistributed TracingFlink

0 likes · 10 min read

Flink Web UI Monitoring and End‑to‑End Latency Implementation Guide

Big Data Technology & Architecture

Jun 3, 2025 · Big Data

Query Optimization Techniques for Paimon Real-Time Data Lake

This article explains how to improve Paimon's query performance by optimizing table schemas, storage settings, query parameters, and index designs, covering table mode choices, partitioning, file formats, parallelism, batch reads, and various index types such as Bloom filters and clustering indexes.

Big DataLSM‑TreePaimon

0 likes · 8 min read

Query Optimization Techniques for Paimon Real-Time Data Lake

Big Data Technology & Architecture

May 26, 2025 · Artificial Intelligence

Spring AI 1.0 Released: Maven Dependency, Comparison with LangChain4j, and Future Trends

The article announces the Spring AI 1.0 release, provides the Maven BOM dependency, compares Spring AI with LangChain4j for Java AI development, and outlines upcoming integration, performance, and observability improvements for both frameworks amid rapid large‑model advancements.

Artificial IntelligenceJavaLangchain4j

0 likes · 4 min read

Spring AI 1.0 Released: Maven Dependency, Comparison with LangChain4j, and Future Trends