Tag

big-data

0 views collected around this technical thread.

macrozheng
macrozheng
May 12, 2025 · Backend Development

Designing a Billion‑User Real‑Time Leaderboard: Redis vs MySQL

This article explores how to build a scalable, high‑performance leaderboard for hundreds of millions of users by comparing traditional database ORDER BY approaches with Redis sorted sets, addressing challenges such as hot keys, memory pressure, persistence risks, and presenting a divide‑and‑conquer implementation strategy.

High ConcurrencyRankingbig-data
0 likes · 11 min read
Designing a Billion‑User Real‑Time Leaderboard: Redis vs MySQL
Youzan Coder
Youzan Coder
Dec 22, 2021 · Big Data

3rd Youzan Big Data Technology Salon: Apache Kylin4, Data Governance, and AI Applications

The 3rd Youzan Big Data Technology Salon, held online for over 200 participants, showcased Apache Kylin 4’s performance boost, GeTui’s five‑step AI method, Kwai’s sustainable data‑governance system, and Youzan’s intelligent copy algorithms, highlighting data governance’s evolution into a core business priority and the shift toward intelligent discovery.

Apache KylinData GovernanceNLP
0 likes · 6 min read
3rd Youzan Big Data Technology Salon: Apache Kylin4, Data Governance, and AI Applications
Big Data Technology Architecture
Big Data Technology Architecture
May 23, 2019 · Big Data

Kafka Performance Design: Sequential I/O, Page Cache, Zero‑Copy, and Partition Segmentation

The article explains how Kafka achieves high throughput and low latency by leveraging sequential disk I/O, operating‑system page cache, zero‑copy transmission, and a partition‑segment storage model, all of which are key design choices for big‑data messaging systems.

KafkaMessagingPage Cache
0 likes · 6 min read
Kafka Performance Design: Sequential I/O, Page Cache, Zero‑Copy, and Partition Segmentation
Tencent Cloud Developer
Tencent Cloud Developer
Jan 10, 2019 · Big Data

2018 Chinese Variety Show Data Analysis: Web Scraping, Rankings, and Reviews

This article demonstrates how to scrape the full 2018 Chinese variety‑show list from Douban using Python Selenium and BeautifulSoup, compile detailed metadata and actor information into Excel, and then analyze popularity rankings, rating distributions, frequent celebrity appearances, and common negative feedback.

DoubanPythonSelenium
0 likes · 24 min read
2018 Chinese Variety Show Data Analysis: Web Scraping, Rankings, and Reviews
Architecture Digest
Architecture Digest
Apr 6, 2016 · Backend Development

Evolution of Kuaidi Dache Architecture: Solving LBS Bottlenecks, Long‑Connection Stability, Distributed Refactoring, Open Platform, Real‑Time Monitoring, and Data‑Layer Transformation

This article details how Kuaidi Dache scaled from 2013 to 2015 by addressing LBS performance limits, redesigning long‑connection services, refactoring monolithic code into layered services with Dubbo and RocketMQ, building a secure open platform, implementing Storm‑based real‑time monitoring, and migrating data storage to sharded MySQL, Canal‑driven sync, and HBase for massive scalability.

Microservicesarchitecturebackend
0 likes · 12 min read
Evolution of Kuaidi Dache Architecture: Solving LBS Bottlenecks, Long‑Connection Stability, Distributed Refactoring, Open Platform, Real‑Time Monitoring, and Data‑Layer Transformation