This tutorial walks through using Python to crawl Weibo, covering environment setup, three login methods, data extraction functions for user info, posts and comments, anti‑crawling strategies, storage to CSV or MySQL, a full example script, and legal considerations.
This article presents a Python‑based project that continuously crawls Weibo hot‑search data, stores it with timestamps, and visualizes trends through dynamic bar, line, and word‑cloud charts using libraries such as BeautifulSoup, pandas, schedule, pyecharts, and jieba.
This article demonstrates how to use Python to scrape Weibo comments about the TV series 'The Legend of the Year', generate word clouds, compute character frequencies and co‑occurrences, and visualize the resulting character relationship network with Gephi.
This guide explains how to extract the UID of a Weibo user who posted a specific image by dissecting the image URL, converting the embedded hexadecimal string to decimal, and verifying the result against the user’s profile.
This article demonstrates how to use LangChain, LLMs, and SerpAPI to create a Weibo influencer‑search tool that extracts UID numbers, scrapes profile data, filters Chinese content, and prepares the information for automated marketing outreach.
Weibo has begun testing a front‑page real‑name system for high‑follower accounts and a visitor‑record function limited to premium members, sparking user backlash over privacy concerns, debates about platform monetisation, and discussions of the broader impact on social media behaviour.
This article explains the architecture of Weibo's popular recommendation system, the role of the weidl online learning platform, and how DeepRec’s performance optimizations—such as oneDNN operator acceleration, cost‑aware scheduling, and adaptive memory allocation—significantly improve training speed, inference latency, and overall service throughput.
This article presents a comprehensive overview of feature embedding modeling in recommendation systems, discussing the necessity of feature modeling, three technical directions (gate threshold, variable‑length embeddings, and enrichment), detailed descriptions of models such as FiBiNet, FiBiNet++, ContextNet, and MaskNet, experimental findings, and a Q&A session that addresses practical challenges and future work.
This tutorial walks you through creating a Python web scraper that logs into Weibo, extracts the real‑time hot search list and top comments using requests, lxml and pandas, saves the data to Excel, and schedules the script to run hourly with Windows Task Scheduler.
This tutorial shows how to use Python's requests, lxml, and pandas libraries to crawl Weibo's real‑time hot search list, extract ranking, popularity, titles, and detailed comments, and schedule the script to run automatically each hour.
This article walks through using Python to scrape Wang Leehom’s divorce‑related Weibo comments, clean the noisy dataset, visualize hourly comment trends, compare with his ex‑wife’s posts, generate word‑clouds and emoji frequency charts, and provides full code and data for reproducible analysis.
This tutorial demonstrates how to use Python to scrape Weibo for the most common National Day leave reasons, extract comment data via Ajax requests, store it in CSV files, and create visualizations such as top‑liked users and word‑clouds to reveal popular motifs.
This tutorial demonstrates how to use Python's requests library to programmatically log into Weibo, publish a status update, and automatically like posts by handling cookies, sessions, and the required st token, complete with full code examples.
This article explains the fundamentals of contrastive learning, reviews typical models such as SimCLR, MoCo, SwAV, BYOL, SimSiam and Barlow Twins, and demonstrates how these methods are applied to Weibo text and multimodal (text‑image) representation tasks like hashtag generation and image‑text matching.
This tutorial shows how to use Python to fetch the Weibo hot search list, clean and organize the data with pandas, send the results via QQ email, and schedule the whole process to run automatically each evening.
This article explains the concept of contrastive learning, its relationship to self‑supervised and metric learning, describes key system components and loss functions, reviews major image, NLP and multimodal models such as SimCLR, MoCo, SwAV, BYOL, and demonstrates how contrastive learning is applied to Weibo hashtag generation, similar‑post retrieval, and text‑image matching using CD‑TOM and W‑CLIP models.
This article details Weibo's large‑scale multimodal content understanding platform, covering its background, data and model heterogeneity challenges, the end‑to‑end workflow, GPU‑heterogeneous cluster design, resource scheduling, performance optimization for distributed training and online inference, and comprehensive monitoring to ensure stable, low‑latency AI services.
This article details Weibo's multimodal content understanding platform, covering its massive data challenges, heterogeneous model support, standardized pipelines, platformization, workflow architecture, GPU heterogeneous cluster management, resource scheduling, performance optimization, and full‑stack monitoring to achieve stable, low‑latency AI services at scale.
The article explores the fundamental concepts of system architecture, illustrating how large‑scale services like Weibo handle massive traffic through layered design, abstraction, caching, service decomposition, monitoring, and operational practices to achieve scalability, reliability, and performance.
This article explores the fundamental principles of system architecture by analyzing Weibo’s evolution to a multi‑layer, high‑traffic platform, covering scalability, service decomposition, caching strategies, distributed tracing, and operational best practices for building robust backend systems.
This article explains how to locate Weibo comment APIs, work around rate limits by using the mobile site, extract required parameters, and implement a Python script that handles cookies, pagination, emoji cleaning, deduplication, and scheduled execution to collect comments efficiently.
This report recaps a technical closed‑door session where experts from Weibo and Ziroom shared real‑world API gateway implementations, discussed centralized versus decentralized designs, explored the evolution of Ziroom's gateway, introduced the Gamma‑RPC framework, and debated future trends like service mesh.
This article shares Weibo’s experience in building and evolving its recommendation algorithms, covering the recommendation scenario, machine learning workflow, feature engineering, model upgrades, large‑scale challenges, deployment via the Weiflow platform, and the capabilities of its machine‑learning infrastructure.
This article presents an in‑depth overview of Weibo's large‑scale machine learning platform, detailing its multi‑layer architecture, development workflow, CTR model evolution, and how Apache Flink is employed for real‑time data processing, sample services, multi‑stream joins, multimedia feature generation, and future roadmap plans.
This article explains how Weibo's advertising platform leverages content understanding—covering system architecture, problems caused by insufficient comprehension, the construction of NLP and vision capabilities, content‑based ad strategies, and a celebrity‑brand knowledge graph—to improve ad relevance and ROI.
The article details how a data analyst scraped comments from Zhihu, Weibo, and Bilibili about the 2020 Youth Day video “后浪”, applied Baidu AI sentiment analysis and Python word‑cloud generation, and compared the differing emotional tones across platforms.
This article explains how to automate the extraction and recognition of Weibo's four‑grid hand‑drawn captcha by enumerating all 24 possible patterns, capturing the captcha image with Selenium, and performing template matching using pixel‑wise comparison in Python.
The article explains the recent Weibo data breach, describing how attackers used credential‑stuffing (撞库), data leakage (漏水), database dumping (拖库) and data laundering (洗库) to obtain millions of user records, the technical steps involved, and the security implications for both users and platforms.
This guide walks through the challenges of accessing Weibo comment data, demonstrates how to locate and bypass API restrictions using the mobile site, and provides step‑by‑step Python code for fetching, cleaning, de‑duplicating, and scheduling comment extraction into text files.
This article reviews the specific NLP difficulties encountered in Sina Weibo's feed—such as short text, informal language, and ambiguous user behavior—and details the multi‑stage tagging system, material library, multimodal modeling, multi‑task learning, and large‑scale pre‑training techniques used to address them.
This guide details a Python‑based Scrapy spider that crawls Sina Weibo user profiles, posts, followers and followings from the WAP site, explains the required environment, configuration steps, and the MongoDB schema used to store the collected data.
This article details how Weibo leverages Redis across billions of requests, describing its massive scale, the challenges of trillion‑level reads/writes, the technical choices and customizations made—including LongSet, HA solutions, multi‑level caching, RocksDB integration—and outlines ongoing capacity and future development strategies.
The presentation explains how Weibo leverages big‑data technologies, user profiling, and social‑first advertising models to drive commercial growth, detailing data‑driven product development, real‑time and offline data warehouses, scientific experiments, and case studies that illustrate the impact on revenue and user engagement.
This article presents a comprehensive overview of the evolution of Weibo's advertising strategy engineering architecture, detailing the system's growth from early banner ads to a sophisticated, multi‑layered online advertising platform that integrates algorithmic models, A/B experimentation, real‑time data pipelines, and precision targeting to support scalable, high‑performance ad delivery.
This article traces the evolution of Weibo's recommendation system architecture across three major stages—independent 1.0, layered 2.0, and platform‑style 3.0—detailing the environmental drivers, technical goals, component composition, advantages, shortcomings, and the concrete outcomes of each phase.
The article presents a detailed analysis of Weibo's short‑video platform architecture, covering team background, business scenarios, micro‑service design, feed‑pull model, multi‑level distributed caching, multi‑datacenter HA deployment, circuit‑breaker mechanisms, and elastic scaling to achieve high availability under unpredictable traffic spikes.
Learn how to automate the download of your Sina Weibo favorites—including posts, comments, images, and video links—by simulating login with Python, saving cookies, crawling paginated collection pages, and extracting content for later processing into documents.
In this interview, Weibo’s microservices expert Hu Zhongxiang discusses the advantages of microservice architecture for handling sudden traffic spikes, challenges in implementation, service splitting strategies, the link between microservices and cloud‑native, and offers learning recommendations and insights for the upcoming GIAC conference.
This tutorial walks through building a Weibo monitoring tool in Python by simulating mobile login, extracting API endpoints, fetching user posts via JSON, handling containers, and implementing a session‑based class that polls for new tweets and sends email alerts.
This article explains how Weibo handles massive daily traffic of over a hundred billion requests by employing a five‑layer feed system and a six‑layer cache architecture that evolved from simple KV storage to sophisticated counter and existence‑check services, highlighting design choices, performance optimizations, and future directions.
This article traces the evolution of Weibo's recommendation system architecture across three major phases—independent 1.0, layered 2.0, and platform‑centric 3.0—detailing the environmental drivers, technical components, advantages, drawbacks, and key outcomes of each stage.
This article explains how to create a Python‑based Scrapy spider that logs into Sina Weibo using cookies, crawls user profiles, posts, followers and followees from the WAP site at speeds exceeding 13 million records per day, and stores the data in MongoDB.
This article walks through using Python to collect thousands of Weibo comments about Xue Zhi Qian's response, perform text segmentation, remove stop words, and generate word‑cloud visualizations that reveal shifts in public sentiment before and after the reply.
This article walks through building a Weibo monitoring tool using Python's requests library, covering API analysis, login simulation, endpoint handling, data extraction, and a simple polling mechanism that detects new posts and sends email alerts.
This article walks through the complete process of simulating a Weibo login using Python, covering pre‑login analysis, RSA password encryption, three implementation approaches, and full code examples to help developers avoid common pitfalls when building web‑scraping tools.
In this interview, Sina Weibo platform architect Chen Bo shares the history, design principles, and scaling techniques of Weibo's feed caching system, covering early LAMP setups, hybrid push‑pull models, service‑oriented migration, cloud integration, and practical advice for engineers.
This tutorial explains how to use Python to log into Sina Weibo, download your favorite posts, comments, images, and video links, save the raw HTML, and outlines the steps and script needed to automate the extraction and later processing of the collected data.
The article describes how Weibo evolved from a LAMP‑based first generation to a third‑generation, orthogonal‑decomposed backend architecture featuring three horizontal layers (interface, service, resource) and vertical technical components such as Web V4, MCQ queue, Motan RPC, object storage, SSD cache, and the WatchMan distributed tracing system for monitoring and service governance.
This article introduces a range of algorithms for Sina Weibo—including tag propagation, user similarity via LDA, time‑aware weighting, community detection, PageRank‑based influence ranking, and spam user identification—to illustrate how social network analysis can uncover user interests, influence, and malicious behavior.
This article details the evolution of Weibo's database platform—from its early single‑master design through rapid scaling, vertical and horizontal sharding, automation, Redis enhancements, custom middleware, and future plans—offering practical insights into high‑availability, high‑performance data engineering for large‑scale social media services.
This article walks through creating a Python 2.7 script that logs into Weibo, fetches red‑envelope lists, evaluates their value with a custom algorithm, and automatically claims them, covering required libraries, cookie handling, HTTP GET/POST functions, RSA encryption, and result logging.
The article outlines the design requirements, sharding strategies, capacity planning, cost trade‑offs, and indexing techniques for building a highly available, scalable distributed storage solution for Weibo, focusing on storage‑layer decisions without involving caching or ID generation.
This article describes the background, challenges, technical research, architectural design, module implementation, performance testing, and lessons learned when building a Golang‑based, high‑concurrency, easily extensible proxy to replace twemproxy for Weibo's recommendation system.
The article presents a comprehensive analysis of Weibo's social graph, introducing metrics such as propagation power, intimacy, fan and follow similarity, two‑degree relationships, and relationship circles to model and quantify user interactions and information diffusion within the platform.
The article details how Weibo evolved from a LAMP‑based first generation to a third‑generation, orthogonal three‑layer (interface, service, resource) and four‑vertical (business, technology, monitoring, governance) architecture that supports massive high‑availability, high‑concurrency workloads with middleware such as Web V4, MCQ, Motan, object store, SSDCache, and the WatchMan distributed tracing system.
This article outlines the four‑layer architecture of Weibo’s recommendation system—front‑end (RFront), application (RApp), compute (RCompute), and data (RStore)—detailing each layer’s purpose, technologies such as CRF, nginx+Lua, C++ frameworks, and supporting tools for logging, monitoring, and evaluation.
This article traces the evolution of Weibo's recommendation architecture across three major phases—independent 1.0, layered 2.0, and platform‑centric 3.0—detailing the driving business and technical factors, architectural components, advantages, shortcomings, and key outcomes of each stage.
This article traces the evolution of Weibo’s recommendation architecture across three major phases—independent 1.0, layered 2.0, and platform‑centric 3.0—detailing the environmental drivers, technical components, advantages, shortcomings, and key outcomes of each stage.
This article chronicles the step‑by‑step evolution of Weibo's recommendation system architecture—from the early independent 1.0 stage through the layered 2.0 phase to the current platform‑centric 3.0 design—detailing the driving business and technical factors, architectural components, advantages, shortcomings, and key outcomes at each stage.
The article describes Weibo's large‑scale cross‑IDC distributed image storage system, its upload and download workflows, the challenges of massive seasonal traffic spikes, and the custom pipeline and lightweight compression library designed to achieve strong consistency and low‑latency image delivery.
The article details Weibo's evolution toward multi‑active cross‑datacenter deployment, describing initial motivations, early synchronization attempts, the eventual MCQ‑based messaging solution, operational challenges such as latency and data consistency, and best‑practice recommendations for large‑scale distributed systems.
The article presents a comprehensive overview of Sina Weibo’s third-generation backend architecture, detailing its orthogonal three-tier horizontal model, vertical technical layers, key middleware components such as MCQ, Motan, Object Store and SSDCache, and the WatchMan distributed tracing system that together ensure high availability, scalability, and efficient service delivery.
