This guide walks through essential Linux shell commands—from basic file viewing and simple statistics to powerful exploratory analysis tools—showing how data engineers can efficiently inspect, process, and batch‑handle logs and other data files using the command line.
This article investigates whether the 13‑digit user IDs used in Chinese car‑license lotteries are truly random, revealing how the ID generation, seed‑based selection, and hidden patterns—especially the influential seventh digit—affect outcomes, and demonstrates that simple linear models can achieve an AUC of around 0.8 in predicting winners, while also discussing the system’s opacity across major cities.
This article explains how comprehensive user profiling—combining static demographics and dynamic behavior logs—feeds recommendation engines, detailing data sources, feature extraction, ranking formulas, and the long‑term goals of delivering personalized, high‑quality content to users.
This article explains how to build a comprehensive mobile app quality monitoring framework by defining quality standards, setting capability indicators, and leveraging data acquisition, analysis, and visualization to continuously improve product reliability and user experience across different development stages.
The article explains how internet products use data analysis—covering industry traits, retention curves, magic moments, and extensive A/B testing—to drive user growth, evaluate experiments, and align product improvements with measurable metrics.
This article introduces dataplay2, an open‑source self‑service BI platform built with Flask, pandas, scikit‑learn on the backend and React, ECharts, D3, and other JavaScript libraries on the frontend, detailing its architecture, installation steps, core features such as data upload, visualization, classification, clustering, and future improvement ideas.
This article details Baidu Doctor’s comprehensive testing and monitoring strategies, covering user experience data analysis, source data trust, online monitoring systems, log‑based automated checks, retrieval backend testing, evaluation metrics, Badcase mining, and user search habit analysis to ensure high‑quality medical O2O services.
This article introduces Splunk as a specialized IT search engine for big data, detailing its data collection, indexing, analysis, reporting, security integration, scalability, and practical use cases such as operational troubleshooting, compliance auditing, and application performance monitoring.
The article examines what big data really means, its core 4 V characteristics, current limitations in China, the overhyped value of data, the importance of business‑driven applications, and why starting from small, relevant data is essential for true predictive power.
The article explains why operators should start from clear goals rather than events, defines meaningful metrics such as user retention and API call volume, shows how to break down and evaluate these metrics, and offers practical advice on data collection, benchmarking, and continuous improvement.
The article examines the shortcomings of visual programming tools for data scientists, highlighting issues such as poor version control, limited IDE flexibility, lack of open‑source integration, and reduced modularity, and argues that code‑first approaches remain more effective in enterprise environments.
This article introduces the GWPY Python package for analyzing LIGO gravitational‑wave data, explains the physics of gravitational waves and LIGO, shows how to install and use the library with object‑oriented examples, and demonstrates data visualization through complete code snippets.
This article explains what product evaluation is, why it matters, and outlines a step‑by‑step process—including defining goals, setting standards, selecting methods, executing tests, analyzing results, and presenting findings—to ensure both functional and experiential quality of products.
The article reveals the inner workings of Facebook’s Growth Team, detailing its data‑driven culture, organizational structure, key leaders, and engineering‑focused strategies that have propelled the platform to billions of active users, offering a rare behind‑the‑scenes look.
The article outlines data science as an interdisciplinary field centered on three key steps—pre‑processing, interpretation, and modeling—while providing concise recommendations of foundational books for R, Python, exploratory analysis, machine learning, and essential tools to guide practitioners.
Aspiring data specialists should cultivate curiosity and hands‑on experience with production‑grade tools, and this guide highlights seven indispensable Python libraries—IPython, GraphLab Create, pandas, PuLP, matplotlib, scikit‑learn, and Spark—each explained with key features to boost your data‑science career.
This article presents a PHP‑based web‑scraping experiment that collects and visualizes several months of data from the “Da Gai Er” board of the Caoliu forum, revealing user activity patterns, image hosting distribution, registration trends, and overall forum health through charts and statistical summaries.
This article recounts a recent cross‑border shopping user research project, detailing preparation, requirement gathering, timeline planning, execution, data analysis, report writing, and final sharing, while highlighting practical insights and best practices for effective research and project management.
This article demystifies big data by exposing common misconceptions about its sheer volume, critiquing industry definitions, and highlighting how value emerges from thoughtful analysis, dynamic correlations, and user‑centric perspectives rather than raw data size alone.
The October 2014 100offer programmer auction data reveals which developer skills, locations, and experience levels attract the most interview invitations and highest salary offers, highlighting trends such as higher demand for mobile and web front‑end engineers, the impact of salary increase ranges, and common reasons candidates reject offers.
This guide explains how to conduct comprehensive product data analysis for games, covering methods like 5W2H, multi‑dimensional analysis, user modeling, and the AARRR framework, while detailing key metrics such as DAU, ARPU, LTV, CAC and ROI to drive informed decisions.
At Baidu’s Technology Salon #53 on August 16, a near‑full audience heard data scientist Shen Zhiyong explain the origins of Baidu’s tourist‑flow prediction tool and Liu Ying discuss the evolution of social‑economic forecasting from classic “beer‑and‑diapers” models to modern big‑data analytics, followed by an interactive Q&A and prize‑giving session.
The article argues that predictability—evident in Baidu’s Gaokao essay forecasts and formulaic Hollywood blockbusters—arises when markets favor safe, widely appealing content, making simple systems like baseball easy to predict while complex systems with feedback loops resist accurate forecasting despite apparent creativity.
This guide explains how interaction designers can develop data‑analysis skills—from collecting and understanding data sources to applying analytical methods and communicating results—to make evidence‑based design decisions and improve product outcomes.
This guide explains how interaction designers can systematically collect, analyze, and apply data—clarifying purpose, understanding sources, using core analytical methods, communicating results, and avoiding common pitfalls—to make evidence‑driven design decisions.
