This guide explains how to use industry sector classification to support value investing by scraping sector and stock data, parsing JSON responses, filtering by keywords, and exposing the results via FastAPI services, then displaying them in a Uniapp front‑end table for easy stock selection.
This article explores how AI-driven models and data pipelines are transforming the chemistry and pharmaceutical sectors by accelerating drug design, improving protein‑antibody predictions, automating patent data extraction, and outlining future goals for end‑to‑end AI‑enabled scientific discovery.
Eight Kuashou research papers covering recommendation systems, multi‑task learning, multimodal large models, large language models, and combinatorial optimization have been accepted to the premier AI data‑mining conference KDD 2025, highlighting the company’s cutting‑edge innovations and their potential impact on the field.
This article walks through using Pandas to directly read HTML pages, extract table data, handle AJAX‑loaded JSON and CSV formats, and save results, providing concise code examples and visual steps for effective Python web scraping and data mining.
This comprehensive guide explores the K‑Means clustering algorithm, covering its mathematical foundation, step‑by‑step procedure, centroid initialization strategies, practical implementation with Python’s Scikit‑learn on the Iris dataset, evaluation metrics, optimization techniques, and diverse applications ranging from image segmentation to bioinformatics.
This article introduces eleven classic dimensionality reduction techniques—including PCA, LDA, MDS, LLE, and t‑SNE—explains their principles, advantages, and limitations, and provides complete Python code examples and resources for each method, making it a valuable guide for beginners in machine learning and data mining.
This guide explains why data analysis is essential, breaks it into three core stages—data collection, data mining, and data visualization—offers practical tool recommendations, and presents principles for efficient learning and skill development.
This article introduces eight core data analysis methods—including association, comparative, clustering, cross, Pareto, quadrant, funnel, and full‑path analysis—explaining their principles, typical use cases, key metrics, and visual examples to help professionals make data‑driven decisions.
On September 5 at the 2024 Inclusion·Bund Conference, Ant InTech announced its first ten award-winning young scholars from top Chinese universities, highlighting their research in artificial intelligence, data processing, cloud computing, security, and related fields, each receiving a 200,000‑RMB grant.
The article presents a curated collection of Ant Group's research papers accepted at KDD2024, summarizing each paper's title, type, link, source, relevant fields, and abstract, covering topics such as graph mining, large language models, fraud detection, recommendation systems, and multimodal medical AI.
The BlackPearl team from Meituan’s Dazhong Dianping division swept all three KDD 2024 OAG‑Challenge Cup tracks—WhoIsWho, PST, and AQA—by deploying innovative large‑model techniques such as iterative text clustering, graft‑learning‑enhanced BERT RAG pipelines, and a Boosting LLM‑for‑Vector search, and have released the code publicly on GitHub.
This article presents a data‑driven football match prediction system that extracts match features, builds machine‑learning models—including linear, SVM, random forest, and deep neural networks—and evaluates their accuracy on European league data, then analyzes betting strategies, limitations, and extensions to stock forecasting.
The study combines team statistics and bookmaker odds with machine‑learning models—including Poisson, regression, Bayesian, SVM, Random Forest, DNN, and LSTM—to predict football match outcomes, identify confidence‑based betting intervals that yield profit, and suggests extensions to broader data, features, and financial trading.
This article demonstrates how to perform data mining with scikit-learn by detailing the full workflow—from data acquisition and feature engineering, through parallel and pipeline processing, to automated hyper‑parameter tuning and model persistence—using the Iris dataset as an example.
This article presents a comprehensive overview of building a user profile tag system—including tag taxonomy, platform architecture, construction methods, update cycles, access patterns, common algorithmic tags, and real‑world applications such as marketing, metric attribution, and A/B testing—illustrated with examples and a detailed Q&A session from a data‑mining senior manager at Qunar.
The article explains why pinpointing core variables is crucial, outlines domain‑knowledge and technical methods such as sensitivity analysis and data mining to discover them, and describes practical ways to turn those variables into quantitative indicators like scoring systems, composite indices, and real‑world examples.
This article walks through solving a Pandas data‑analysis problem raised in a Python community, showcasing multiple Python techniques—including string manipulation, zero‑width spaces, and slicing—to clean and transform data, complete with code snippets and result screenshots.
This article details the design, integration, and operational practices of a comprehensive user and item profiling tag system, covering tag taxonomy, construction methods, update cycles, access strategies, algorithmic implementations, and real‑world applications such as marketing, attribution analysis, and A/B testing.
This article introduces ten data‑mining applications for marketing—including customer segmentation, purchase forecasting, market‑basket analysis, churn prediction, sentiment analysis, response modeling, recommendation systems, brand reputation, competitive analysis, and public‑opinion monitoring—each illustrated with concise Python code examples.
This article demonstrates how to apply the Apriori algorithm and association rule mining using Python's mlxtend library across ten real‑world business scenarios, providing step‑by‑step code examples for retail, e‑commerce, marketing, healthcare, security, CRM, social networks, travel, market basket, and online advertising.
This presentation by senior data mining manager Zhou Yuanwei of Qunar outlines the architecture of an image tagging platform, the construction of a comprehensive tagging system, common algorithmic tags, and real-world applications such as look‑alike marketing, A/B test efficiency analysis, and business attribution, helping audiences understand tag types, design considerations, and value‑driven use cases.
The article showcases seven Meituan research papers accepted at KDD 2023—spanning feed‑stream, cross‑domain, takeaway, bonus allocation, contour‑based segmentation, living‑needs prediction, and multilingual recommendation—detailing their novel methods, real‑world deployments, and concluding with an invitation for academic collaboration.
This article presents Fabarta's application of graph‑based artificial intelligence to anti‑money‑laundering, detailing the business challenges, a five‑step analytical workflow, a synthetic data case study, and a Q&A that explores practical deployment and future directions.
This article introduces user path analysis and the SessionAnalytics open‑source framework, covering business scenarios, technical architecture, data integration, session segmentation, data cleaning, sampling, graph structures, NLP‑based mining, clustering, and visualization techniques for extracting insights from large‑scale user behavior data.
This article explains Python machine‑learning basics, covering data splitting, feature and label concepts, key factors for feature selection, and practical tips for building predictive models, while also offering code snippets and visual illustrations to help readers apply these techniques effectively.
This article walks through a Python web‑scraping challenge involving encrypted parameters, demonstrates a Selenium‑based solution with full code, shares community tips that fix the request payload, and shows how the issue was resolved step by step.
This article provides an extensive overview of big‑data interview subjects, covering browser and mobile log collection methods, data synchronization techniques (batch, real‑time, sharding), offline data development platforms, streaming architectures, data service evolution, performance optimization, and data‑mining layers and applications.
This article presents an end‑to‑end intelligent testing framework that mines development‑stage risk dimensions, conducts admission risk assessment, extracts multi‑dimensional activity data such as coverage metrics, and applies model‑based risk evaluation to guide quality‑assurance decisions and improve release safety.
The article describes an intelligent test‑evaluation framework that gathers performance data, quantifies project, personnel, and code risk dimensions, feeds them into rule‑based and logistic‑regression models to produce risk scores and risk‑driven testing plans, and demonstrates how this approach identified thousands of high‑risk projects, prevented hundreds of bugs, and saved thousands of person‑days.
This tutorial introduces unsupervised clustering, explains its purpose, and walks through installing scikit‑learn and implementing ten popular clustering algorithms—including AffinityPropagation, Agglomerative, BIRCH, DBSCAN, K‑Means, Mini‑Batch K‑Means, MeanShift, OPTICS, Spectral Clustering, and Gaussian Mixture—complete with code examples and visualizations.
The article explains the principles of grey system theory, introduces grey relational analysis as a method for handling sparse information, outlines its mathematical foundations, step‑by‑step modeling process, and demonstrates how the grey comprehensive evaluation method can rank and compare multiple alternatives without requiring large sample sizes or strict statistical assumptions.
This guide walks you through using Python to locate the API endpoint for Eastmoney sector capital flow, send HTTP requests, clean the returned string into proper JSON, convert it to a Pandas DataFrame, and finally save the data locally for further analysis.
This tutorial introduces clustering, explains why no single algorithm fits all data, and provides step‑by‑step Python examples using scikit‑learn for ten popular unsupervised learning methods, complete with code snippets and visualizations to illustrate results.
The article explains classification as a data‑analysis task that builds models to assign new observations to predefined categories, outlines its implementation steps, describes various data types (boolean, nominal, ordinal, continuous, discrete), presents common machine‑learning classifiers such as decision trees and neural networks, and highlights practical applications like crime detection, disease risk prediction, and credit assessment.
This tutorial explains clustering as an unsupervised learning task, outlines why no single algorithm fits all data, and provides step‑by‑step Python code using scikit‑learn to install the library, generate synthetic datasets, and apply ten popular clustering algorithms with visualizations.
This tutorial walks you through using Python and BeautifulSoup to crawl the 8684.cn website, extract city‑specific bus route information, parse classification pages, and collect detailed schedule and station data, while providing reusable code snippets and data‑cleaning suggestions.
This talk reviews the concept of alternative data, illustrates its early use in John Snow's cholera map, explores contemporary AI‑powered systems such as IBM's Debater and satellite‑based poverty estimation, and presents the speaker's own research on using unconventional data for financial‑market risk detection and prediction.
The article explains how the DMAIC framework from Six Sigma and the five‑layer user‑experience model can be combined to guide the definition, measurement, analysis, improvement, and control of data products, especially in gaming contexts, emphasizing systematic design, visualization, and iterative refinement.
This article explains why data mining matters, defines the discipline, outlines its five‑step workflow, and dives into core techniques such as association‑rule mining, classification, clustering, and regression, illustrated with practical examples and visual diagrams.
Drawing on seven Kaggle and KDD Cup victories, the article outlines a multi‑domain modeling optimization strategy—covering recommendation, time‑series, and AutoML problems—alongside a three‑module AutoML pipeline and a three‑stage workflow that emphasize systematic evaluation, bias‑variance balance, and robust model‑fusion for competition and industry success.
This article explains why recommendation systems are essential for B2B construction e‑commerce, describes the types of data they rely on, outlines multi‑channel recall methods, details collaborative‑filtering algorithms with similarity calculations, and presents the four‑stage recommendation pipeline from recall to re‑ranking.
This article explains the two most common data‑mining frameworks—CRISP‑DM and SEMMA—detailing their six and five stages respectively, illustrating each phase with diagrams and highlighting how the iterative nature of data mining drives continuous improvement.
This article describes how Meituan builds a multi‑level, multi‑dimensional retail product knowledge graph to support new‑retail scenarios, detailing its architecture, data acquisition challenges, labeling pipelines, attribute extraction methods, efficiency improvements, human‑machine collaboration, and downstream search and recommendation applications.
This article introduces the ten classic data‑mining algorithms selected by IEEE ICDM—C4.5, k‑Means, SVM, Apriori, EM, PageRank, AdaBoost, k‑NN, Naive Bayes, and CART—explaining their main ideas, advantages, and typical applications for readers seeking a solid foundation in data analysis.
The paper describes Meituan’s retail product knowledge graph—a multi‑layered, multi‑modal system that structures billions of SKUs, attributes, and user insights using hierarchical categories, graph‑enhanced NER, semi‑supervised learning, and expert‑in‑the‑loop validation, enabling precise search, ranking, recommendation, and real‑time merchant optimization.
This article explains the complete data‑mining workflow for the restaurant industry—from defining business goals and sampling relevant data to exploring, preprocessing, modeling, evaluating results, and selecting suitable tools—enabling intelligent dish recommendation, customer segmentation, sales forecasting, and optimal store placement.
This article details Meituan's development of a multi‑level, multi‑dimensional retail product knowledge graph, covering its background in new retail, hierarchical design, attribute modeling, challenges, efficiency improvements, human‑machine collaboration, and its impact on search, recommendation and both C‑ and B‑side services.
This article introduces the third edition of Stanford’s Data Mining Tutorial, highlighting its panoramic roadmap of data‑mining techniques for massive datasets, core features, comprehensive topic coverage, target audience, and supplementary resources while noting its popularity among students and professionals.
iQIYI built an in‑house user‑feedback analysis system that automatically ingests multi‑channel data, classifies and clusters issues, assesses feedback quality, localizes problems, and streamlines repair closure, boosting recall accuracy, alarm precision, closure rates and reducing cycle time across business lines to enhance user experience.
This article explains the distinct concepts, applications, and techniques of data mining and data analysis, highlighting their definitions, typical tools, and how each contributes to extracting insights from large datasets in various industries.
This article examines the technical evolution of POI expiration detection, covering attribute‑based, behavior‑based, and human‑place relationship mining methods, their machine‑learning models, and how they collectively improve map freshness and user experience at scale.
This article demonstrates how to use Python's requests, fake_useragent, and lxml libraries to batch‑scrape nearly a thousand new‑house listings from the 惠民之家 website, extracting 41 fields such as name, price, layout, opening date, plot ratio and green ratio, while handling pagination and anti‑scraping measures.
Learn how to build a Python web scraper that extracts iQIYI movie titles, actors, and scores, parses the data with regex or BeautifulSoup, displays results in a Tkinter GUI with a combobox, and saves the information to a file—all explained with code snippets and screenshots.
This article walks through the complete process of designing a Python web scraper to collect Zhihu answer data, parse author, ID and excerpt fields, store the results in CSV files, and finally visualize the text with a word‑cloud, including all necessary code snippets and explanations.
This article outlines how to design and implement a comprehensive mobile data collection and analysis system using the open‑source Countly platform, covering background requirements, solution selection, architecture, customizations for client, server and dashboard, SDK integration for Android and H5, and practical data mining insights.
The second‑place team tackled KDD Cup 2020’s Multimodal Recall challenge by fine‑tuning ImageBERT and LXMERT on query‑image pairs, generating negatives, applying AMSoftmax and multi‑similarity losses, ensembling weighted predictions, and using score‑based post‑processing, boosting NDCG@5 to 0.8352 and powering Meituan’s multimodal search pipeline.
The Xianyu interest‑tagging system profiles post‑95 users by matching expert and hot‑search keywords to product text, weighting user actions with a TF‑IDF‑based behavior‑statistics pipeline, producing over twenty tags that cover more than half the target cohort and have already doubled click‑through rates for interest‑aligned live streams.
This article explains how intelligent recommendation differs from generic marketing, introduces association‑rule concepts such as support, confidence, and lift, and provides a step‑by‑step Python implementation using the Apriori algorithm to generate and interpret market‑basket recommendations.
This article uses Python to scrape and analyze Douban ratings, user comments, and Tencent video danmu for the TV drama “30 Only”, revealing the show’s explosive popularity, the most discussed characters, and audience sentiment through statistical charts and word‑cloud visualizations.
This article introduces eight essential Python libraries—Gensim, TensorFlow, SciPy, NumPy, Matplotlib, Pandas, Scikit‑Learn, and Keras—that empower developers to clean, prepare, merge, and accurately analyze data for effective data mining.
This tutorial shows how to fetch ten years of stock fundamentals using Tushare, compute a probabilistic "temperature" metric with NumPy and SciPy, and visualize the temperature alongside closing prices in Matplotlib, providing a complete end‑to‑end Python data‑analysis workflow.
This article examines six core data‑science concepts—Big Data, Machine Learning, Data Mining, Deep Learning, Artificial Intelligence, and Data Science itself—explaining their definitions, interrelationships, and how they fit together as pieces of a larger analytical puzzle.
This tutorial explains how to crawl 58.com job listings with Python, extract location, company, and salary information, handle anti‑scraping defenses using realistic headers and random User‑Agents, and save the results into a text file.
This article describes how to collect, clean, and analyze 90,000+ GitHub issues and pull requests from the Kubernetes repository using Kubeflow, TensorFlow, and a fully automated CI/CD pipeline, then build, train, and serve a simple MLP model that classifies release‑note texts as bugs or non‑bugs.
TAL Education's AI Engineering Institute recently had seven of its machine‑learning papers selected for prestigious conferences such as AIED 2020, EDM 2020, ICASSP 2020 and WWW 2020, showcasing advances in speech recognition, data mining, multimodal learning and educational AI applications.
The article details how a data analyst scraped comments from Zhihu, Weibo, and Bilibili about the 2020 Youth Day video “后浪”, applied Baidu AI sentiment analysis and Python word‑cloud generation, and compared the differing emotional tones across platforms.
This tutorial explains how to obtain free proxy IPs, extract their addresses using Python's requests and BeautifulSoup, and continuously validate them to overcome anti‑scraping restrictions when crawling sites such as Baidu Baike for data mining tasks.
The article describes a risk‑control pipeline for detecting high‑volume fraudulent game accounts, detailing data collection from game logs, extensive feature engineering and statistical tests, enrichment via a Neo4j knowledge graph, and a hybrid RandomForest‑GBDT model combined with methods to filter personal accounts.
This article outlines practical ways for individuals, especially students, to generate side income using Python by taking on web‑scraping freelance projects, building data‑driven websites, creating simple automation tools, running blogs or media channels, and even modest stock‑analysis scripts.
By applying sequence pattern mining and unsupervised clustering to Xianyu’s massive event logs, the study abstracts high‑level user behaviors, discovers frequent subsequences, uncovers unknown fraudulent account patterns, expands known fraud cohorts with 99 % precision, and enables richer analyses such as PCA‑based cross‑group comparisons.
This article presents a comprehensive workflow for group risk perception, covering business requirement mining, data acquisition and understanding, feature engineering, model training and evaluation, deployment, and practical user applications, with detailed objectives, methods, and deliverables for each stage.
This article explains core data‑warehouse concepts—including DB, DW, ODS, OLTP, OLAP, BI, and the differing meanings of DM—as well as their relationships, integration examples, and why OLAP cannot replace data mining, providing a concise reference for beginners in data analytics.
This article explains how MaFengWo builds a three‑dimensional user content contribution model—combining activity, popularity, and sharing willingness—to objectively score UGC creators, improve recommendation strategies, and drive more effective travel‑related services.
In an interview, JD data platform leader Li Wei discusses how dynamic balance between business demands and technical improvement, knowledge computing, and AI-driven product quality control can drive innovation, enhance user experience, and shape future R&D management strategies.
The article examines how telemarketing, a data‑intensive industry, leverages various data‑mining methods—including supervised classification, clustering, operations research optimization, anomaly detection, and text mining—to improve lead selection, agent allocation, churn prediction, and voice analysis, while also outlining the key data‑talent roles needed for successful implementation.
This report analyzes 2018 AMAC industry index data using association‑rule mining and several machine‑learning models (Apriori, KNN, Bayesian, decision tree, neural network) to uncover sector linkages, predict index and stock movements, compare model performance, and suggest future improvements.
This tutorial walks you through installing the WordCloud library, resolving the Microsoft Visual C++ compiler error, fixing Chinese font issues, and creating both basic and image‑masked word clouds with sample Python code.
This comprehensive guide introduces data mining and machine learning concepts, walks through Python data preprocessing techniques, reviews common classification algorithms, demonstrates an Iris flower classification case, and offers practical tips for selecting the most suitable algorithm for a given problem.
As online information explodes, traditional data collection methods fall short, prompting the rise of Python web crawlers that use URLs and libraries like urllib, urllib2, and re, while frameworks boost efficiency, enabling fast, accurate, and automated extraction of web data for analysis.
Meituan’s AI‑driven “Superbrain” platform combines real‑time big‑data processing, fine‑grained location perception, high‑precision ETA forecasting, multi‑rider dispatch and dynamic pricing to cut instant food‑delivery times from about an hour to roughly thirty minutes while boosting efficiency, cost savings and user experience.
This article outlines five widely used data‑mining methods—association rules, classification/tagging, clustering, decision trees, and sequential pattern mining—explaining their principles, real‑world examples, and how they help organizations extract actionable insights from massive datasets.
The article outlines a comprehensive data‑analyst competency framework, covering data collection, storage, extraction, mining, analysis, visualization, and practical application, and provides concrete questions, techniques, and tool recommendations to help analysts turn raw data into actionable business insights.
This article outlines the five fundamental aspects of big data analysis—visualization, data‑mining algorithms, predictive analytics, semantic engines, and data quality management—and explains four primary analytical approaches: descriptive, diagnostic, predictive, and prescriptive analysis.
By leveraging tablet-based learning platforms, schools can collect rich usage data, which, when mined, reveals student habits, sentiment, and learning patterns, enabling educators to personalize instruction, improve curricula, and guide strategic decisions, while highlighting the need for data protection and dedicated research centers.
This paper introduces the SCW-LOL algorithm, a local online learning method based on Soft Confidence Weighted that extends a global model with multiple local classifiers, uses online K‑Means for sample assignment, provides theoretical error bounds, and demonstrates superior performance on ten benchmark datasets, especially for multi‑class classification.
The paper presents two inexpensive techniques for extracting geographic aliases of points of interest—comparing high‑dimensional contextual vectors of nearby shipping addresses and analyzing co‑occurring words in identical addresses—to construct a knowledge base that links official names with their synonyms, improving location‑based service accuracy.
Learn how to export WeChat Moments using a third‑party service, set up a Scrapy project in Python, analyze the dynamic JSON responses, and write a crawler to extract timeline data, complete with screenshots and command‑line instructions for a fully functional scraper.
This article explains the importance of modeling stability, describes how to assess feature and model stability using the Population Stability Index (PSI), provides step‑by‑step calculation methods, and shares practical monitoring practices such as rank mapping and daily SQL‑based checks.
This comprehensive tutorial walks you through Python data mining and machine learning fundamentals, covering data preprocessing techniques, common classification algorithms, an Iris flower classification case study, and practical tips for selecting the right algorithm, all illustrated with clear code examples and visualizations.
The JD Big Data R&D team announced that three of its research papers—covering cross‑domain human parsing, multi‑view outlier detection, and orthogonal weight normalization for deep neural networks—were accepted at the prestigious AAAI‑2018 conference, highlighting the department's contributions to computer vision, data mining, and deep learning.
This article explains how to create a Python‑based Scrapy spider that logs into Sina Weibo using cookies, crawls user profiles, posts, followers and followees from the WAP site at speeds exceeding 13 million records per day, and stores the data in MongoDB.
This article explains how to assess unsupervised clustering algorithms by describing realistic user‑watching scenarios, outlining common cluster and algorithm types, presenting five key evaluation criteria, and introducing practical metrics such as RMSSTD, R‑Square, and the improved Hubert‑Gamma statistic.
This article explains what user profiling is, why it’s crucial for recommendation systems, outlines key dimensions such as personal attributes, status, and interests, describes algorithms like classification and autoregressive models, and details offline and real‑time computation methods, evaluation techniques, and practical examples.
This guide explains the fundamentals of association‑rule mining, introduces support, confidence and lift metrics, and demonstrates step‑by‑step how to use the discoverR R package with Apriori and FP‑Growth algorithms to extract and visualize recommendation rules from the classic groceries dataset.
This article explores how simple tagging and user profiling underpin modern recommendation systems, contrasting tag‑based, flexible representations with traditional hierarchical classifications, and examines practical applications such as personalized advertising, industry research, and product optimization.
This article recounts a year-long journey to become a data mining engineer, explaining the role’s value, required skills, tools such as Excel, Tableau, SQL, Python, Scala, Spark, and machine‑learning techniques, and offers practical steps for aspiring professionals.
Python is favored for data‑mining algorithms because its clear syntax, built‑in advanced data structures, easy text handling, extensive libraries, and widespread community support outweigh its slower execution speed compared to Java or C, allowing rapid development and seamless integration with high‑performance code when needed.
By mining over a million computer‑related job postings with weak‑supervised learning and a BiLSTM NER model, this article quantifies how programming languages, development tools, and hardware skills translate into salary value, offering data‑driven guidance for developers and new graduates.
The first Alibaba data‑mining forum in Hangzhou gathered top academics and industry leaders who discussed large‑scale online precise matching, the role of distributed versus single‑machine learning, and the benefits and limitations of deep learning in modern AI applications.
In a Q&A session, a Tencent social ads data mining expert addressed competition participants' questions on data delays, full‑set versus sliding‑window statistics, dataset authenticity, Bayesian smoothing, feature selection, handling missing values, large‑scale training, feature interactions, model stacking, online mini‑batch training, and provided reference resources.
