Tagged articles
data preprocessing
92 articles · Page 1 of 1
This guide details a complete pipeline for constructing LLM pre-training datasets: splitting Markdown documents into semantic chunks (1500 chars, no overlap) via heading-aware parsing, greedy packing, and oversized-unit handling, then filtering noise through rule-based checks (length, citations, symbols) and LLM-based quality judgment (completeness, relevance, cleanliness) to produce LLamaFactory-ready JSONL corpora.
LLM pre-trainingLLM-as-JudgeLlamaFactory
0 likes · 22 min read

Many analysts rush to build dashboards on raw data, only to discover mismatched numbers and business push‑back, because the data was never properly cleaned; this article outlines a structured, rule‑based, verifiable and reusable data‑cleaning process that starts with clear goals, proceeds through inventory, rule definition, standardization, mapping, validation, and ends with documented deliverables.
AnalyticsETLdata cleaning
0 likes · 13 min read

This article walks through the essential steps of turning raw natural language into machine‑readable numbers, covering categorical vs. numerical features, one‑hot encoding of categorical data, tokenization, building vocabularies, and using word embeddings, illustrated with an IMDB sentiment‑analysis example in Keras.
IMDB sentiment analysisKerasNLP
0 likes · 7 min read

This guide breaks down the complete workflow for building a large language model—from tokenization and pre‑training to data curation, scaling laws, alignment via RLHF/DPO, and robust evaluation—showing why architecture is less critical than data, scaling, and engineering.
AI EngineeringLLM trainingModel Evaluation
0 likes · 12 min read

This guide walks AI engineers and project leaders through the full lifecycle of high‑quality dataset creation—from defining requirements and setting annotation standards to data collection, preprocessing, labeling, augmentation, evaluation, and continuous iteration—providing concrete metrics, compliance rules, and tool recommendations to avoid common pitfalls.
Model Evaluationai datasetannotation standards
0 likes · 16 min read

The article explains that data cleaning is a prerequisite for any AI or data‑driven project, describes common dirty‑data problems, outlines real‑world use cases, details the major pain points, and provides a practical, step‑by‑step workflow plus tool‑based solutions to achieve reliable, scalable data quality.
AIETLdata cleaning
0 likes · 12 min read

This article walks through the entire feature‑engineering pipeline—data cleaning, missing‑value imputation, encoding, outlier handling, scaling, feature construction, and selection—using Pandas and Scikit‑learn, and shows how to wrap the steps into a reproducible Scikit‑learn Pipeline.
PandasPipelinedata preprocessing
0 likes · 9 min read

This article walks through a full feature‑engineering pipeline using Pandas and Scikit‑Learn, covering data inspection, missing‑value imputation, categorical encoding, outlier handling, scaling, feature construction, selection, and a final Pipeline that prepares clean, predictive features for a logistic‑regression model.
PandasPipelinedata preprocessing
0 likes · 9 min read

The article explains that setting the mean to zero recenters data around the origin, making optimization algorithms converge faster, while scaling variance to one equalizes feature scales so no single feature dominates, illustrated with examples and visualizations of how standardization improves machine‑learning models.
data preprocessingfeature scalingmachine learning
0 likes · 5 min read

The article breaks down a big‑data project into four essential phases—data ingestion, preprocessing, storage, and analysis—explaining common pitfalls, practical tool choices, and why aligning technology with business needs is crucial for success.
Data IngestionData StorageHadoop
0 likes · 10 min read

The article walks through nine essential data preprocessing techniques—handling missing values, detecting outliers, removing duplicates, standardizing features, applying transformations, encoding categorical variables, extracting time‑based features, integrating multiple data sources, and selecting relevant subsets—to improve model accuracy and reliability.
categorical encodingdata integrationdata preprocessing
0 likes · 12 min read

The article explains various learning paradigms (supervised, unsupervised, self‑supervised, semi‑supervised, and reinforcement), describes dataset types and quality considerations, outlines preprocessing steps like filtering, deduplication, and tokenization, and discusses scaling laws linking model size, data volume, and compute resources, with concrete examples and code.
Model TrainingScaling LawsSelf-Supervised Learning
0 likes · 26 min read

Data cleaning is the foundation of any analysis; the article walks through profiling, handling missing values, removing duplicates, correcting outliers, standardizing formats, and linking tables, while illustrating each step with real‑world examples and warning against common shortcuts that waste time.
data cleaningdata preprocessingdata quality
0 likes · 10 min read

This article outlines eight practical techniques—including data augmentation, handling missing values, feature engineering, algorithm selection, hyperparameter tuning, ensemble methods, and cross‑validation—to systematically improve the accuracy of Python machine‑learning models, supported by explanations, examples, and code snippets.
Ensemble Methodscross-validationdata preprocessing
0 likes · 16 min read

This article details how Pinterest’s senior staff engineer Dr. Luo leveraged the open‑source Ray framework to overcome LLM data‑preprocessing bottlenecks, describing the system’s architecture, key features such as map_batches, Carry‑Over Columns and Accumulators, and the dramatic performance and cost improvements achieved.
LLMPinterestRay
0 likes · 12 min read

This article presents 19 practical Sklearn functions—ranging from outlier detection to hyper‑parameter search—that replace manual data‑science steps, each illustrated with concise code examples and performance comparisons.
Model EvaluationPipelinedata preprocessing
0 likes · 24 min read

Data‑Juicer is an open‑source, one‑stop multimodal data processing system that provides fine‑grained operators, scalable pipelines, and ready‑made recipes to deliver high‑quality, diverse, and model‑friendly data for large language model pre‑training, fine‑tuning, and multimodal applications.
AILLMdata preprocessing
0 likes · 22 min read

This article presents ten concise, practical one‑line Python code snippets—ranging from loading CSV data with Pandas to building sophisticated Scikit‑learn pipelines—that streamline common machine‑learning tasks such as data cleaning, encoding, splitting, scaling, model training, evaluation, cross‑validation, and prediction.
One-hot encodingPandasPipeline
0 likes · 10 min read

This article provides a comprehensive guide to decision tree algorithms, covering their theoretical foundations, key components, construction workflow—including data preprocessing, feature selection, tree growth, stopping criteria, and pruning—followed by an overview of popular variants like ID3, C4.5, CART, practical advantages, applications, and a complete Python implementation using scikit-learn.
ClassificationPythondata preprocessing
0 likes · 29 min read

This tutorial demonstrates how to convert a JSON lyric database into Excel, filter Jay Chou songs, perform Chinese word segmentation with Jieba, compute word frequencies, and create visualizations such as word clouds using Python code and online tools.
PandasPythondata preprocessing
0 likes · 9 min read

This tutorial walks through a complete house price prediction project, covering data collection from Kaggle, preprocessing with pandas and scikit‑learn, model training using RandomForestRegressor, evaluation, and deployment of a Flask API for real‑time predictions, providing full code examples.
FlaskPythondata preprocessing
0 likes · 9 min read

This tutorial walks through consolidating the first ten days of learning by covering data preprocessing, feature engineering, model training with linear regression, decision tree, and random forest, model evaluation using cross‑validation, and finally saving and loading the best model, all illustrated with complete Python code examples.
Model TrainingPythondata preprocessing
0 likes · 9 min read

This guide walks through using Pandas for data cleaning, feature engineering, and preparation, then demonstrates building, evaluating, and persisting a machine‑learning model with Scikit‑learn's pipeline and RandomForestClassifier in Python.
Model Trainingdata preprocessingmachine learning
0 likes · 5 min read

This article provides a comprehensive, step‑by‑step guide to training customized large language models, covering industry‑specific needs, data privacy, meticulous data cleaning, optimal data‑ratio balancing, token budgeting, GPU memory accounting, LoRA fine‑tuning techniques, and practical evaluation metrics for robust AI deployment.
AI TrainingGPU memoryLoRA
0 likes · 23 min read

This article introduces the Daily_and_Sports_Activities sensor dataset, details its structure and characteristics, provides a Python preprocessing pipeline with sliding‑window segmentation and Z‑score normalization, and reports CNN training results achieving 87.93% accuracy on activity classification.
CNNUCIdata preprocessing
0 likes · 9 min read

This article explains the importance of data dimensionality reduction and feature extraction in machine learning, and provides a step‑by‑step guide with PHP code examples—including library installation, data preprocessing, PCA‑based reduction, and feature selection techniques—demonstrating how to handle large datasets efficiently.
PCAPHPdata preprocessing
0 likes · 6 min read

This article provides a comprehensive guide to data cleaning in Python, covering common data issues, methods for handling missing values, duplicates, categorical inconsistencies, and text normalization, illustrated with 21 detailed code examples using pandas and matplotlib.
Pandasdata cleaningdata preprocessing
0 likes · 16 min read


HelloTechMar 14, 2024 · Artificial Intelligence
Feature engineering transforms existing data into new predictive variables through manual analysis or automated pipelines, encompassing single‑variable encoding, pairwise arithmetic, group‑statistics, multi‑variable combinations, time‑series and text derivations, with tools like Deep Feature Synthesis and beam‑search to generate and select useful features.
Time Seriesautomated featuresdata preprocessing
0 likes · 17 min read

This article presents ten essential data preprocessing methods—including handling missing values, type conversion, standardization, encoding, smoothing, outlier treatment, text cleaning, word frequency counting, sentiment analysis, and topic modeling—each explained with clear Python code snippets.
PandasPythondata cleaning
0 likes · 9 min read

This article walks through cleaning irregular fan data—removing spaces, Chinese characters, asterisks, and missing brackets—using Python's pandas library, provides the full code snippet, demonstrates the resulting DataFrame, and shares practical tips for preparing data before analysis.
Code TutorialPandasdata cleaning
0 likes · 5 min read

This article presents a comprehensive collection of Python code snippets demonstrating essential data cleaning methods—including handling missing values, outlier detection, type conversion, formatting, duplicate removal, normalization, one‑hot encoding, text preprocessing, and dataset merging—providing practical guidance for preparing data for analysis or machine‑learning tasks.
Pandasdata cleaningdata preprocessing
0 likes · 7 min read

This tutorial demonstrates how to preprocess a Chinese lyrics JSON dataset, extract Jay Chou's songs using Python, perform word segmentation with Jieba, compute word frequencies, and create visualizations such as word clouds both programmatically and with online tools.
NLPdata preprocessingjieba
0 likes · 9 min read

This article presents practical feature‑engineering methods for ten common business domains, explaining the purpose of each feature, the extraction technique, and providing ready‑to‑run Python code snippets to help build more accurate predictive models.
Business Analyticsdata preprocessingfeature engineering
0 likes · 7 min read

This guide walks through ten core data preprocessing methods—including handling missing values, type conversion, standardization, encoding, smoothing, outlier treatment, text cleaning, word‑frequency counting, sentiment analysis, and topic modeling—each illustrated with concise Python code examples.
PandasPythondata preprocessing
0 likes · 9 min read


政采云技术Sep 21, 2023 · Fundamentals
RFM analysis is a powerful tool for understanding customer value and behavior, enabling businesses to optimize marketing strategies, improve customer satisfaction, and increase business revenue through data-driven customer segmentation.
Customer BehaviorMarketing OptimizationRFM Analysis
0 likes · 16 min read

This article analyzes upgrade paths of major LLMs—ChatGLM, LLaMA, Baichuan—detailing performance, context length, and architectural changes, then examines essential capabilities, data cleaning, tokenizer and attention design, and offers practical guidance for balanced scaling and efficient model construction.
BaichuanChatGLMLLM Architecture
0 likes · 32 min read

This article outlines essential data cleaning steps—including handling missing and duplicate values, type conversion, outlier treatment, text processing, standardization, sampling, and merging—providing concise Python pandas code snippets for each technique to improve data quality for analysis.
PandasPythonbig data
0 likes · 5 min read

GoldMiner, a new system from Alibaba Cloud’s PAI platform, elastically scales deep learning data pre‑processing pipelines, dramatically improving training performance up to 12.1× and GPU cluster utilization by 2.5×, and its underlying research was accepted at SIGMOD 2023.
GPU utilizationSIGMODdata preprocessing
0 likes · 5 min read

This article demonstrates how to use Python's concurrent.futures module to parallelize image resizing, turning a single‑process script into a multi‑core solution with just three additional lines of code, achieving up to a six‑fold speed‑up on typical CPUs.
Pythonconcurrent.futuresdata preprocessing
0 likes · 7 min read

Before training large language models like LLaMA, MetaAI applies a multi‑stage CCNet pipeline that crawls web data, stores it in WET format, deduplicates paragraphs, detects and filters languages using fastText, and further refines content by similarity to Wikipedia and citation‑based linear models.
CCNetLlamadata preprocessing
0 likes · 7 min read

Factor analysis, a statistical technique for uncovering underlying common factors among variables, is explained alongside its distinction from PCA, detailed procedural steps, adequacy tests, and a hands‑on Python implementation using the factor_analyzer library with visualizations and factor rotation methods.
Pythondata preprocessingfactor analysis
0 likes · 10 min read

This tutorial walks beginners through the essential steps of data preprocessing for any machine learning model, covering library imports, dataset loading, handling missing values, encoding categorical features, splitting into train‑test sets, and applying feature scaling using Python’s scikit‑learn.
One-hot encodingPythondata preprocessing
0 likes · 11 min read

This article details a data‑mining project that uses historical World Cup match data, extensive feature engineering, and various machine‑learning algorithms—including neural networks, logistic regression, SVM, decision trees, and random forests—to predict the champion of the 2022 tournament, while analyzing model errors and proposing improvements.
ClassificationModel EvaluationWorld Cup
0 likes · 7 min read

Before statistical analysis in R, you need to preprocess data by sorting vectors with sort(), rank(), order() or arrange(), merging datasets horizontally with merge() or cbind() and vertically with rbind(), and handling missing values using NA, NaN, na.rm, and na.omit functions.
Rdata preprocessingmerging
0 likes · 4 min read

This guide walks through a comprehensive set of feature‑selection strategies—from removing unused or missing columns to handling multicollinearity, low‑variance features, and using PCA—complete with Python code examples and visualizations to help you build leaner, more interpretable machine‑learning models.
Pythondata preprocessingdimensionality reduction
0 likes · 18 min read


ITPUBSep 15, 2022 · Artificial Intelligence
In the era of deep learning, feature engineering remains crucial for recommendation and search advertising because it bridges raw relational data and models, improves performance, reduces complexity, and handles high‑cardinality, large‑scale, and time‑sensitive scenarios with robust transformations and statistical encoding.
AIdata preprocessingfeature engineering
0 likes · 20 min read

This article explains how feature engineering in game data involves classifying structured and unstructured, quantitative and qualitative features, and details practical methods for handling missing values and outliers to improve machine‑learning model performance.
data preprocessingfeature engineeringgame data
0 likes · 9 min read

This article explains why meticulous feature engineering remains crucial in recommendation and search advertising, outlines what constitutes good features, describes common transformation techniques such as scaling, binning, and encoding, and provides practical examples and Q&A for practitioners.
AIdata preprocessingfeature engineering
0 likes · 18 min read

This article explains why discretizing continuous variables improves model stability, introduces three common binning techniques—equal-width, equal-frequency, and clustering—and demonstrates how to implement each using scikit‑learn's KBinsDiscretizer with Python code examples on a synthetic score dataset.
KBinsDiscretizerPythondata preprocessing
0 likes · 5 min read

This article demonstrates four Pandas techniques—between with loc, cut, qcut, and value_counts—to discretize numeric variables into bins, assigning grades A, B, C to exam scores, and shows how to generate synthetic data, define bin boundaries, and count records per bin.
PandasPythonbinning
0 likes · 9 min read

This article explains the GM(1,1) grey prediction model, covering data preprocessing, model construction, parameter estimation, error testing methods, and how to generate forecasts, providing a practical guide for applying the technique to time‑series data.
Error AnalysisGM(1,1)Grey Model
0 likes · 4 min read

This article explains essential time series preprocessing techniques—including data sorting, handling missing values with interpolation methods, applying rolling averages, Fourier transform denoising, and detecting anomalies using rolling statistics, isolation forests, and K‑means clustering—illustrated with Python code on the AirPassengers and Google stock datasets.
DenoisingPythonTime Series
0 likes · 9 min read

This article presents a comprehensive, practical methodology for feature engineering that combines brute‑force AutoML‑style dictionary searches, business‑logic‑driven feature creation, and feature‑importance‑guided refinement, illustrating each approach with real Kaggle competition examples and concrete code snippets.
AutoMLKaggleModel Performance
0 likes · 12 min read

This article explains how a five‑line pandas script can generate frequency‑based encodings for categorical columns, why such encodings help low‑frequency categories, and how they integrate with feature crossing and industry use cases like ad exposure, PV/UV counting, and fraud detection.
GBDTcategorical encodingdata preprocessing
0 likes · 6 min read

This article explains the concept of feature engineering, illustrates it with a height‑weight classification example, compares kernel‑enhanced models to handcrafted features like BMI, discusses its impact on model performance, and highlights practical tips and domain‑specific considerations.
AIdata preprocessingfeature engineering
0 likes · 10 min read


ITPUBSep 10, 2021 · Fundamentals
This guide explains why data cleaning is essential, categorizes common problems such as missing values, noise/outliers, and inconsistent records, and provides step‑by‑step procedures and Pandas code snippets to detect, diagnose, and remediate each issue in real‑world datasets.
Pandasdata cleaningdata inconsistency
0 likes · 9 min read

This article presents a collection of straightforward yet practical Python functions for data cleaning tasks—including dropping columns, changing data types, converting categorical variables, handling missing values, removing unwanted characters, trimming whitespace, conditional concatenation, and converting string timestamps—designed to streamline preprocessing in data analysis projects.
data preprocessingdata science
0 likes · 7 min read

This article details how the logistics HR data preprocessing platform processes around 20 million daily records by adopting HBase for high‑performance, scalable, column‑oriented storage, covering its architecture, read/write mechanisms, best practices, and performance considerations.
HBaseNoSQLbig data
0 likes · 10 min read


58 TechJul 23, 2021 · Artificial Intelligence
This article details the background, MMoE model architecture, baseline setup, environment configuration, data preprocessing, training process, evaluation results, and department information for the 58.com recruitment recommendation AI competition using the WPAI platform.
MMoEPyTorchdata preprocessing
0 likes · 11 min read

This article introduces the HR performance data preprocessing platform’s requirements, explains why HBase was selected as the storage solution, details its core concepts, architecture, data write/read processes, best practices, limitations, and presents performance metrics demonstrating its suitability for large‑scale, high‑throughput workloads.
Database ArchitectureDistributed DatabaseHBase
0 likes · 12 min read

This article explains how HBase’s distributed column‑oriented architecture, high‑performance read/write capabilities, and flexible schema make it a cost‑effective solution for handling massive, unstructured HR performance data, covering its core concepts, cluster operation, best practices, and performance metrics.
Distributed DatabaseHBasebig data
0 likes · 11 min read

The article reviews the most important additions in scikit‑learn 0.24, including faster hyper‑parameter search methods, ICE plots, histogram‑based boosting improvements, new feature‑selection tools, polynomial‑feature approximations, a semi‑supervised classifier, MAPE metric, enhanced OneHotEncoder and OrdinalEncoder handling, and a more flexible RFE interface.
Model EvaluationPythondata preprocessing
0 likes · 8 min read

This tutorial explains why and how to reduce the number of features in the Kaggle Mushroom Classification dataset with Python, covering preprocessing, various feature‑selection methods (filter, wrapper, embedded), code examples, model training, performance impact, and visualisation of results.
Mushroom datasetPythonRandom Forest
0 likes · 14 min read

This article explains how feature engineering transforms raw data into numerical representations for machine‑learning models, covering mapping of numeric and categorical values, one‑hot and multi‑hot encoding, sparse representations, scaling, handling outliers, binning, data quality checks, and feature interactions to capture non‑linear relationships.
data preprocessingencodingfeature engineering
0 likes · 20 min read

This article explains the purpose of data exploration and cleaning, outlines core analysis tasks, details missing‑value and outlier handling techniques—including various imputation methods—and illustrates the complete workflow with example images and a histogram‑based distribution analysis.
data cleaningdata explorationdata preprocessing
0 likes · 3 min read

This article walks through five common data‑cleaning scenarios—dropping/renaming columns, handling duplicates and nulls, string manipulation, merging tables, and window‑function ranking—showing practical SQL and Python code that can be applied to large‑scale data warehouses.
PythonSQLdata analysis
0 likes · 9 min read

This guide explains the fundamentals of tensors in TensorFlow.js—including scalars, vectors, and higher‑dimensional tensors—demonstrates how to convert real‑world data such as the Titanic dataset into tensors, and shows how to build, compile, and train a simple neural network model using appropriate layers, loss functions, and optimizers.
JavaScriptNeural NetworkTensorFlow.js
0 likes · 7 min read

This article provides an extensive overview of feature engineering, covering feature understanding, cleaning, construction, selection, transformation, and dimensionality reduction techniques, illustrated with Python code using the Titanic dataset, and offers practical guidelines for improving data quality and model performance in machine learning projects.
PythonTitanic datasetdata preprocessing
0 likes · 44 min read

This tutorial walks through the complete feature‑engineering pipeline for the Alibaba Tianchi “Tmall User Repeat Purchase Prediction” competition, covering data acquisition, memory‑efficient preprocessing, multi‑entity feature construction, statistical aggregations, text vectorisation, embedding generation and stacking‑based model features, all illustrated with Python code and diagrams.
Stackingdata preprocessingfeature engineering
0 likes · 16 min read

This tutorial walks through using pandas for data import, inspection, preprocessing, analysis, and visualization, providing practical code snippets and explanations to help readers understand essential data manipulation techniques in Python.
Pandasdata preprocessing
0 likes · 19 min read

This article presents a hand‑drawn, illustrated walkthrough of the entire machine‑learning pipeline—from dataset definition, exploratory data analysis, preprocessing, and data splitting to model building, algorithm selection, hyper‑parameter tuning, feature selection, and evaluation for both classification and regression tasks.
ClassificationModel Evaluationcross-validation
0 likes · 17 min read

This tutorial walks through collecting and cleaning the MovieLens dataset, constructing rating and record matrices, normalizing ratings, defining a collaborative‑filtering model in TensorFlow, training it with Adam optimizer, evaluating performance, and finally generating personalized movie recommendations for a chosen user.
TensorFlowcollaborative filteringdata preprocessing
0 likes · 10 min read

Feature engineering, which occupies the majority of data scientists' time, is essential for building high‑performing machine‑learning models and involves careful data quality control, diverse construction techniques, rigorous selection, and emerging automation efforts, all of which demand domain expertise and systematic practice.
AIModel Stabilitydata preprocessing
0 likes · 14 min read

This article explains why feature engineering consumes most of a data scientist's time, outlines its critical steps—including data observation, cleaning, transformation, selection, and reduction—covers practical issues such as missing‑value handling, data leakage, and feature stability, and discusses both manual and automated approaches for building effective machine‑learning models.
Model Stabilitydata preprocessingfeature engineering
0 likes · 14 min read

The article presents a complete workflow for predicting next‑day API request volumes by exploring per‑minute traffic data, handling missing values, applying seasonal decomposition, training an ARIMA model on the trend component, and generating confidence intervals to flag anomalous spikes.
ARIMATime Seriesanomaly detection
0 likes · 12 min read

This article provides a comprehensive, step‑by‑step guide to using pandas for data preprocessing, merging, indexing, sorting, grouping, extraction, filtering, sampling, and statistical analysis, complete with clear code examples and visual results to help readers master essential data manipulation in Python.
Pythondata analysisdata merging
0 likes · 13 min read

This article provides a beginner-friendly overview of machine learning fundamentals, covering the definition of data, the distinction between features and labels, types of features, dimensionality, training and test datasets, normalization, supervised and unsupervised learning methods, algorithm selection, development workflow, and recommended Python libraries such as NumPy.
Unsupervised Learningdata preprocessingfeatures
0 likes · 12 min read

This article describes how a GBDT‑based prediction model was built to forecast file release application parameters such as volume ratio, target audience, and gray level, covering data collection, feature engineering, model training, service deployment, and practical considerations for handling bad cases.
GBDTdata preprocessingfile release
0 likes · 8 min read

Effective short‑video recommendation relies on meticulous feature engineering that transforms raw signals—numerical counts, categorical IDs, content and user embeddings, context and session data—through bucketization, scaling, crossing, and smoothing, then selects and evaluates them via filtering, wrapping, regularization, and importance analysis to mitigate business biases and improve multi‑objective ranking performance.
EmbeddingRecommendation Systembias mitigation
0 likes · 32 min read


58 TechNov 11, 2019 · Artificial Intelligence
The article describes the end‑to‑end architecture, data collection, preprocessing, feature engineering, model selection, training, and hyper‑parameter tuning of 58’s car price estimation platform, which leverages Spark, XGBoost, LightGBM and custom business rules to predict vehicle resale values.
LightGBMXGBoostcar price estimation
0 likes · 11 min read

This comprehensive guide introduces data mining and machine learning concepts, walks through Python data preprocessing techniques, reviews common classification algorithms, demonstrates an Iris flower classification case, and offers practical tips for selecting the most suitable algorithm for a given problem.
Pythonclassification algorithmsdata mining
0 likes · 21 min read

This article explains how to use machine‑learning techniques—including data preprocessing, N‑gram, CBOW, and various RNN/LSTM models—to automatically recommend the next function in a test‑case step sequence, improving writing speed and efficiency for developers.
CBOWLSTMN-gram
0 likes · 4 min read

This article introduces logistic regression, explains its theoretical basis, details key scikit-learn parameters, and provides a complete Python example for breast cancer classification, covering data preprocessing, model training, prediction, and evaluation with classification reports.
ClassificationPythondata preprocessing
0 likes · 7 min read

This article explains that there is no universal solution for selecting machine learning algorithms and outlines practical factors—such as data characteristics, problem type, business constraints, and algorithm complexity—to help practitioners systematically narrow down and pick the most suitable models.
Algorithm SelectionModel Evaluationdata preprocessing
0 likes · 14 min read

A participant from Harbin Institute of Technology shares practical data‑preprocessing tricks, model choices, useful feature ideas, and a resilient mindset gained while competing in the Tencent Advertising Algorithm Contest, offering tips that can help other data scientists handle large‑scale ad data.
Competitiondata preprocessingmindset
0 likes · 5 min read

This comprehensive tutorial walks you through Python data mining and machine learning fundamentals, covering data preprocessing techniques, common classification algorithms, an Iris flower classification case study, and practical tips for selecting the right algorithm, all illustrated with clear code examples and visualizations.
KNNNaive BayesPython
0 likes · 22 min read

This article breaks down the Kaggle toxic comment classification competition, detailing thorough data cleaning, advanced word‑vector techniques, pseudo‑labeling, BPE tokenization, diverse neural models and ensemble strategies, and shares practical insights and pitfalls from the author's nine‑month competition journey.
BPEKaggleNLP
0 likes · 9 min read

This article walks through a complete machine‑learning workflow on the UCI Census Income dataset, covering data exploration, preprocessing (including log‑transformation and scaling), model training with Naïve Bayes, Decision Tree and SVM, performance evaluation, hyper‑parameter tuning via grid search, feature importance analysis, and feature selection, providing code snippets and visualizations.
Model EvaluationPythondata preprocessing
0 likes · 24 min read

The week‑3 champion team from Peking University shares their experience in the Tencent social advertising algorithm contest, detailing data preparation, feature engineering, and model training strategies that helped them secure the top spot.
Algorithm CompetitionTencentXGBoost
0 likes · 5 min read

This article provides practical guidance on data cleaning, covering the importance of data wrangling, using assertions, handling incomplete records, checkpointing, testing on subsets, logging, optional raw data storage, and validating the cleaned dataset to ensure reliable downstream analysis.
Loggingassertionscheckpoint
0 likes · 7 min read

This article explains how feature engineering and principal component analysis (PCA) can be applied to a two‑feature binary classification problem in R, illustrating data exploration, model evaluation with ROC AUC, and the impact of dimensionality reduction on predictive performance.
PCARbinary classification
0 likes · 9 min read

This article presents nine fundamental laws of data mining—covering goals, knowledge, preparation, experimentation, patterns, insight, prediction, value, and change—explaining how business objectives and domain expertise drive each stage of the CRISP‑DM process and why technical metrics alone cannot guarantee success.
Business KnowledgeCRISP-DMdata mining
0 likes · 19 min read
