Mastering Statistics: From Data Basics to Regression Analysis

Statistics

Statistics is the discipline that studies the collection, organization, description, analysis, and interpretation of data. Its purpose is to uncover inherent patterns and regularities in data sets, enabling accurate predictions and informed decision‑making.

Basic Concepts

Data ("data") originates from the Latin word meaning "known facts". Typical data types include:

Qualitative (categorical) data : non‑numeric attributes such as gender or color. Sub‑types: nominal variables (no order) and ordinal variables (ordered categories).

Quantitative (numerical) data : numeric values that support arithmetic operations. Sub‑types: interval variables (equal intervals, no true zero, e.g., temperature) and ratio variables (true zero, e.g., height, weight).

Dataization is the process of converting phenomena into quantifiable forms suitable for tabular analysis.

Data collection sources include statistical surveys, sensor recordings, and new‑media network data (social media, messages, images, video, audio).

Data analysis involves describing system characteristics, identifying dynamic patterns, and forecasting future states.

Descriptive Statistics

Descriptive statistics visualizes data, turning abstract concepts into concrete visual insights.

"Placing numbers in visual space makes hidden patterns easier to see and yields surprising results." – Nathan Yau

Purpose : Use data to describe patterns, tell the truth, and solve problems.

Mindset : Problem‑oriented storytelling with data.

Techniques : Strong data awareness, deep system thinking, sensitivity to information points, especially outliers.

Tools : Charts.

Accuracy – clear expression of the theme.

Prominence – highlight key information points.

Beauty – use colorful graphics.

Balance – combine text with charts.

Conciseness – avoid unnecessary detail.

Data Visualization Chart Tools

Bar Chart : Shows distribution of qualitative data.

Pie Chart : Illustrates structural features of data.

Area Chart : Shows dynamic ratio structures for quantitative data.

Line Chart : Depicts dynamic changes of quantitative data.

Scatter Plot : Shows relationship between two variables.

Bubble Chart : Visualizes three variables across multiple samples.

Radar Chart : Displays quantitative relationships among several qualitative variables.

Summary Statistics

Measures of Central Tendency

Median, mode, and mean describe data distribution, each with distinct properties and use cases.

Median : The middle value after sorting; unique for a given data set; insensitive to extreme values; applicable to ordinal data.

Mode : The most frequent value; may be non‑unique; works for nominal data; also robust to extremes.

Mean : Arithmetic average; unique; sensitive to extreme values; suitable for ratio/interval data.

Measures of Dispersion

Range, variance, standard deviation, and coefficient of variation quantify how spread out data are.

Range : Difference between maximum and minimum; affected by outliers; interquartile range reduces this effect.

Variance : Average squared deviation from the mean; distinguishes population variance and sample variance.

Standard Deviation : Square root of variance; returns to original units.

Coefficient of Variation : Standard deviation divided by mean; dimensionless, useful for comparing variability across datasets.

Correlation Analysis

Linear relationships are expressed as Y = f(X) where each X maps to a unique Y. Stochastic relationships involve a distribution of Y for a given X. Correlation coefficients quantify the strength of association.

Pearson Correlation

Pearson's r measures linear correlation between two continuous variables, calculated as the covariance divided by the product of standard deviations, ranging from –1 to 1.

Spearman Correlation

Spearman's rho assesses monotonic relationships by ranking data, making it suitable for ordinal variables, non‑linear trends, and data with outliers.

Regression Analysis

Historical Background

Regression originated with Francis Galton's study of inheritance in the 19th century, later formalized by Karl Pearson. Modern regression models quantify the relationship between independent variables X and dependent variable Y for prediction and classification.

Model Types

Simple Linear Regression : Y = β₀ + β₁X + ε.

Multiple Linear Regression : Y = β₀ + β₁X₁ + … + β_kX_k + ε.

Non‑linear Regression : Polynomial, exponential, etc.

Logistic Regression : Uses a sigmoid function for classification.

Parameter Estimation

The least‑squares method minimizes the sum of squared residuals to estimate model parameters, improving fit accuracy.

Model Evaluation

R² (Coefficient of Determination) – proportion of variance explained.

F‑test – assesses overall linear relationship (significance < 0.05).

t‑test – tests significance of individual coefficients (p‑value < 0.05).

Overfitting

Overfitting occurs when a model performs well on training data but poorly on new data, often due to insufficient samples, excessive model complexity, or noisy features.

Mitigation strategies include increasing data volume, simplifying the model, and cleaning noisy data, along with validation techniques such as cross‑validation and leave‑one‑out validation.

Quasi‑Linear Regression

For non‑linear relationships, quasi‑linear regression linearizes the problem (e.g., polynomial models) before applying linear techniques.