How to Solve Multiple Linear Regression with sklearn and statsmodels in Python

Solving Multiple Linear Regression with sklearn.linear_model

Using sklearn.linear_model's LinearRegression function you can solve multiple linear regression problems, but the built‑in model evaluation provides only a single metric, so you need to program additional statistical tests yourself. The model is built and fitted with LinearRegression().fit(X, y) where X is the matrix of explanatory variables (excluding the intercept column) and y is the response vector.

Example

Problem

The heat released during cement setting is related to two main chemical components. Given a set of measurements, determine a linear regression model.

Data (x1, x2, y):

<code>1  7  26  78.5
2  1  29  74.3
3 11  56 104.3
4 11  31  87.6
5  7  52  95.9
6 11  55 109.2
7  3  71 102.7
8  1  31  72.5
9  2  54  93.1
10 21 47 115.9
11  1  40  83.8
12 11 66 113.3
13 10 68 109.4</code>

Computation

Code

The obtained regression model shows a high coefficient of determination, indicating a good fit.

Using statsmodels library

statsmodels offers two ways to fit regression models: a formula‑based approach and an array‑based approach.

Formula‑based

<code>import statsmodels as sm
sm.formula.ols(formula, data=df)</code>

where formula is a string such as 'y ~ x1 + x2' and df is a DataFrame or dictionary containing the data.

Array‑based

<code>import statsmodels.api as sm
sm.OLS(y, X).fit()</code>

where y is the response vector and X is the design matrix with a constant column added.

Code

Formula example

<code>import numpy as np; import statsmodels.api as sm
a = np.loadtxt("data/cement.txt")
d = {'x1': a[:,0], 'x2': a[:,1], 'y': a[:,2]}
md = sm.formula.ols('y~x1+x2', d).fit()
print(md.summary())
ypred = md.predict({'x1': a[:,0], 'x2': a[:,1]})</code>

Array example

<code>import numpy as np; import statsmodels.api as sm
a = np.loadtxt("data/cement.txt")
X = sm.add_constant(a[:,:2])
md = sm.OLS(a[:,2], X).fit()
print(md.params)
print(md.summary2())</code>

Reference: 司守奎，孙玺菁 Python数学实验与建模