Master the Perceptron: From Theory to Scikit‑Learn Implementation

What Is a Perceptron?

The Perceptron is a linear machine‑learning algorithm for binary classification and one of the earliest artificial neural network models. It differs from logistic regression by using stochastic gradient descent for learning and does not output calibrated probabilities.

Key Learning Objectives

Understand that the Perceptron is a linear binary classifier.

Fit, evaluate, and predict with Scikit‑Learn’s Perceptron class.

Adjust hyper‑parameters such as learning rate and number of epochs.

Tutorial Structure

Perceptron algorithm fundamentals.

Using the Perceptron with Scikit‑Learn.

Hyper‑parameter tuning for the Perceptron.

Perceptron Algorithm Details

The model consists of a single neuron that computes an activation: activation = weight * input + bias If activation > 0 the output is 1, otherwise 0.

The weight update rule is:

weight(t+1) = weight(t) + learning_rate * (expected_i - predicted_i) * input_i

Training proceeds over epochs, shuffling the data each epoch, and stops when the error is low or a maximum number of epochs is reached. Initial weights are small random values; learning rate and max_iter are the main hyper‑parameters.

Perceptron with Scikit‑Learn

Scikit‑Learn provides a Perceptron class where you can set eta0 (learning rate) and max_iter (epochs).

# define model
model = Perceptron(eta0=1.0)

# define model with custom epochs
model = Perceptron(max_iter=1000)

Example dataset creation:

# test classification dataset
from sklearn.datasets import make_classification
X, y = make_classification(n_samples=1000, n_features=10, n_informative=10, n_redundant=0, random_state=1)
print(X.shape, y.shape)

Evaluation using repeated stratified 10‑fold cross‑validation:

from sklearn.model_selection import RepeatedStratifiedKFold, cross_val_score
cv = RepeatedStratifiedKFold(n_splits=10, n_repeats=3, random_state=1)
scores = cross_val_score(model, X, y, scoring='accuracy', cv=cv, n_jobs=-1)
print('Mean Accuracy: %.3f (%.3f)' % (mean(scores), std(scores)))

Typical output: Mean Accuracy: 0.847 (0.052).

Prediction on a new sample after fitting:

# fit model
model.fit(X, y)
# new data row
row = [0.12777556, -3.64400522, -2.23268854, -1.82114386, 1.75466361, 0.1243966, 1.03397657, 2.35822076, 1.01001752, 0.56768485]
yhat = model.predict([row])
print('Predicted Class: %d' % yhat)

Hyper‑Parameter Tuning

The learning rate ( eta0) is crucial. A grid search over several values shows better accuracy with smaller rates.

# define grid for learning rate
grid = {'eta0': [0.0001, 0.001, 0.01, 0.1, 1.0]}
search = GridSearchCV(Perceptron(), grid, scoring='accuracy', cv=cv, n_jobs=-1)
results = search.fit(X, y)
print('Mean Accuracy: %.3f' % results.best_score_)
print('Config:', results.best_params_)

Typical result: Mean Accuracy: 0.857 with eta0 = 0.0001 or 0.001.

Another important hyper‑parameter is the number of training epochs ( max_iter). Using the best learning rate, a grid search over epochs yields similar accuracy across a wide range.

# define grid for epochs
grid = {'max_iter': [1, 10, 100, 1000, 10000]}
model = Perceptron(eta0=0.0001)
search = GridSearchCV(model, grid, scoring='accuracy', cv=cv, n_jobs=-1)
results = search.fit(X, y)
print('Mean Accuracy: %.3f' % results.best_score_)
print('Config:', results.best_params_)

Typical result: Mean Accuracy: 0.857 with max_iter = 10, showing that increasing epochs beyond this point offers little gain.