Understanding Support Vector Machines: Theory, Example, and Python Code

1 Support Vector Machine Model

In machine learning, a Support Vector Machine (SVM) is a supervised learning model used for classification and regression analysis. It defines a discriminant classifier by a separating hyperplane, outputting the optimal hyperplane that classifies new examples based on labeled training data.

The SVM represents samples as points in space and, through mapping, separates different classes with the widest possible margin. Besides linear classification, SVM can efficiently perform nonlinear classification by implicitly mapping inputs into a high‑dimensional feature space.

2 Example

Assume data points are represented on a two‑dimensional plane:

<code># importing scikit-learn with make_blobs
from sklearn.datasets import make_blobs
import matplotlib.pyplot as plt
# creating datasets X containing n_samples
# Y containing two classes
X, Y = make_blobs(n_samples=500, centers=2,
                  random_state=0, cluster_std=0.40)
import matplotlib.pyplot as plt
# plotting scatters
plt.scatter(X[:, 0], X[:, 1], c=Y, s=50, cmap='spring')
plt.show()
</code>

We can draw several lines as classifiers to separate the classes:

Intuitively, an SVM finds the optimal line or hyperplane that best separates two or more classes.

<code># creating linspace between -1 to 3.5
xfit = np.linspace(-1, 3.5)
# plotting scatter
plt.scatter(X[:, 0], X[:, 1], c=Y, s=50, cmap='spring')
# plot a line between the different sets of data
for m, b, d in [(1, 0.65, 0.33), (0.5, 1.6, 0.55), (-0.2, 2.9, 0.2)]:
    yfit = m * xfit + b
    plt.plot(xfit, yfit, '-k')
    plt.fill_between(xfit, yfit - d, yfit + d, edgecolor='none', color='#AAAAAA', alpha=0.4)
plt.xlim(-1, 3.5)
plt.show()
</code>

Reference

https://www.geeksforgeeks.org/classifying-data-using-support-vector-machinessvms-in-python/