Scikit‑Optimize (skopt): Features, Use Cases, and Code Examples

Scikit‑Optimize (skopt) is an open‑source Python library designed for black‑box optimization, where the objective function can only be evaluated via calls without gradient or analytical information.

Key features include strong adaptability to various black‑box functions (continuous, discrete, constrained or unconstrained), efficient global optimization algorithms such as random‑forest, Gaussian‑process‑based Bayesian optimization, and built‑in support for hyper‑parameter tuning, interactive monitoring, and visualization. It integrates smoothly with the Scikit‑Learn API.

Typical use cases cover machine‑learning hyper‑parameter tuning (e.g., neural networks, SVMs, random forests), experimental design in physics or chemistry, financial and business decision optimization, and engineering problems such as structural design, circuit layout, or robot path planning.

Below are five code examples demonstrating basic usage, hyper‑parameter optimization, constrained optimization, interactive optimization with callbacks, and result visualization.

from skopt import gp_minimize
import numpy as np
# define objective
def objective(x):
    return -(np.sin(5 * x) * (1 - np.tanh(x ** 2)))

bounds = [(-2, 2)]
result = gp_minimize(objective, bounds, n_calls=20)
print(f"Best solution: {result.x[0]}, Objective value: {-result.fun}")

from sklearn.ensemble import RandomForestClassifier
from skopt import BayesSearchCV
# build classifier
rf_clf = RandomForestClassifier(random_state=42)
# define search space
param_space = {
    'n_estimators': (10, 500),
    'max_depth': (None, 30),
    'min_samples_split': (2, 20),
    'min_samples_leaf': (1, 20),
}
# Bayesian optimization
bayes_search = BayesSearchCV(
    rf_clf,
    param_space,
    n_iter=50,
    cv=5,
    scoring='accuracy',
    random_state=42,
)
bayes_search.fit(X_train, y_train)
print(f"Best hyperparameters: {bayes_search.best_params_}")
print(f"Best cross-validation score: {bayes_search.best_score_}")

from skopt import forest_minimize
from skopt.space import Real
# constrained objective
def constrained_objective(x):
    if x < -1 or x > 1:
        return float("inf")  # infeasible
    return x**2

search_space = [Real(-2, 2)]
result = forest_minimize(constrained_objective, search_space)
print(f"Best solution under constraints: {result.x[0]}, Objective value: {result.fun}")

from skopt import gp_minimize, dump, load
from skopt.callbacks import DeltaYStopper, VerboseCallback
# assume objective and bounds are defined
optimizer = gp_minimize(
    objective,
    bounds,
    callback=[
        VerboseCallback(10),  # print every 10 iterations
        DeltaYStopper(0.01),   # stop when improvement <1%
    ],
)
optimizer.run()
# save result
dump(optimizer, 'optimizer_result.pkl')
# later load and continue
optimizer = load('optimizer_result.pkl')
optimizer.run()

from skopt.plots import plot_convergence, plot_objective
# plot convergence curve
plot_convergence(result)
# plot objective contour (2‑D example)
plot_objective(result, dimensions=[(-2, 2), (-2, 2)], n_points=50)

The above examples demonstrate how Scikit‑Optimize can be applied to a variety of black‑box optimization scenarios, from basic function minimization to complex hyper‑parameter tuning, constrained problems, interactive control, and visual analysis of optimization progress.