User Guide

Comprehensive guide to the spotoptim package: surrogate-model-based optimization of expensive black-box functions.

spotoptim is a Python toolbox for Sequential Parameter Optimization (SPO). It optimizes expensive black-box functions by building a surrogate model (typically Kriging) from a small initial design, then iteratively selecting new evaluation points using an acquisition function. The result is a scipy-compatible OptimizeResult object.

The core loop looks like this:

Initial Design  -->  Evaluate  -->  Fit Surrogate  -->  Optimize Acquisition
     ^                                                         |
     |                   Infill (new point)                    |
     +---------------------------------------------------------+

Quick Start

import numpy as np
from spotoptim import SpotOptim
from spotoptim.function import sphere

opt = SpotOptim(
    fun=sphere,
    bounds=[(-5, 5), (-5, 5)],
    max_iter=20,
    n_initial=10,
    seed=0,
)
result = opt.optimize()

print(f"Best x    : {result.x}")
print(f"Best f(x) : {result.fun:.6f}")
print(f"Evaluations: {result.nfev}")

Best x    : [-0.00016718  0.00071419]
Best f(x) : 0.000001
Evaluations: 20

How to Read This Guide

Each page below covers one spotoptim module. Start with The SpotOptim Class for the core workflow, then explore the modules that match your use case.

Module Index

Core Workflow

The SpotOptim Class — Central optimizer: constructor, optimize(), OptimizeResult, variable types, noisy optimization
Acquisition and Infill — Acquisition functions ("y", "ei", "pi"), infill strategies, acquisition optimizers
Surrogate Models — Kriging, SimpleKriging, MLPSurrogate, custom sklearn surrogates

Functions and Designs

Built-in Test Functions — Single- and multi-objective benchmark functions, custom objectives
Sampling and Designs — Latin Hypercube, Sobol, grid, and clustered experimental designs

Analysis and Visualization

Visualization — Optimization progress, surrogate surfaces, contour plots
Reporting and Analysis — Results tables, variable importance, sensitivity analysis
Model Inspection — Feature importance (MDI, permutation), actual-vs-predicted plots

Utilities

Utilities — Boundaries, transforms, PCA, OCBA, parallel helpers
Neural Networks — PyTorch MLP and LinearRegressor for building objectives
Datasets — DiabetesDataset and data loaders for PyTorch workflows

Advanced Topics

Multi-Objective Optimization — Pareto fronts, desirability functions, ZDT/DTLZ benchmarks
Core Infrastructure — SpotOptimProtocol, storage, experiment control
Hyperparameter Definition — ParameterSet fluent API for defining search spaces
Factor Analysis — Exploratory and confirmatory factor analysis
Exploratory Data Analysis — Histograms and boxplots for optimization data
Triangulation Candidates — Delaunay-based candidate point generation

Module Map

src/spotoptim/
├── SpotOptim.py          # Core optimizer
├── core/                 # Protocol, storage, experiment control
├── optimizer/            # Acquisition, steady-state, scipy wrapper
├── surrogate/            # Kriging, MLP surrogate, Nystroem, sklearn pipeline
├── nn/                   # PyTorch MLP, LinearRegressor
├── function/             # Objective functions (single/multi-objective, remote, torch)
├── sampling/             # LHS, Sobol, grid, clustered designs
├── reporting/            # Results extraction, analysis utilities
├── plot/                 # Surrogate visualization, contour, multi-objective
├── utils/                # Boundaries, transforms, PCA, OCBA, TensorBoard, parallel
├── mo/                   # Multi-objective: Morris-Mitchell, Pareto front
├── hyperparameters/      # Parameter set management for NN tuning
├── data/                 # Dataset loaders (e.g., DiabetesDataset)
├── inspection/           # Model/surrogate inspection (importance, predictions)
├── factor_analyzer/      # Factor analysis
├── eda/                  # Exploratory data analysis
└── tricands/             # Triangulation-based candidate generation