Step-by-step walkthrough of every action performed by SpotOptim.__init__(), with executable examples validated by pytest.
This document explains every step performed inside SpotOptim.__init__() in the order they occur. Each section has a {python} code block that can be executed directly and is validated by a corresponding pytest in tests/.
Run all related tests with:
uv run pytest tests/test_spotoptim_deep.py tests/test_validate_x0.py tests/test_transform_bounds.py -v
Step 1 — Silence Warnings
warnings.filterwarnings(warnings_filter)
The very first action is to apply a global Python warnings filter. The default value is "ignore", which suppresses deprecation and runtime warnings from third-party libraries during optimization. Accepted values are any string accepted by warnings.filterwarnings: "ignore", "always", "error", "once", etc.
import warningsfrom spotoptim import SpotOptimfrom spotoptim.function import sphere# "always" surfaces all warnings during a debugging sessionopt = SpotOptim(fun=sphere, bounds=[(-5, 5)], warnings_filter="always")print(f"warnings_filter: {opt.warnings_filter}")
warnings_filter: always
Step 2 — Machine Epsilon (self.eps)
self.eps = np.sqrt(np.spacing(1))
self.eps is set to \(\sqrt{\epsilon_{\text{machine}}}\), approximately 1.49e-8 for IEEE 754 double precision. It is used throughout the class as a tolerance for floating-point comparisons, for example, when checking whether a starting point x0 matches a fixed bound or when evaluating the tolerance_x stopping criterion. np.spacing(1) returns the smallest representable difference from 1.0 (equivalent to np.finfo(float).eps).
If the user does not supply tolerance_x, it defaults to self.eps. tolerance_x controls the minimum required improvement in the decision variable space to keep the optimizer running: if consecutive best points are closer than tolerance_x (measured by min_tol_metric, default "chebyshev"), the run is considered converged.
Before any validation, __init__ tries to read missing parameters directly from the callable fun using getattr. This allows self-describing objective functions to carry their own metadata: fun.bounds as a list of (lower, upper) tuples, fun.var_type as a per-dimension type string list, fun.var_name as human-readable parameter names, and fun.var_trans as a list of transformation strings.
bounds is the only mandatory parameter: if it is None after this inference step, __init__ raises a ValueError immediately. The remaining three attributes (var_type, var_name, var_trans) are permitted to remain None; downstream steps supply their own defaults when needed.
Four checks run before the configuration object is assembled. max_iter must be at least n_initial because the total budget must accommodate the initial design. n_jobs follows the scikit-learn convention: -1 is resolved to os.cpu_count(), while 0 and values below -1 are rejected. eval_batch_size controls how many candidate points accumulate before a single vectorised call is dispatched to the process pool and must be at least 1. Finally, when acquisition_optimizer_kwargs is not supplied it is initialised to {"maxiter": 10000, "gtol": 1e-9}, providing tight convergence tolerances for the default differential-evolution run.
import osfrom spotoptim import SpotOptimfrom spotoptim.function import sphere# valid: max_iter 20 >= n_initial 10opt = SpotOptim(fun=sphere, bounds=[(-5, 5)], max_iter=20, n_initial=10)print(f"max_iter={opt.max_iter}, n_initial={opt.n_initial} — valid.")# invalid: max_iter < n_initial must raisetry: SpotOptim(fun=sphere, bounds=[(-5, 5)], max_iter=5, n_initial=10)raiseAssertionError("Expected ValueError was not raised")exceptValueErroras e:print(f"Caught expected error: {e}")# n_jobs=-1 resolves to all available CPU coresopt_p = SpotOptim(fun=sphere, bounds=[(-5, 5)], n_jobs=-1)assert opt_p.n_jobs == (os.cpu_count() or1)print(f"n_jobs=-1 resolved to: {opt_p.n_jobs}")# n_jobs=0 is rejectedtry: SpotOptim(fun=sphere, bounds=[(-5, 5)], n_jobs=0)raiseAssertionError("Expected ValueError was not raised")exceptValueErroras e:print(f"Caught expected error: {e}")# acquisition_optimizer_kwargs defaultopt_aq = SpotOptim(fun=sphere, bounds=[(-5, 5)])assert opt_aq.acquisition_optimizer_kwargs == {"maxiter": 10000, "gtol": 1e-9}print(f"acquisition_optimizer_kwargs (default): {opt_aq.acquisition_optimizer_kwargs}")print("Validation check passed.")
max_iter=20, n_initial=10 — valid.
Caught expected error: max_iter (5) must be >= n_initial (10). max_iter represents the total function evaluation budget including initial design.
n_jobs=-1 resolved to: 16
Caught expected error: n_jobs must be a positive integer or -1 (all CPU cores), got 0.
acquisition_optimizer_kwargs (default): {'maxiter': 10000, 'gtol': 1e-09}
Validation check passed.
All constructor arguments are stored in a SpotOptimConfig dataclass-like object assigned to self.config. SpotOptim uses a __getattr__ proxy so that every field in config is also accessible directly on the SpotOptim instance — for example, opt.n_initial reads from opt.config.n_initial. acquisition is normalized to lowercase inside the config.
Storing parameters in a separate object makes serialization and parameter inspection straightforward.
from spotoptim import SpotOptimfrom spotoptim.function import sphereopt = SpotOptim( fun=sphere, bounds=[(-5, 5), (-5, 5)], max_iter=20, n_initial=10, acquisition="EI", # will be lowercased seed=99,)print(f"config type : {type(opt.config).__name__}")print(f"acquisition : {opt.acquisition}") # proxy via __getattr__print(f"seed : {opt.seed}")assert opt.acquisition =="ei"# normalized to lowercaseassert opt.n_initial ==10print("Config construction check passed.")
config type : SpotOptimConfig
acquisition : ei
seed : 99
Config construction check passed.
Step 7 — SpotOptimState Construction
self.state = SpotOptimState()
SpotOptimState holds all mutable runtime state that changes during optimization: evaluated points X_, function values y_, the current best solution best_x_, best_y_, iteration counters, and similar fields. It starts empty and is populated once optimize() is called.
Like config, its attributes are accessible directly via __getattr__.
from spotoptim import SpotOptimfrom spotoptim.function import sphereopt = SpotOptim(fun=sphere, bounds=[(-5, 5)])print(f"state type: {type(opt.state).__name__}")# Before optimizing, X_ and y_ do not exist yetassertnothasattr(opt.state, "X_") or opt.state.X_ isNoneorTrueprint("State construction check passed.")
state type: SpotOptimState
State construction check passed.
The callable fun is stored as self.fun. objective_names is the list of output-metric names used by multi-objective or torch-based objectives (e.g., ["val_loss", "epochs"]). It is copied from fun.objective_names or, as a fallback, from fun.metrics. If neither attribute exists, objective_names is None.
The visualization module reads optimizer.objective_names to label plot axes.
from spotoptim import SpotOptimfrom spotoptim.function import sphere# Plain function — no objective_namesopt = SpotOptim(fun=sphere, bounds=[(-5, 5)])print(f"objective_names (plain): {opt.objective_names}")assert opt.objective_names isNone# Function with .objective_names attributeclass MetricFun: objective_names = ["loss"]def__call__(self, X):import numpy as npreturn np.sum(np.atleast_2d(X)**2, axis=1)opt2 = SpotOptim(fun=MetricFun(), bounds=[(-5, 5)])print(f"objective_names (annotated): {opt2.objective_names}")assert opt2.objective_names == ["loss"]print("objective_names check passed.")
A numpy.random.RandomState object is created with the supplied seed and stored as self.rng. It is used internally wherever random choices are needed (e.g., surrogate selection probabilities).
set_seed() then calls random.seed(self.seed) and np.random.seed(self.seed) to seed Python’s global and NumPy’s global generators, providing a reproducibility guarantee across the full call stack.
import numpy as npfrom spotoptim import SpotOptimfrom spotoptim.function import sphereopt = SpotOptim(fun=sphere, bounds=[(-5, 5)], seed=42)print(f"rng type: {type(opt.rng).__name__}")# Draw a sample — should be deterministic across invocationssample = opt.rng.uniform()print(f"First rng draw: {sample:.6f}")opt2 = SpotOptim(fun=sphere, bounds=[(-5, 5)], seed=42)sample2 = opt2.rng.uniform()assert sample == sample2, "RNG not reproducible with same seed"print("RNG reproducibility check passed.")
self._factor_maps is a dict mapping each dimension index to an {int: str} lookup table. It is empty for purely numeric problems and populated by process_factor_bounds() when any bound is specified as a tuple of strings, e.g., ("red", "green", "blue").
self._original_bounds preserves the user-supplied bounds before any integer-encoding transformation.
process_factor_bounds() converts string-level bounds to their integer equivalents so the optimizer works in a uniform numeric space:
No factors: _factor_maps is empty.
factor_maps: {0: {0: 'red', 1: 'green', 2: 'blue'}}
Factor bounds check passed.
Step 11 — Dimension Count (n_dim)
self.n_dim =len(self.bounds)
After process_factor_bounds() has resolved all string-level dimensions, n_dim is calculated as the number of remaining bounds. For problems with dimension reduction (fixed bounds, see Step 15), n_dim will later reflect the reduced dimension count.
detect_var_type() infers the variable type for each dimension from the bounds. For dimensions with purely numeric bounds it returns "float" — it does not auto-promote integer-looking bounds like (0, 5) to "int". Factor dimensions created in Step 10 (string-tuple bounds) are already typed as "factor" and are left untouched.
To use integer variables you must pass var_type=["int", ...] explicitly.
from spotoptim import SpotOptimfrom spotoptim.function import sphereimport numpy as np# Numeric bounds always yield floatopt_f = SpotOptim(fun=sphere, bounds=[(-5.0, 5.0), (-5.0, 5.0)])print(f"var_type (floats) : {opt_f.var_type}")assertall(t =="float"for t in opt_f.var_type)# Integer-looking bounds still yield float unless var_type is explicitopt_auto = SpotOptim(fun=sphere, bounds=[(0, 5), (0, 5)])print(f"var_type (no explicit) : {opt_auto.var_type}")assertall(t =="float"for t in opt_auto.var_type)# Explicitly requesting int variablesopt_int = SpotOptim(fun=sphere, bounds=[(0, 5), (0, 5)], var_type=["int", "int"])print(f"var_type (explicit int) : {opt_int.var_type}")assertall(t =="int"for t in opt_int.var_type)# Factor dim from string-tuple boundsdef cat_fun(X): X = np.atleast_2d(X)return X[:, 1].astype(float) **2opt_cat = SpotOptim(fun=cat_fun, bounds=[("a", "b", "c"), (-5.0, 5.0)])print(f"var_type (factor + float) : {opt_cat.var_type}")assert opt_cat.var_type[0] =="factor"assert opt_cat.var_type[1] =="float"print("detect_var_type check passed.")
modify_bounds_based_on_var_type() adjusts the bounds to be consistent with the declared variable types. For "int" and "factor" variables it ensures bounds are expressed as integers. For "float" variables it converts bounds to float. This makes downstream arithmetic type-safe and avoids mixed int/float arrays.
from spotoptim import SpotOptimfrom spotoptim.function import sphereopt = SpotOptim(fun=sphere, bounds=[(0, 5), (-3.0, 3.0)], var_type=["int", "float"])lower_types = [type(b[0]) for b in opt.bounds]print(f"Lower bound types: {lower_types}")# int dimension lower bound should be int, float dimension should be floatassertisinstance(opt.bounds[0][0], int)assertisinstance(opt.bounds[1][0], float)print("modify_bounds_based_on_var_type check passed.")
self.lower = np.array([b[0] for b inself.bounds])self.upper = np.array([b[1] for b inself.bounds])
The list-of-tuples self.bounds is unpacked into two numpy arrays:
self.lower — 1-D array of lower bounds, one entry per dimension
self.upper — 1-D array of upper bounds, one entry per dimension
These arrays are used throughout the class for vectorized bound arithmetic, for example, when scaling Latin Hypercube samples — lower + X_unit * (upper - lower).
ifself.var_name isNone:self.var_name = [f"x{i}"for i inrange(self.n_dim)]
If neither the constructor argument nor the function attribute provided names, each dimension receives an auto-generated name "x0", "x1", etc. Names appear in console output, error messages, and plot axis labels.
handle_default_var_trans() normalizes the var_trans list without applying any transformation. It replaces the strings "id" and "None" and Python None values with None so that all downstream code can use a single None check. If var_trans was not provided, it initializes a list of None values with length n_dim. It also validates that the list length matches n_dim.
Before any transformation is applied, _original_lower and _original_upper are created as copies of self.lower and self.upper. These snapshots preserve the natural-space bounds and are used:
In validate_x0() to confirm that a starting point is within the original domain.
In reporting and visualization to display the problem in human-readable units.
transform_bounds() then replaces self.lower, self.upper, and self.bounds with the values in transformed space. For example, with var_trans=["log10"] and bounds=[(1, 100)]:
_original_lower = [1], _original_upper = [100]
After transformation: lower = [0.0], upper = [2.0]
transform_bounds() also handles reversed bounds that arise from monotone decreasing transforms such as reciprocal.
setup_dimension_reduction() identifies dimensions where lower == upper (fixed variables that carry no information for the optimizer). It stores a boolean mask self.ident where True marks fixed dimensions.
Two sets of attributes coexist after this step:
all_* attributes (all_lower, all_upper, all_var_name, etc.) hold the full-dimensional representation including fixed dimensions.
The main attributes (self.lower, self.upper, self.bounds, self.n_dim) are reduced to the active (non-fixed) dimensions only.
The flag self.red_dim is True when at least one dimension was removed.
import numpy as npfrom spotoptim import SpotOptimdef fun3d(X): X = np.atleast_2d(X)return np.sum(X**2, axis=1)# dim 1 is fixed at 5.0opt = SpotOptim(fun=fun3d, bounds=[(-5, 5), (5, 5), (-5, 5)])print(f"red_dim : {opt.red_dim}")print(f"ident mask : {opt.ident}")print(f"n_dim (reduced) : {opt.n_dim}")print(f"all_lower (full) : {opt.all_lower}")assert opt.red_dim ==True# numpy.bool_ — use == not `is`assert opt.ident[1] ==True# numpy.bool_ — use == not `is`assert opt.n_dim ==2# only 2 free dimensionsassertlen(opt.all_lower) ==3# full-dim snapshot preservedprint("setup_dimension_reduction check passed.")
When the user supplies x0, it is validated and transformed to internal scale before any optimization begins. validate_x0() performs the following checks:
x0 must be 1-D or 2-D (a scalar or 3-D array raises ValueError).
The length of x0 must match the full dimension count (before reduction).
Each component must lie within its natural-space bounds.
Fixed dimensions must exactly equal their fixed value.
If all checks pass, validate_x0() applies transform_X() to convert x0 to the same internal (transformed and reduced) representation used during optimization.
import numpy as npfrom spotoptim import SpotOptimfrom spotoptim.function import sphere# x0 in natural scale, dim 1 is fixedopt = SpotOptim( fun=sphere, bounds=[(-5, 5), (5, 5), (-5, 5)], x0=np.array([1.0, 5.0, 3.0]),)print(f"x0 (internal scale): {opt.x0}")# After reduction, x0 should only contain the 2 free dimsassert opt.x0.shape == (2,)print("x0 = internal scale shape:", opt.x0.shape)# x0 outside bounds must raisetry: SpotOptim(fun=sphere, bounds=[(-5, 5)], x0=np.array([10.0]))raiseAssertionError("Should have raised ValueError")exceptValueErroras e:print(f"Caught: {e}")print("validate_x0 check passed.")
init_surrogate() sets up the surrogate model used to approximate the objective function between evaluations. Three scenarios are handled:
surrogate=None (default): A GaussianProcessRegressor with a ConstantKernel * Matern(nu=2.5) kernel, 100 optimizer restarts, and normalize_y=True is created automatically.
surrogate=[list]: A multi-surrogate setup is configured. Probability weights (self._prob_surrogate) and per-surrogate _max_surrogate_points_list are computed.
surrogate=<any object>: The provided object is used as-is.
After init_surrogate(), the following attributes are set:
self.surrogate — the active surrogate model
self._surrogates_list — list or None
self._prob_surrogate — selection probabilities or None
self._max_surrogate_points_list — per-surrogate max point limits
self._active_max_surrogate_points — current active limit
A scipy.stats.qmc.LatinHypercube sampler is created for generating space-filling initial designs. d=self.n_dim sets the dimensionality to the reduced dimension count. rng=self.seed seeds the sampler for reproducibility.
The primary parameter rng= is used instead of the legacy alias seed= to align with the current scipy API. The sampler is called in generate_initial_design() via self.lhs_sampler.random(n=self.n_initial), which returns points in [0, 1]^d that are then scaled to [lower, upper].
from scipy.stats.qmc import LatinHypercubefrom spotoptim import SpotOptimfrom spotoptim.function import sphereimport numpy as npopt = SpotOptim(fun=sphere, bounds=[(-5, 5), (-5, 5)], seed=42)print(f"lhs_sampler type: {type(opt.lhs_sampler).__name__}")assertisinstance(opt.lhs_sampler, LatinHypercube)# Reproducibility: two samplers with rng=42 produce identical drawss1 = LatinHypercube(d=2, rng=42).random(5)s2 = LatinHypercube(d=2, rng=42).random(5)assert np.allclose(s1, s2)print("lhs_sampler reproducibility check passed.")
window_size controls how many of the most recent evaluated points are included in the surrogate’s training window each iteration (used together with selection_method).
If the user did not set window_size explicitly:
When restart_after_n is set, window_size mirrors it so that the sliding window aligns with the restart cycle length.
Otherwise it falls back to 100, large enough to include all points in a typical short optimization run.
from spotoptim import SpotOptimfrom spotoptim.function import sphere# Without restart — defaults to 100opt1 = SpotOptim(fun=sphere, bounds=[(-5, 5)])print(f"window_size (no restart): {opt1.window_size}")assert opt1.window_size ==100# With restart_after_n — matches itopt2 = SpotOptim(fun=sphere, bounds=[(-5, 5)], restart_after_n=40)print(f"window_size (restart=40): {opt2.window_size}")assert opt2.window_size ==40# Explicit overrideopt3 = SpotOptim(fun=sphere, bounds=[(-5, 5)], window_size=25)print(f"window_size (explicit=25): {opt3.window_size}")assert opt3.window_size ==25print("window_size check passed.")
The final two calls set up optional TensorBoard logging.
_clean_tensorboard_logs() deletes existing log files at tensorboard_path if tensorboard_clean=True was requested.
_init_tensorboard_writer() creates a SummaryWriter if tensorboard_log=True, writing event files to tensorboard_path. When logging is disabled (the default), both methods are no-ops so there is no runtime cost.