7  Parallel Evaluation

This chapter explains how SpotOptim evaluates the objective function and how a user can still exploit several processor cores even though the optimizer itself runs sequentially.

7.1 Sequential evaluation

As of version 1.0.0, SpotOptim evaluates the objective function sequentially. The earlier steady-state asynchronous scheme, controlled by an n_jobs argument, was removed when the core was reduced to a lean sequential implementation. The constructor therefore no longer accepts n_jobs or eval_batch_size, and the public entry point is the single optimize() call shown throughout this book.

The motivation for the sequential design is reproducibility and a smaller dependency surface. With a fixed seed, a sequential run reproduces the same sequence of infill points on every machine, which matters for the benchmark and leaderboard results that depend on this library. Removing the parallel executor also dropped the heavy process-management dependencies from the core install.

Sequential evaluation does not prevent the use of multiple cores. Two complementary strategies remain available, and they cover the cases that the old n_jobs option addressed.

7.2 Parallelism inside the objective function

SpotOptim passes the candidate points to the objective function as a two-dimensional array whose rows are individual points. A vectorized objective that operates on the whole array uses the optimized linear-algebra backend, which is already multi-threaded, so a single call evaluates the batch efficiently without any explicit parallel code.

import warnings
import numpy as np
from spotoptim import SpotOptim

warnings.filterwarnings("ignore")


def sphere_batch(X):
    # X has shape (n_points, n_dim); evaluate all rows at once.
    return np.sum(X**2, axis=1)


optimizer = SpotOptim(
    fun=sphere_batch,
    bounds=[(-5, 5)] * 2,
    n_initial=5,
    max_iter=15,
    seed=42,
    verbose=False,
)
result = optimizer.optimize()

print(f"best x   : {np.round(result.x, 4)}")
print(f"best f(x): {result.fun:.6f}")
print(f"n eval   : {result.nfev}")
best x   : [0.0002 0.0004]
best f(x): 0.000000
n eval   : 15

When the per-point cost is dominated by a routine that releases the global interpreter lock, such as a NumPy or PyTorch operation, the objective can also spread the rows across a thread pool inside the function body. The optimizer sees a single call and remains deterministic, while the heavy work runs concurrently.

7.3 Independent optimizations across processes

The second strategy parallelizes at the level of whole optimization runs rather than single evaluations. Independent runs, for example repeated restarts from different seeds or the same tuning problem applied to several data splits, are embarrassingly parallel and can be dispatched to separate processes with the standard library. Each process executes one self-contained sequential optimize() call.

The following pattern illustrates the approach. It is shown for reference and is not executed during the rendering of this page, because process pools require a script entry point rather than a notebook cell.

from concurrent.futures import ProcessPoolExecutor
import numpy as np
from spotoptim import SpotOptim


def sphere_batch(X):
    return np.sum(X**2, axis=1)


def run_once(seed):
    optimizer = SpotOptim(
        fun=sphere_batch,
        bounds=[(-5, 5)] * 2,
        n_initial=5,
        max_iter=15,
        seed=seed,
        verbose=False,
    )
    result = optimizer.optimize()
    return seed, float(result.fun)


if __name__ == "__main__":
    seeds = [42, 43, 44, 45]
    with ProcessPoolExecutor() as pool:
        for seed, best in pool.map(run_once, seeds):
            print(f"seed {seed}: best f(x) = {best:.6f}")

This keeps each optimization reproducible, since the seed is fixed per run, while the wall-clock time scales with the number of available cores rather than with the number of runs.

7.4 Jupyter Notebook

Note