12  Save and Load in SpotOptim

SpotOptim provides comprehensive save and load functionality for serializing optimization configurations and results. This enables distributed workflows where experiments are defined locally, executed remotely, and analyzed back on the local machine.

12.1 Key Concepts

12.1.1 Experiments vs Results

SpotOptim distinguishes between two types of saved data:

  • Experiment (*_exp.pkl): Configuration only, excluding the objective function and results. Used to transfer optimization setup to remote machines.
  • Result (*_res.pkl): Complete optimization state including configuration, all evaluations, and results. Used to save and analyze completed optimizations.

Table 12.1 shows what gets saved in each file type.

Table 12.1: Robust Function Saving
Component Experiment Result
Configuration (bounds, parameters) ✓ ✓
Objective function ✓ ✓
Evaluations (X, y) ✗ ✓
Best solution ✗ ✓
Surrogate model Excluded* ✓
TensorBoard writer ✗ ✗

SpotOptim uses dill for serialization, which allows robust saving of objective functions, including lambda functions and locally defined functions.

12.2 Quick Start

import numpy as np
from spotoptim import SpotOptim

def sphere(X):
    """Simple sphere function"""
    return np.sum(X**2, axis=1)

# Create and configure optimizer
optimizer = SpotOptim(
    fun=sphere,
    bounds=[(-5, 5), (-5, 5)],
    max_iter=20,
    n_initial=10,
    seed=42
)

# Run optimization
result = optimizer.optimize()
print(f"Best value: {result.fun:.6f}")

# Save complete results
optimizer.save_result(prefix="sphere_opt")
# Creates: sphere_opt_res.pkl

# Later: load and analyze results
loaded_opt = SpotOptim.load_result("sphere_opt_res.pkl")
print(f"Loaded best value: {loaded_opt.best_y_:.6f}")
print(f"Total evaluations: {loaded_opt.counter}")
Best value: 0.000000
Experiment saved to sphere_opt_res.pkl
Result saved to sphere_opt_res.pkl
Loaded result from sphere_opt_res.pkl
Loaded best value: 0.000000
Total evaluations: 20

12.3 Distributed Workflow

The save/load functionality enables a powerful workflow for distributed optimization:

12.3.1 Step 1: Define Experiment Locally

Generate the remote_job_001_exp.pkl file by running the following code:

import numpy as np
from spotoptim import SpotOptim
from spotoptim.function import rosenbrock
 
optimizer = SpotOptim(
    fun=rosenbrock,
    bounds=[(-2, 2), (-2, 2)],
    max_iter=50,
    n_initial=10,
    seed=42,
    verbose=True)
 
optimizer.save_experiment(prefix="remote_job_001")
TensorBoard logging disabled
Experiment saved to remote_job_001_exp.pkl

12.3.2 Step 2: Execute on Remote Machine

Generate the remote_job_001_res.pkl file by running the following code on the remote machine:

from spotoptim import SpotOptim
optimizer = SpotOptim.load_experiment("remote_job_001_exp.pkl")
result = optimizer.optimize()
optimizer.save_result(prefix="remote_job_001")
Loaded experiment from remote_job_001_exp.pkl
Initial best: f(x) = 0.401500
Iter 1 | Best: 0.401500 | Curr: 3.144045 | Rate: 0.00 | Evals: 22.0%
Iter 2 | Best: 0.401500 | Curr: 3.090288 | Rate: 0.00 | Evals: 24.0%
Iter 3 | Best: 0.401500 | Curr: 3.134246 | Rate: 0.00 | Evals: 26.0%
Iter 4 | Best: 0.401500 | Curr: 4.923271 | Rate: 0.00 | Evals: 28.0%
Iter 5 | Best: 0.401500 | Curr: 8.969317 | Rate: 0.00 | Evals: 30.0%
Iter 6 | Best: 0.401500 | Curr: 2.900465 | Rate: 0.00 | Evals: 32.0%
Iter 7 | Best: 0.401500 | Curr: 2.575117 | Rate: 0.00 | Evals: 34.0%
Iter 8 | Best: 0.053916 | Rate: 0.12 | Evals: 36.0%
Iter 9 | Best: 0.053916 | Curr: 4.105500 | Rate: 0.11 | Evals: 38.0%
Iter 10 | Best: 0.053916 | Curr: 2.608834 | Rate: 0.10 | Evals: 40.0%
Iter 11 | Best: 0.027893 | Rate: 0.18 | Evals: 42.0%
Iter 12 | Best: 0.027893 | Curr: 0.036137 | Rate: 0.17 | Evals: 44.0%
Iter 13 | Best: 0.007228 | Rate: 0.23 | Evals: 46.0%
Iter 14 | Best: 0.007228 | Curr: 0.007297 | Rate: 0.21 | Evals: 48.0%
Iter 15 | Best: 0.007228 | Curr: 2.301500 | Rate: 0.20 | Evals: 50.0%
Iter 16 | Best: 0.005260 | Rate: 0.25 | Evals: 52.0%
Iter 17 | Best: 0.005260 | Curr: 0.005260 | Rate: 0.24 | Evals: 54.0%
Iter 18 | Best: 0.005260 | Curr: 2.149526 | Rate: 0.22 | Evals: 56.0%
Iter 19 | Best: 0.005260 | Curr: 2.137122 | Rate: 0.21 | Evals: 58.0%
Iter 20 | Best: 0.005260 | Curr: 2.082718 | Rate: 0.20 | Evals: 60.0%
Iter 21 | Best: 0.005260 | Curr: 1.987013 | Rate: 0.19 | Evals: 62.0%
Optimizer candidate 1/3 was duplicate/invalid.
Iter 22 | Best: 0.005260 | Rate: 0.23 | Evals: 64.0%
Iter 23 | Best: 0.005260 | Curr: 1.836752 | Rate: 0.22 | Evals: 66.0%
Iter 24 | Best: 0.005260 | Curr: 0.005263 | Rate: 0.21 | Evals: 68.0%
Iter 25 | Best: 0.005260 | Curr: 1.729247 | Rate: 0.20 | Evals: 70.0%
Iter 26 | Best: 0.005260 | Curr: 1.649939 | Rate: 0.19 | Evals: 72.0%
Iter 27 | Best: 0.005239 | Rate: 0.22 | Evals: 74.0%
Iter 28 | Best: 0.005239 | Curr: 0.005249 | Rate: 0.21 | Evals: 76.0%
Iter 29 | Best: 0.005239 | Curr: 1.567640 | Rate: 0.21 | Evals: 78.0%
Iter 30 | Best: 0.005239 | Curr: 1.521007 | Rate: 0.20 | Evals: 80.0%
Iter 31 | Best: 0.005222 | Rate: 0.23 | Evals: 82.0%
Iter 32 | Best: 0.005173 | Rate: 0.25 | Evals: 84.0%
Iter 33 | Best: 0.005173 | Curr: 1.508398 | Rate: 0.24 | Evals: 86.0%
Iter 34 | Best: 0.005173 | Curr: 1.501679 | Rate: 0.24 | Evals: 88.0%
Iter 35 | Best: 0.005173 | Curr: 1.453558 | Rate: 0.23 | Evals: 90.0%
Iter 36 | Best: 0.005148 | Rate: 0.25 | Evals: 92.0%
Iter 37 | Best: 0.005139 | Rate: 0.27 | Evals: 94.0%
Iter 38 | Best: 0.005072 | Rate: 0.29 | Evals: 96.0%
Iter 39 | Best: 0.005072 | Curr: 1.178987 | Rate: 0.28 | Evals: 98.0%
Iter 40 | Best: 0.005072 | Curr: 1.171901 | Rate: 0.28 | Evals: 100.0%
Experiment saved to remote_job_001_res.pkl
Result saved to remote_job_001_res.pkl

12.3.3 Step 3: Analyze Results Locally

from spotoptim import SpotOptim
optimizer = SpotOptim.load_result("remote_job_001_res.pkl")
print(f"Best value found: {optimizer.best_y_:.6f}")
print(f"Best point: {optimizer.best_x_}")
print(f"Total evaluations: {optimizer.counter}")
print(f"Number of iterations: {optimizer.n_iter_}")
Loaded result from remote_job_001_res.pkl
Best value found: 0.005072
Best point: [1.07114126 1.14766756]
Total evaluations: 50
Number of iterations: 40
optimizer.plot_progress(log_y=True)
Figure 12.1: Optimization Progress

Access all evaluated points as follows:

print(f"\nAll evaluated points shape: {optimizer.X_.shape}")
print(f"All objective values shape: {optimizer.y_.shape}")

All evaluated points shape: (50, 2)
All objective values shape: (50,)

12.4 Advanced Usage

12.4.1 Custom Filenames and Paths

import os
from spotoptim import SpotOptim
import numpy as np

def objective(X):
    return np.sum(X**2, axis=1)

optimizer = SpotOptim(
    fun=objective,
    bounds=[(-5, 5), (-5, 5)],
    max_iter=30,
    seed=42
)

# Save with custom filename
optimizer.save_experiment(
    filename="custom_name.pkl",
    verbosity=1
)

# Save to specific directory
os.makedirs("experiments/batch_001", exist_ok=True)
optimizer.save_experiment(
    prefix="exp_001",
    path="experiments/batch_001",
    verbosity=1
)
# Creates: experiments/batch_001/exp_001_exp.pkl
Experiment saved to custom_name.pkl
Experiment saved to experiments/batch_001/exp_001_exp.pkl

12.4.2 Overwrite Protection

from spotoptim import SpotOptim
import numpy as np

def sphere(X):
    return np.sum(X**2, axis=1)

optimizer = SpotOptim(fun=sphere, bounds=[(-5, 5), (-5, 5)], max_iter=20)
result = optimizer.optimize()

# First save
optimizer.save_result(prefix="my_result")

# Try to save again - raises FileExistsError by default
try:
    optimizer.save_result(prefix="my_result")
except FileExistsError as e:
    print(f"File already exists: {e}")

# Explicitly allow overwriting
optimizer.save_result(prefix="my_result", overwrite=True)
print("File overwritten successfully")
Experiment saved to my_result_res.pkl
Result saved to my_result_res.pkl
Experiment saved to my_result_res.pkl
Result saved to my_result_res.pkl
Experiment saved to my_result_res.pkl
Result saved to my_result_res.pkl
File overwritten successfully

12.4.3 Loading and Continuing Optimization

from spotoptim import SpotOptim
import numpy as np

def objective(X):
    return np.sum(X**2, axis=1)

# Initial optimization
opt1 = SpotOptim(
    fun=objective,
    bounds=[(-5, 5), (-5, 5)],
    max_iter=20,
    seed=42
)
result1 = opt1.optimize()
opt1.save_result(prefix="checkpoint")

print(f"Initial optimization: {result1.nfev} evaluations, best={result1.fun:.6f}")

# Load and continue
opt2 = SpotOptim.load_result("checkpoint_res.pkl")
# No need to re-attach function as it is loaded with dill
opt2.max_iter = 25  # Increase budget

# Continue optimization
result2 = opt2.optimize()
print(f"After continuation: {result2.nfev} evaluations, best={result2.fun:.6f}")
Experiment saved to checkpoint_res.pkl
Result saved to checkpoint_res.pkl
Initial optimization: 20 evaluations, best=0.000000
Loaded result from checkpoint_res.pkl
After continuation: 25 evaluations, best=0.000000

12.5 Working with Noisy Functions

Save and load preserves noise statistics for reproducible analysis:

import numpy as np
from spotoptim import SpotOptim

def noisy_objective(X):
    """Objective with measurement noise"""
    true_value = np.sum(X**2, axis=1)
    noise = np.random.normal(0, 0.1, X.shape[0])
    return true_value + noise

# Optimize noisy function with repeated evaluations
optimizer = SpotOptim(
    fun=noisy_objective,
    bounds=[(-5, 5), (-5, 5)],
    max_iter=20,
    n_initial=10,
    repeats_initial=3,    # Repeat initial points
    repeats_surrogate=2,  # Repeat surrogate points
    seed=42,
    verbose=True
)

result = optimizer.optimize()

# Save results (includes noise statistics)
optimizer.save_result(prefix="noisy_opt")

# Load and analyze noise statistics
loaded_opt = SpotOptim.load_result("noisy_opt_res.pkl")

print(f"Noise handling enabled: {loaded_opt.noise}")
print(f"Best mean value: {loaded_opt.best_y_:.6f}")

if loaded_opt.mean_y is not None:
    print(f"Mean values available: {len(loaded_opt.mean_y)}")
    print(f"Variance values available: {len(loaded_opt.var_y)}")
TensorBoard logging disabled
Initial best: f(x) = 2.305707, mean best: f(x) = 2.367991
Experiment saved to noisy_opt_res.pkl
Result saved to noisy_opt_res.pkl
Loaded result from noisy_opt_res.pkl
Noise handling enabled: True
Best mean value: 2.305707
Mean values available: 10
Variance values available: 10

12.6 Working with Different Variable Types

Save and load preserves variable type information:

import numpy as np
from spotoptim import SpotOptim

def mixed_objective(X):
    """Objective with mixed variable types"""
    return np.sum(X**2, axis=1)

# Create optimizer with mixed variable types
optimizer = SpotOptim(
    fun=mixed_objective,
    bounds=[(-5, 5), (-5, 5), (-5, 5), (-5, 5)],
    var_type=["float", "int", "factor", "float"],
    var_name=["continuous", "integer", "categorical", "another_cont"],
    max_iter=20,
    n_initial=10,
    seed=42
)

result = optimizer.optimize()

# Save results
optimizer.save_result(prefix="mixed_vars")

# Load results
loaded_opt = SpotOptim.load_result("mixed_vars_res.pkl")

print("Variable types preserved:")
print(f"  var_type: {loaded_opt.var_type}")
print(f"  var_name: {loaded_opt.var_name}")

# Verify integer variables are still integers
print(f"\nInteger variable (dim 1) values:")
print(loaded_opt.X_[:5, 1])  # Should be integers
Experiment saved to mixed_vars_res.pkl
Result saved to mixed_vars_res.pkl
Loaded result from mixed_vars_res.pkl
Variable types preserved:
  var_type: ['float', 'int', 'factor', 'float']
  var_name: ['continuous', 'integer', 'categorical', 'another_cont']

Integer variable (dim 1) values:
[ 3. -2. -0. -3.  4.]

12.7 Best Practices

12.7.1 1. Re-attaching the Objective Function (Optional)

Since dill is used for serialization, the objective function is automatically loaded. You only need to re-attach the function if:

  1. You want to change the objective function (e.g., to a different implementation).
  2. The function relies on external resources that cannot be serialized.
# Load experiment
optimizer = SpotOptim.load_experiment("experiment_exp.pkl")

# OPTIONAL: Replace the function if needed
# optimizer.fun = new_objective_function

# Run optimization
result = optimizer.optimize()

12.7.2 2. Use Meaningful Prefixes

Organize your experiments with descriptive prefixes:

# Good practice: descriptive prefixes
optimizer.save_experiment(prefix="sphere_d10_seed42")
optimizer.save_experiment(prefix="rosenbrock_n100_lhs")
optimizer.save_result(prefix="final_run_2024_11_15")

# Avoid: generic names
optimizer.save_experiment(prefix="exp1")  # Not descriptive
optimizer.save_result(prefix="result")     # Hard to track

12.7.3 3. Save Experiments Before Remote Execution

# Define locally
optimizer = SpotOptim(bounds=bounds, max_iter=20, seed=42)
optimizer.save_experiment(prefix="remote_job")

# Transfer file to remote machine
# Execute remotely
# Transfer results back
# Analyze locally

12.7.4 4. Version Your Experiments

import datetime

# Add timestamp to prefix
timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
prefix = f"experiment_{timestamp}"

optimizer.save_experiment(prefix=prefix)
# Creates: experiment_20241115_143022_exp.pkl

12.7.5 5. Handle File Paths Robustly

import os

# Create directory structure
exp_dir = "experiments/batch_001"
os.makedirs(exp_dir, exist_ok=True)

# Save with full path
optimizer.save_experiment(
    prefix="exp_001",
    path=exp_dir
)

# Load with full path
exp_file = os.path.join(exp_dir, "exp_001_exp.pkl")
loaded_opt = SpotOptim.load_experiment(exp_file)

12.8 Complete Example: Multi-Machine Workflow

Here’s a complete example demonstrating the entire workflow:

12.8.1 Local Machine (Setup)

# setup_experiment.py
import numpy as np
from spotoptim import SpotOptim
import os

DIRNAME = "experiments_compare_de_tricands_bfgs"

# Placeholder function for experiment setup
def placeholder_func(X):
    return np.sum(X**2, axis=1)



# Create experiments directory
os.makedirs(DIRNAME, exist_ok=True)
# guarantee that the directory is empty
for file in os.listdir(DIRNAME):
    os.remove(os.path.join(DIRNAME, file))

# Define multiple experiments
experiments = [
    {"acquisition_optimizer": "differential_evolution", "max_iter": 20, "prefix": "exp_de"},
    {"acquisition_optimizer": "tricands", "max_iter": 20, "prefix": "exp_tricands"},
    {"acquisition_optimizer": "L-BFGS-B", "max_iter": 20, "prefix": "exp_bfgs"},
]

for exp_config in experiments:
    optimizer = SpotOptim(
        fun=placeholder_func,  # Placeholder - will be replaced remotely
        bounds=[(-10, 10), (-10, 10), (-10, 10)],
        max_iter=exp_config["max_iter"],
        n_initial=10,
        seed=42,
        acquisition_optimizer=exp_config["acquisition_optimizer"],
        verbose=True
    )
    
    optimizer.save_experiment(
        prefix=exp_config["prefix"],
        path=DIRNAME
    )
    
    print(f"Created: {DIRNAME}/{exp_config['prefix']}_exp.pkl")

print("\nAll experiments created. Transfer '{DIRNAME}' folder to remote machine.")
TensorBoard logging disabled
Experiment saved to experiments_compare_de_tricands_bfgs/exp_de_exp.pkl
Created: experiments_compare_de_tricands_bfgs/exp_de_exp.pkl
TensorBoard logging disabled
Experiment saved to experiments_compare_de_tricands_bfgs/exp_tricands_exp.pkl
Created: experiments_compare_de_tricands_bfgs/exp_tricands_exp.pkl
TensorBoard logging disabled
Experiment saved to experiments_compare_de_tricands_bfgs/exp_bfgs_exp.pkl
Created: experiments_compare_de_tricands_bfgs/exp_bfgs_exp.pkl

All experiments created. Transfer '{DIRNAME}' folder to remote machine.

12.8.2 Remote Machine (Execution)

# run_experiments.py
import numpy as np
from spotoptim import SpotOptim
import os
import glob

def complex_objective(X):
    """Complex multimodal objective function"""
    term1 = np.sum(X**2, axis=1)
    term2 = 10 * np.sum(np.cos(2 * np.pi * X), axis=1)
    term3 = 0.1 * np.sum(np.sin(5 * np.pi * X), axis=1)
    return term1 - term2 + term3

# Find all experiment files
exp_files = glob.glob(f"{DIRNAME}/*_exp.pkl")
print(f"Found {len(exp_files)} experiments to run")

# Run each experiment
for exp_file in exp_files:
    print(f"\nProcessing: {exp_file}")
    
    # Load experiment
    optimizer = SpotOptim.load_experiment(exp_file)
    
    # Attach objective
    optimizer.fun = complex_objective
    
    # Run optimization
    result = optimizer.optimize()
    print(f"  Best value: {result.fun:.6f}")
    
    # Save result (same prefix, different suffix)
    prefix = os.path.basename(exp_file).replace("_exp.pkl", "")
    optimizer.save_result(
        prefix=prefix,
        path=DIRNAME
    )
    print(f"  Saved: {DIRNAME}/{prefix}_res.pkl")

print("\nAll experiments completed. Transfer results back to local machine.")
Found 3 experiments to run

Processing: experiments_compare_de_tricands_bfgs/exp_de_exp.pkl
Loaded experiment from experiments_compare_de_tricands_bfgs/exp_de_exp.pkl
Initial best: f(x) = 41.376730
Iter 1 | Best: 38.767750 | Rate: 1.00 | Evals: 55.0%
Iter 2 | Best: 38.767750 | Curr: 44.456273 | Rate: 0.50 | Evals: 60.0%
Iter 3 | Best: 15.731275 | Rate: 0.67 | Evals: 65.0%
Iter 4 | Best: 14.986076 | Rate: 0.75 | Evals: 70.0%
Iter 5 | Best: 14.986076 | Curr: 17.176588 | Rate: 0.60 | Evals: 75.0%
Iter 6 | Best: 14.986076 | Curr: 15.675938 | Rate: 0.50 | Evals: 80.0%
Iter 7 | Best: 13.315934 | Rate: 0.57 | Evals: 85.0%
Iter 8 | Best: 13.274080 | Rate: 0.62 | Evals: 90.0%
Iter 9 | Best: 11.810027 | Rate: 0.67 | Evals: 95.0%
Iter 10 | Best: 11.810027 | Curr: 12.317492 | Rate: 0.60 | Evals: 100.0%
  Best value: 11.810027
Experiment saved to experiments_compare_de_tricands_bfgs/exp_de_res.pkl
Result saved to experiments_compare_de_tricands_bfgs/exp_de_res.pkl
  Saved: experiments_compare_de_tricands_bfgs/exp_de_res.pkl

Processing: experiments_compare_de_tricands_bfgs/exp_bfgs_exp.pkl
Loaded experiment from experiments_compare_de_tricands_bfgs/exp_bfgs_exp.pkl
Initial best: f(x) = 41.376730
Iter 1 | Best: 41.376730 | Curr: 68.943918 | Rate: 0.00 | Evals: 55.0%
Iter 2 | Best: 41.376730 | Curr: 72.037984 | Rate: 0.00 | Evals: 60.0%
Iter 3 | Best: 41.376730 | Curr: 79.347103 | Rate: 0.00 | Evals: 65.0%
Iter 4 | Best: 41.376730 | Curr: 79.115429 | Rate: 0.00 | Evals: 70.0%
Iter 5 | Best: 41.376730 | Curr: 78.586507 | Rate: 0.00 | Evals: 75.0%
Iter 6 | Best: 41.376730 | Curr: 76.761528 | Rate: 0.00 | Evals: 80.0%
Iter 7 | Best: 41.376730 | Curr: 74.476499 | Rate: 0.00 | Evals: 85.0%
Iter 8 | Best: 41.376730 | Curr: 52.174959 | Rate: 0.00 | Evals: 90.0%
Iter 9 | Best: 41.376730 | Curr: 47.225532 | Rate: 0.00 | Evals: 95.0%
Iter 10 | Best: 41.376730 | Curr: 73.156671 | Rate: 0.00 | Evals: 100.0%
  Best value: 41.376730
Experiment saved to experiments_compare_de_tricands_bfgs/exp_bfgs_res.pkl
Result saved to experiments_compare_de_tricands_bfgs/exp_bfgs_res.pkl
  Saved: experiments_compare_de_tricands_bfgs/exp_bfgs_res.pkl

Processing: experiments_compare_de_tricands_bfgs/exp_tricands_exp.pkl
Loaded experiment from experiments_compare_de_tricands_bfgs/exp_tricands_exp.pkl
Initial best: f(x) = 41.376730
Iter 1 | Best: 14.514498 | Rate: 1.00 | Evals: 55.0%
Iter 2 | Best: 14.514498 | Curr: 26.247406 | Rate: 0.50 | Evals: 60.0%
Iter 3 | Best: 14.514498 | Curr: 24.520997 | Rate: 0.33 | Evals: 65.0%
Iter 4 | Best: 14.514498 | Curr: 31.092661 | Rate: 0.25 | Evals: 70.0%
Iter 5 | Best: -4.704984 | Rate: 0.40 | Evals: 75.0%
Iter 6 | Best: -4.704984 | Curr: 34.061204 | Rate: 0.33 | Evals: 80.0%
Iter 7 | Best: -4.704984 | Curr: 28.330614 | Rate: 0.29 | Evals: 85.0%
Iter 8 | Best: -4.704984 | Curr: 27.866062 | Rate: 0.25 | Evals: 90.0%
Iter 9 | Best: -4.704984 | Curr: 11.701660 | Rate: 0.22 | Evals: 95.0%
Iter 10 | Best: -4.704984 | Curr: 8.697120 | Rate: 0.20 | Evals: 100.0%
  Best value: -4.704984
Experiment saved to experiments_compare_de_tricands_bfgs/exp_tricands_res.pkl
Result saved to experiments_compare_de_tricands_bfgs/exp_tricands_res.pkl
  Saved: experiments_compare_de_tricands_bfgs/exp_tricands_res.pkl

All experiments completed. Transfer results back to local machine.

12.8.3 Local Machine (Analysis)

# analyze_results.py
import numpy as np
from spotoptim import SpotOptim
import glob
import matplotlib.pyplot as plt

# Find all result files
result_files = glob.glob(f"{DIRNAME}/*_res.pkl")
print(f"Found {len(result_files)} results to analyze")

# Load and compare results
results = []
for res_file in result_files:
    opt = SpotOptim.load_result(res_file)
    results.append({
        "file": res_file,
        "best_value": opt.best_y_,
        "best_point": opt.best_x_,
        "n_evals": opt.counter,
        "seed": opt.seed,
        "acquisition_optimizer": opt.acquisition_optimizer,
    })
    print(f"{res_file}: best={opt.best_y_:.6f}, evals={opt.counter}")

# Find best overall result
best = min(results, key=lambda x: x["best_value"])
print(f"\nBest result:")
print(f"  File: {best['file']}")
print(f"  Value: {best['best_value']:.6f}")
print(f"  Point: {best['best_point']}")
print(f"  Seed: {best['seed']}")
print(f"  Acquisition optimizer: {best['acquisition_optimizer']}")

# Plot convergence comparison
plt.figure(figsize=(12, 6))

for res_file in result_files:
    opt = SpotOptim.load_result(res_file)
    seed = opt.seed
    acquisition_optimizer = opt.acquisition_optimizer
    cummin = [opt.y_[:i+1].min() for i in range(len(opt.y_))]
    plt.plot(cummin, label=f"{acquisition_optimizer}", linewidth=2, alpha=0.7)

plt.xlabel("Iteration", fontsize=12)
plt.ylabel("Best Value Found", fontsize=12)
plt.title("Optimization Progress Comparison", fontsize=14)
plt.legend()
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig(f"{DIRNAME}/convergence_comparison.png", dpi=150)
print("\nConvergence plot saved to: {DIRNAME}/convergence_comparison.png")
Found 3 results to analyze
Loaded result from experiments_compare_de_tricands_bfgs/exp_bfgs_res.pkl
experiments_compare_de_tricands_bfgs/exp_bfgs_res.pkl: best=41.376730, evals=20
Loaded result from experiments_compare_de_tricands_bfgs/exp_de_res.pkl
experiments_compare_de_tricands_bfgs/exp_de_res.pkl: best=11.810027, evals=20
Loaded result from experiments_compare_de_tricands_bfgs/exp_tricands_res.pkl
experiments_compare_de_tricands_bfgs/exp_tricands_res.pkl: best=-4.704984, evals=20

Best result:
  File: experiments_compare_de_tricands_bfgs/exp_tricands_res.pkl
  Value: -4.704984
  Point: [ 0.98627463 -0.20745672  3.90603586]
  Seed: 42
  Acquisition optimizer: tricands
Loaded result from experiments_compare_de_tricands_bfgs/exp_bfgs_res.pkl
Loaded result from experiments_compare_de_tricands_bfgs/exp_de_res.pkl
Loaded result from experiments_compare_de_tricands_bfgs/exp_tricands_res.pkl

Convergence plot saved to: {DIRNAME}/convergence_comparison.png

12.9 Technical Details

12.9.1 Serialization Method

SpotOptim uses Python’s built-in pickle module for serialization. This provides:

  • Standard library: No additional dependencies required
  • Compatibility: Works with numpy arrays, sklearn models, scipy functions
  • Performance: Efficient serialization of large datasets

12.9.2 Component Reinitialization

When loading experiments, certain components are automatically recreated:

  • Surrogate model: Gaussian Process with default kernel
  • LHS sampler: Latin Hypercube Sampler with original seed

This ensures loaded experiments can continue optimization without manual configuration.

12.9.3 Excluded Components

Some components cannot be pickled and are automatically excluded:

  • Objective function (fun): Included (via dill) in both experiment and result files.
  • TensorBoard writer (tb_writer): File handles cannot be serialized
  • Surrogate model (experiments only): Recreated on load for experiments

12.9.4 File Format

Files are saved using pickle’s highest protocol:

with open(filename, "wb") as handle:
    pickle.dump(optimizer_state, handle, protocol=pickle.HIGHEST_PROTOCOL)

12.10 Troubleshooting

12.10.1 Issue: “AttributeError: ‘SpotOptim’ object has no attribute ‘fun’”

This should not happen if dill is working correctly. If it does, your function might not be picklable. Try defining the function at the top level of your script or module. If all else fails, you can manually re-attach it:

#| eval: false
opt = SpotOptim.load_experiment("exp.pkl")
opt.fun = my_objective_function # Manually re-attach
result = opt.optimize()

12.10.2 Issue: “FileNotFoundError: Experiment file not found”

Cause: Incorrect file path or file doesn’t exist.

Solution: Check file path and ensure file exists:

import os

filename = "experiment_exp.pkl"
if os.path.exists(filename):
    opt = SpotOptim.load_experiment(filename)
else:
    print(f"File not found: {filename}")

12.10.3 Issue: “FileExistsError: File already exists”

Cause: Attempting to save over an existing file without overwrite=True.

Solution: Either use a different prefix or enable overwriting:

# Option 1: Use different prefix
optimizer.save_result(prefix="my_result_v2")

# Option 2: Enable overwriting
optimizer.save_result(prefix="my_result", overwrite=True)

12.10.4 Issue: Results differ after loading

Cause: Random state not preserved or function behavior changed.

Solution: Ensure you’re using the same seed and function definition:

# When saving
optimizer = SpotOptim(..., seed=42)  # Use fixed seed

# When loading and continuing
loaded_opt = SpotOptim.load_result("result_res.pkl")
# loaded_opt.fun is already attached

12.11 Jupyter Notebook

Note