30  SpotOptim Execution Flow

31 SpotOptim Execution Flow

31.1 Example Without Restarts

This document outlines the sequence of methods called when SpotOptim.optimize() is executed, explaining the input, processing, and output at each step. As an example, we will use the following code:

from spotoptim import SpotOptim
from spotoptim.functions import sphere

opt = SpotOptim(
    fun=sphere,
    bounds=[(-5, 5), (-5, 5)],
    n_initial=5,
    max_iter=20,
    verbose=True
)
result = opt.optimize()

31.1.1 SpotOptim.__init__

  • Input: Configuration parameters (fun, bounds, n_initial, max_iter, surrogate, etc.).
  • Processing:
    • Validates arguments (e.g., checks max_iter >= n_initial).
    • Initializes SpotOptimConfig (configuration) and SpotOptimState (mutable state).
    • Sets random seeds via _set_seed.
    • Determines variable types and transformations.
    • Initializes the default Gaussian Process surrogate if none is provided.
    • Sets up the Latin Hypercube Sampler (lhs_sampler).
  • Output: An initialized SpotOptim instance ready for optimization.

31.1.2 SpotOptim.optimize

  • Input: Optional initial point X0 (Natural Space).
  • Processing:
    • Initializes the restart results list.
    • Enters the restart loop.
    • Calls _optimize_single_run to perform the optimization.
    • Manages restarts based on the return status of the single run.
  • Output: OptimizeResult object containing the best parameters x, best value fun, and optimization history.

31.1.3 SpotOptim._optimize_single_run

  • Input: timeout_start, X0 (initial points), y0_known (for injected restarts).
  • Processing:
    • Orchestrates the lifecycle of a single optimization run (initial design -> loop -> termination).
    • Calls sub-methods for generating design, evaluating points, fitting models, and selecting candidates.
  • Output: Tuple (status, OptimizeResult), where status is “FINISHED” or “RESTART”.

31.1.4 SpotOptim.get_initial_design

  • Input: X0 (from optimize call).
  • Processing:
    • Generates a space-filling design using lhs_sampler.random().
    • Scales the design from \([0, 1]\) to the problem bounds.
    • Combines the generated design with any user-provided X0.
  • Output: Array X of initial design points in Natural Space.

31.1.5 SpotOptim._curate_initial_design

  • Input: Raw initial design points X.
  • Processing:
    • Removes duplicate points to avoid redundant evaluations.
    • Verification against existing points (relevant during restarts).
  • Output: Unique array of initial design points X to be evaluated.

31.1.6 SpotOptim._evaluate_function (Initial Design)

  • Input: Batch of design points X.
  • Processing:
    • Iterates through points and calls the user-provided objective function fun(x).
    • Handles NaN/Inf results via penalty mechanism (if enabled).
    • Converts multi-objective results to single-objective if needed (fun_mo2so).
  • Output: Array of objective values y.

31.1.7 SpotOptim._init_storage

  • Input: Initial X and y.
  • Processing:
    • Populates self.state.X_ and self.state.y_ with the initial design data.
    • Initializes statistics tracking (mean/variance) for noisy problems.
  • Output: Updated internal state with initial data.

31.1.8 Optimization Loop Start

The process enters a while loop that continues until max_iter evaluations are reached or time runs out.

31.1.9 SpotOptim._fit_scheduler

  • Input: Current history X_, y_.
  • Processing:
    • Determines if the surrogate model needs retraining (based on iteration intervals).
    • Calls self.surrogate.fit(X, y) using the current available data.
  • Output: A trained surrogate model capable of predicting y for unseen X.

31.1.10 SpotOptim.suggest_next_infill_point

  • Input: None (implicilty uses trained surrogate).
  • Processing:
    • Dispatcher that determines how to find the next point.
    • Calls _try_optimizer_candidates.
  • Output: New candidate point x_next (in Internal/Transformed Space).

31.1.11 SpotOptim._try_optimizer_candidates

  • Input: None.
  • Processing:
    • Calls optimize_acquisition_func to maximize the acquisition function (e.g., Expected Improvement).
    • Rounds/repairs the candidate based on variable types (e.g., integers).
    • Checks that x_next is not too close to existing points (tolerance_x).
  • Output: Valid, unique candidate point(s).

31.1.12 SpotOptim.optimize_acquisition_func

  • Input: Surrogate model, Acquisition function type (configured in __init__).
  • Processing:
    • Uses a global optimizer (e.g., differential_evolution) to find inputs x that maximize the acquisition function derived from the surrogate’s predictions.
  • Output: The point(s) x that maximize potential improvement.

31.1.13 SpotOptim._evaluate_function (Loop)

  • Input: Candidate point x_next.
  • Processing:
    • Evaluates the user’s objective function at the new candidate point.
  • Output: New objective value y_next.

31.1.14 SpotOptim._update_storage

  • Input: New x_next, y_next.
  • Processing:
    • Appends the new evaluation to the history X_ and y_.
  • Output: Updated dataset used for the next surrogate fit.

31.1.15 SpotOptim._update_best_main_loop

  • Input: New y_next.
  • Processing:
    • Compares new value against self.state.best_y_.
    • Updates best_x_ and best_y_ if a better solution is found.
  • Output: Updated best known solution.

31.1.16 Termination

  • Condition: Validates if len(y_) >= max_iter.
  • Processing:
    • Exits the loop.
    • Prepares the final OptimizeResult.
  • Output: Final optimization result returned to the user.

31.2 Example With Restarts

Now we can look at an example with restarts. Since the argument restart_after_n is set to 10, the optimizer will restart if the success rate remains zero for 10 consecutive iterations.

opt = SpotOptim(
    fun=sphere,
    bounds=[(-5, 5), (-5, 5)],
    n_initial=5,
    max_iter=100,
    verbose=True,
    restart_after_n=10
)

result = opt.optimize()

31.2.1 SpotOptim.optimize (Global Loop)

  • Input: Global configuration (max_iter, restart_after_n).
  • Processing:
    • Initializes restarts_results_ to store results from each run.
    • Enters a while loop that manages the global budget.
    • Checks if enough budget remains for a new run (must allow at least n_initial evaluations).
  • Output: Orchestrates the sequence of optimization runs.

31.2.2 SpotOptim._optimize_single_run (First Run)

  • Input: Initial random state.
  • Processing:
    • Performs standard optimization (Initial Design -> Loop).
    • Restart Check: During the loop, monitors the success_rate.
    • If success_rate stays at 0.0 for restart_after_n (10) consecutive iterations, the run is terminated early with status="RESTART".
  • Output: Partial OptimizeResult and status RESTART.

31.2.3 Restart Injection (Between Runs)

  • Input: Results from previous runs (restarts_results_).
  • Processing:
    • Identifies the best solution (x_best, y_best) found so far.
    • Injection: If restart_inject_best=True (default), sets self.x0 = x_best. This ensures the next run starts with the best known solution.
    • Prepares y0_known = y_best to skip re-evaluating this expensive point.

31.2.4 SpotOptim._optimize_single_run (Subsequent Runs)

  • Input: y0_known (Value of injected best point).
  • Processing:
    • Fresh Initial Design:
      • Generates a new Latin Hypercube Design (continuing the sequence, providing fresh random points).
      • Injects the previous best point (self.x0) into this design, replacing one of the random points.
    • Evaluation:
      • Evaluates the new LHS points.
      • Skips evaluation for the injected point, using y0_known.
    • Optimization Loop:
      • Fits a new Surrogate Model on this fresh dataset.
      • Optimizes to refine the best solution or explore new regions.
  • Output: New OptimizeResult for this specific run.

31.2.5 Termination

  • Condition: The process repeats until the total number of evaluations (summing all runs) reaches max_iter (100).
  • Output: The OptimizeResult with the best objective value found across all runs is returned.

31.3 Description Based on Chunks

The source code is split into chunks of functionality.

31.3.1 Configuration & Helpers

Methods for initializing the optimizer, handling configuration, and processing variable types.

31.3.1.1 _set_seed

Sets global random seeds for reproducibility across random, numpy, and torch.

spot = SpotOptim(fun=lambda x: x, bounds=[(0, 1)], seed=42)
spot._set_seed()

31.3.1.2 detect_var_type

Auto-detects variable types (‘factor’ or ‘float’) based on the provided bounds.

spot = SpotOptim(fun=lambda x: x, bounds=[('a', 'b'), (0, 1)])
types = spot.detect_var_type() # ['factor', 'float']

31.3.1.3 modify_bounds_based_on_var_type

Adjusts bounds representation based on variable types (e.g., converting to integer bounds).

spot = SpotOptim(fun=lambda x: x, bounds=[(0.5, 10.5)], var_type=['int'])
spot.modify_bounds_based_on_var_type()
print(spot.bounds) # [(1, 10)]

31.3.1.4 process_factor_bounds

Processes factor variable bounds (tuples of strings) into integer indices and stores the mapping.

spot = SpotOptim(fun=lambda x: x, bounds=[('red', 'green')], var_name=['color'])
spot.process_factor_bounds()
# Bounds become [(0, 1)] and mapping is stored

31.3.1.5 _repair_non_numeric

Rounds values for discrete variables (int, factor) to the nearest integer.

X = np.array([[1.2, 2.8]])
X_repaired = spot._repair_non_numeric(X, var_type=['int', 'int'])
# [[1.0, 3.0]]

31.3.2 Dimension Reduction

Methods for handling fixed dimensions (where lower bound equals upper bound).

31.3.2.1 _setup_dimension_reduction

Identifies fixed dimensions and reduces the optimization search space to only active variables.

spot = SpotOptim(fun=lambda x: x, bounds=[(1, 1), (0, 10)])
spot._setup_dimension_reduction()
# Dimension 0 is fixed, optimization happens only on Dimension 1

31.3.2.2 to_red_dim

Transforms vectors from the full original space to the reduced optimization space.

X_full = np.array([[1, 5]]) # D=2
X_red = spot.to_red_dim(X_full) # [[5]] (D=1)

31.3.2.3 to_all_dim

Transforms vectors from the reduced optimization space back to the full original space by re-inserting fixed values.

X_red = np.array([[5]])
X_full = spot.to_all_dim(X_red) # [[1, 5]]

31.3.3 Variable Transformation

Methods for scaling and transforming variables for the surrogate model.

31.3.3.1 transform_value / inverse_transform_value

Applies or reverses transformations (e.g., log10, ln) on single values.

val = spot.transform_value(100, "log10") # 2.0
orig = spot.inverse_transform_value(2.0, "log10") # 100.0

31.3.3.2 _transform_X / _inverse_transform_X

Applies or reverses transformations on valid X arrays (batch processing).

X = np.array([[10, 100]])
# Assuming log10 transform for both
X_trans = spot._transform_X(X) # [[1.0, 2.0]]

31.3.4 Initial Design

Methods for generating the initial population of points.

31.3.4.1 get_initial_design

Generates or retrieves the initial set of points to evaluate. Supports optional user-provided starting points (X0).

X_init = spot.get_initial_design()

31.3.4.2 _generate_initial_design

Internal method to generate Latin Hypercube Sampling (LHS) design.

X_lhs = spot._generate_initial_design()

31.3.4.3 _curate_initial_design

Combines generated design with user-provided x0 and ensures valid bounds.

X_curated = spot._curate_initial_design(X_lhs)

31.3.5 Surrogate & Acquisition

Methods for fitting the model and selecting new points.

31.3.5.1 _fit_surrogate

Fits the Gaussian Process surrogate model to the observed data.

spot._fit_surrogate(X_train, y_train)

31.3.5.2 _predict_with_uncertainty

Predicts mean and standard deviation for new points using the surrogate.

mean, std = spot._predict_with_uncertainty(X_new)

31.3.5.3 suggest_next_infill_point

Determines the next point to evaluate by optimizing the acquisition function.

X_next = spot.suggest_next_infill_point()

31.3.6 Optimization Loop

Core logic for the optimization process.

31.3.6.1 optimize

Main entry point. Runs the full optimization loop: initial design, surrogate fitting, and sequential infill.

result = spot.optimize()

31.3.6.2 _optimize_single_run

Internal method to run the optimization loop (called by optimize).

spot._optimize_single_run()

31.3.7 Results & Analysis

Methods for saving, loading, and analyzing results.

31.3.7.1 save_result / load_result

Saves or loads the complete optimizer state, including results.

spot.save_result(prefix="run1")
spot2 = SpotOptim.load_result("run1_res.pkl")

31.3.7.2 save_experiment / load_experiment

Saves or loads only the configuration (no results), useful for distributing experiments.

spot.save_experiment(prefix="exp1")

31.3.8 Plotting

Visualizations for the optimization process.

31.3.8.1 plot_progress

Plots the optimization history (objective value vs. iteration).

spot.plot_progress()

31.3.8.2 plot_surrogate

Plots the surrogate landscape for two dimensions.

spot.plot_surrogate(i=0, j=1)

31.3.8.3 plot_important_hyperparameter_contour

Automatically plots landscapes for the most important parameters.

spot.plot_important_hyperparameter_contour()

31.3.9 TensorBoard Integration

Methods for logging training progress to TensorBoard.

31.3.9.1 _init_tensorboard

Sets up the TensorBoard logging directory and writer.

spot = SpotOptim(..., tensorboard_log=True)
spot._init_tensorboard()

31.3.9.2 _write_tensorboard_scalars

Logs scalar metrics (objective value, best so far) to TensorBoard at each iteration.

spot._write_tensorboard_scalars()

31.3.9.3 _write_tensorboard_hparams

Logs hyperparameters and their corresponding metrics for comparison.

spot._write_tensorboard_hparams(X, y)

31.4 Description of the Infill Point Acquisition Steps

Assume that the initial design has been generated and the surrogate model has been fitted. Now we consider in detail, step-by-step, how the next infill point is selected and describe the methods used by SpotOptim in detail.

The main driver for this process is suggest_next_infill_point, which acts as a dispatcher.

31.4.1 1. Optimize Acquisition Function

The optimizer uses the specified acquisition function (e.g., Lower Confidence Bound, Expected Improvement) to find promising candidates in the transformed and mapped search space.

The specific optimization strategy depends on acquisition_optimizer:

  • TriCands (_optimize_acquisition_tricands): Uses a geometric strategy based on Delaunay triangulation of existing points to propose candidates, evaluating the acquisition function on them.
  • Differential Evolution (_optimize_acquisition_de): Uses scipy.optimize.differential_evolution to globally optimize the acquisition function. Can be cold-started or warm-started with the best solution found so far.
  • Scipy Minimize (_optimize_acquisition_scipy): Uses standard gradient-based optimization (e.g., L-BFGS-B, Nelder-Mead).

31.4.2 2. Candidate Validation and Selection

Once a candidate is proposed (_try_optimizer_candidates), it undergoes rigorous validation:

  1. Rounding (_repair_non_numeric): If variables are integer or categorical, the candidate values are rounded to the nearest valid level.
  2. Uniqueness Check (select_new): The candidate is compared against all previously evaluated points (self.X_) using a distance tolerance (self.tolerance_x).
    • If the candidate is unique (distance > tolerance), it is accepted.
    • If the candidate is too close to an existing point, it is rejected to avoid redundant evaluations.
# Simplified logic flow
candidate = optimize_acquisition()
candidate = repair_types(candidate)
if is_unique(candidate, history):
    return candidate
else:
    reject_candidate()

31.4.3 3. Failure Handling and Fallback Strategies

If the primary optimization strategy fails to find a unique, valid candidate (e.g., the optimizer converges to an already visited optimum), SpotOptim triggers a fallback mechanism (_try_fallback_strategy calling _handle_acquisition_failure).

  • Strategy: By default (acquisition_failure_strategy="random"), it generates a random space-filling point using Latin Hypercube Sampling.
  • Retries: This process repeats up to max_attempts (default 10) times until a unique point is found.
  • Last Resort: If even the random fallback fails to find a unique point after all attempts, the last generated candidate is returned, even if it is a duplicate, to allow the loop to continue (though potentially inefficiently).

31.4.4 Key Arguments and Configuration

The behavior of the infill process is controlled by several key arguments passed to the SpotOptim constructor.

31.4.4.1 tolerance_x

  • Description: Defines the minimum distance required between a proposed candidate and any existing point in the transformed search space. It determines when a point is considered “new” versus “duplicate”. The distance metric used is defined by min_tol_metric.
  • Default: None. Internally sets to np.sqrt(np.spacing(1)) (approx. 1.49e-8).
  • Effect of Evaluation:
    • Larger value: Forces the optimizer to explore points further away from existing solutions. This promotes exploration but prevents fine-tuning (exploitation) in the very late stages.
    • Smaller value: Allows the optimizer to sample points very close to existing ones. Useful for high-precision fine-tuning but increases the risk of numerical issues or redundant evaluations if the surface is flat.

31.4.4.2 min_tol_metric

  • Description: Specifies the distance metric used when checking tolerance_x. This allows standardizing the concept of “closeness” based on the problem topology.
  • Default: "chebyshev"
  • Options: Supports metrics from scipy.spatial.distance.cdist:
    • "chebyshev": L-infinity distance. Checks if max coordinate difference is within tolerance (hypercube check). Default. Matches legacy behavior.
    • "euclidean": Standard L2 distance. Checks distances in a hypersphere.
    • "minkowski": Lp distance (default p=2).
    • Others: "cityblock" (Manhattan), "cosine", "canberra", etc.
  • Effect of Evaluation:
    • “euclidean”: Ensures points are separated by a radial distance, appropriate for most continuous spaces.
    • “chebyshev”: Allows points closer on diagonals but strictly separated on axes, useful for box-constrained problems where axis-independence matters.

31.4.4.3 acquisition_failure_strategy

  • Description: Defines the fallback method used when the acquisition optimizer fails to propose a unique point (usually because the model converges to a known optimum).
  • Default: "random"
  • Effect of Evaluation:
    • "random": Uses Latin Hypercube Sampling (LHS) to propose a space-filling point. This effectively “restarts” exploration in a new area when exploitation stalls.
    • Custom: Users could potentially implement other strategies (e.g., pure random uniform) by modifying the class, but "random" (LHS) is the standard robust choice provided.

31.4.4.4 acquisition_optimizer

  • Description: Specifies the numerical optimization algorithm used to maximize/minimize the acquisition function.
  • Default: "differential_evolution"
  • Effect of Evaluation:
    • "differential_evolution": A global optimizer. More robust at finding the true global optimum of the complex, multi-modal acquisition landscape. generally preferred for better exploration.
    • "L-BFGS-B", "Nelder-Mead" (via scipy): Local optimizers. Much faster but liable to get stuck in local optima of the acquisition function, potentially missing the actual best area to sample next.

31.4.4.5 acquisition_fun_return_size

  • Description: Specifies the number of top candidate points to return from the acquisition function optimization step.
  • Default: 3
  • Effect of Evaluation:
    • Value > 1: The optimizer returns the top \(N\) best candidates found by the acquisition optimizer (e.g., DE or TriCands). SpotOptim then iterates through these candidates in order of quality and selects the first one that is unique (i.e., satisfies tolerance_x with respect to existing points). This significantly improves robustness against converging to duplicates.
    • Value = 1: Only the single best candidate is returned. If this candidate is a duplicate of an existing point, the acquisition step fails immediately and triggers the fallback strategy (e.g., random sampling), potentially missing other valid high-quality candidates.

31.4.5 Example Usage

# Manually triggering the infill suggestion process
# 1. Fit the surrogate model
opt._fit_surrogate(opt.X_, opt.y_)

# 2. Ask for the next point
x_next = opt.suggest_next_infill_point()

# 3. Check if it handles duplicates
# Force a duplicate situation by asking again without updating training data
x_next_2 = opt.suggest_next_infill_point()

if np.allclose(x_next, x_next_2):
    print("Warning: Optimizer returned same point (model hasn't updated)")
else:
    print("Optimizer found a new point or used fallback")

31.5 Using the Surrogate Model

SpotOptim allows to use the surrogate model to predict the objective function values for new points. The default surrogate model is sklearn’s Gaussian Process Regressor. Here we compare the default surrogate model with a simple linear model, which is provided as MLP_Surrogate in the spotoptim.surrogate module.

import numpy as np
import pandas as pd
from spotoptim.surrogate import Kriging, MLPSurrogate
from spotoptim.function.so import rosenbrock, ackley
from sklearn.metrics import mean_squared_error, r2_score

def compare_surrogates(func, dim, n_train, n_test=1000, bounds=(-5, 5)):
    # Generate data
    rng = np.random.default_rng(42)
    X_train = rng.uniform(bounds[0], bounds[1], (n_train, dim))
    y_train = func(X_train)
    
    X_test = rng.uniform(bounds[0], bounds[1], (n_test, dim))
    y_test = func(X_test)
    
    # Kriging (Default)
    kriging = Kriging(seed=42)
    kriging.fit(X_train, y_train)
    y_pred_kriging = kriging.predict(X_test)
    
    # MLP Surrogate
    # Note: MLPs typically need more data, but we'll test on the same set
    mlp = MLPSurrogate(seed=42, epochs=500, verbose=False)
    mlp.fit(X_train, y_train)
    y_pred_mlp = mlp.predict(X_test)
    
    # Metrics
    rmse_kriging = np.sqrt(mean_squared_error(y_test, y_pred_kriging))
    rmse_mlp = np.sqrt(mean_squared_error(y_test, y_pred_mlp))
    
    return {
        "Function": func.__name__,
        "Dim": dim,
        "N_Train": n_train,
        "RMSE (Kriging)": rmse_kriging,
        "RMSE (MLP)": rmse_mlp
    }

# Define Scenarios
scenarios = [
    (rosenbrock, 2, 50),
    (rosenbrock, 20, 200),
    (ackley, 2, 50),
    (ackley, 20, 200)
]

results = []
print(f"{'Function':<15} | {'Dim':<5} | {'N_Train':<8} | {'RMSE Kriging':<15} | {'RMSE MLP':<15}")
print("-" * 70)

for func, dim, n_train in scenarios:
    res = compare_surrogates(func, dim, n_train)
    results.append(res)
    print(f"{res['Function']:<15} | {res['Dim']:<5} | {res['N_Train']:<8} | {res['RMSE (Kriging)']:<15.4f} | {res['RMSE (MLP)']:<15.4f}")
Function        | Dim   | N_Train  | RMSE Kriging    | RMSE MLP       
----------------------------------------------------------------------
rosenbrock      | 2     | 50       | 174.1069        | 4868.6898      
rosenbrock      | 20    | 200      | 42638.8468      | 80035.5678     
ackley          | 2     | 50       | 0.8096          | 0.9328         
ackley          | 20    | 200      | 0.2448          | 0.6950         

31.6 Comparing the SpotOptim With the Default Surrogate Model and MLP Surrogate

Here we run SpotOptim using the default Kriging surrogate and the MLPSurrogate on standard test functions to compare their optimization performance.

from spotoptim import SpotOptim
from spotoptim.surrogate import MLPSurrogate
from spotoptim.function.so import rosenbrock, ackley, robot_arm_hard
import numpy as np

def run_spotoptim_comparison(func, dim, bounds, max_iter=50):
    print(f"\n--- optimizing {func.__name__} ({dim}D) ---")
    
    # Define bounds list
    bounds_list = [bounds] * dim
    
    # 1. Default Surrogate (Kriging)
    print("Running with Default Surrogate (Kriging)...")
    opt_kriging = SpotOptim(
        fun=func,
        bounds=bounds_list,
        max_iter=max_iter,
        n_initial=10,
        seed=42,
        verbose=False
    )
    res_kriging = opt_kriging.optimize()
    
    # 2. MLP Surrogate
    print("Running with MLPSurrogate...")
    # Use fewer epochs for speed in this demo, but enough to learn
    # Note: MLPSurrogate improved with standard defaults (128x3 relu)
    mlp = MLPSurrogate(epochs=200, verbose=False, seed=42)
    
    opt_mlp = SpotOptim(
        fun=func,
        bounds=bounds_list,
        surrogate=mlp,
        max_iter=max_iter,
        n_initial=10,
        seed=42,
        verbose=False
    )
    res_mlp = opt_mlp.optimize()
    
    return {
        "Function": func.__name__,
        "Dim": dim,
        "Best_Kriging": res_kriging.fun,
        "Best_MLP": res_mlp.fun
    }

# Scenarios
scenarios_opt = [
    (rosenbrock, 2, (-5, 5)),
    (rosenbrock, 10, (-5, 5)), 
    (ackley, 2, (-32.768, 32.768)),
    (ackley, 10, (-32.768, 32.768)),
    (robot_arm_hard, 10, (0.0, 1.0))
]

results_opt = []
print(f"\n{'Function':<15} | {'Dim':<5} | {'Best y (Default)':<20} | {'Best y (MLP)':<20}")
print("-" * 70)

for func, dim, bnds in scenarios_opt:
    res = run_spotoptim_comparison(func, dim, bnds, max_iter=50)
    results_opt.append(res)
    print(f"{res['Function']:<15} | {res['Dim']:<5} | {res['Best_Kriging']:<20.6e} | {res['Best_MLP']:<20.6e}")

Function        | Dim   | Best y (Default)     | Best y (MLP)        
----------------------------------------------------------------------

--- optimizing rosenbrock (2D) ---
Running with Default Surrogate (Kriging)...
Running with MLPSurrogate...
rosenbrock      | 2     | 6.658168e-01         | 2.106277e+00        

--- optimizing rosenbrock (10D) ---
Running with Default Surrogate (Kriging)...
Running with MLPSurrogate...
rosenbrock      | 10    | 7.417749e+02         | 1.313298e+04        

--- optimizing ackley (2D) ---
Running with Default Surrogate (Kriging)...
Running with MLPSurrogate...
ackley          | 2     | 3.574453e+00         | 3.282092e+00        

--- optimizing ackley (10D) ---
Running with Default Surrogate (Kriging)...
Running with MLPSurrogate...
ackley          | 10    | 1.994314e+01         | 1.824225e+01        

--- optimizing robot_arm_hard (10D) ---
Running with Default Surrogate (Kriging)...
Running with MLPSurrogate...
robot_arm_hard  | 10    | 2.907500e+01         | 3.657783e+01        

31.7 Multi-Surrogate Switching in SpotOptim

SpotOptim also supports probabilistic multi-surrogate optimization. This allows the optimizer to switch between different surrogate models (e.g., Gaussian Process and MLP) during the optimization loop. This can be beneficial for escaping local optima or combining the strengths of different modeling approaches (e.g., local exploitation of GPs vs global approximation of Neural Networks).

To use this feature, pass a list of surrogate objects to the surrogate argument. You can optionally control the selection probability using prob_surrogate.

31.7.1 Comparison: Default (GP) vs Multi-Surrogate (GP + MLP) on Robot Arm

The following example demonstrates how to set up a hybrid optimization strategy combining: 1. GaussianProcessRegressor (default): Good for valid uncertainty estimation and exploitation. 2. MLPSurrogate: Good for global trend modeling and scalability.

We compare the performance on the 10D Robot Arm problem.

from spotoptim import SpotOptim
from spotoptim.surrogate import MLPSurrogate
from spotoptim.function.so import robot_arm_hard
from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import Matern, ConstantKernel
import numpy as np

# 1. Setup the Benchmark
func = robot_arm_hard
dim = 10
bounds = [(0.0, 1.0)] * dim
max_iter = 1000
seed = 42

print(f"--- Optimizing {func.__name__} (10D) ---")

# --- Run 1: Multi-Surrogate (GP + MLP) ---
print("1. Running Multi-Surrogate SpotOptim (GP + MLP)...")

# Define Model A: Gaussian Process (Same as default)
kernel = ConstantKernel(1.0, (1e-2, 1e12)) * Matern(
    length_scale=1.0, length_scale_bounds=(1e-4, 1e2), nu=2.5
)
gp_model = GaussianProcessRegressor(
    kernel=kernel,
    n_restarts_optimizer=10,
    normalize_y=True,
    random_state=seed
)

# Define Model B: MLP Surrogate
mlp_model = MLPSurrogate(
    epochs=1000, 
    verbose=False, 
    seed=seed,
    l1=128, 
    num_hidden_layers=3, 
    activation="relu"
)

# Run Optimization with both surrogates
opt_multi = SpotOptim(
    fun=func,
    bounds=bounds,
    surrogate=[gp_model, mlp_model],   # Pass list of models
    prob_surrogate=[0.5, 0.5],         # 50% chance for each at every step
    max_iter=max_iter,
    n_initial=20, # Increased for 10D to support Delaunay triangulation (needs > d+1 points)
    seed=seed,
    verbose=True,
    acquisition_optimizer="de_tricands",
    prob_de_tricands=0.1,
    restart_after_n=10,
)
res_multi = opt_multi.optimize()
print(f"   Best y (Multi-Surrogate): {res_multi.fun:.6f}")

# --- Run 2: Default SpotOptim (Standard GP) ---
print("2. Running Default SpotOptim (GP only)...")
opt_default = SpotOptim(
    fun=func,
    bounds=bounds,
    max_iter=max_iter,
    n_initial=20,
    seed=seed,
    verbose=True,
    acquisition_optimizer="de_tricands",
    prob_de_tricands=0.1,
    restart_after_n=10,
)
res_default = opt_default.optimize()
print(f"   Best y (Default): {res_default.fun:.6f}")


# --- Summary ---
print("\n--- Results Summary ---")
print(f"{'Method':<25} | {'Best Objective':<15}")
print("-" * 40)
print(f"{'Default (GP Only)':<25} | {res_default.fun:<15.6f}")
print(f"{'Multi-Surrogate (GP+MLP)':<25} | {res_multi.fun:<15.6f}")
--- Optimizing robot_arm_hard (10D) ---
1. Running Multi-Surrogate SpotOptim (GP + MLP)...
TensorBoard logging disabled
Initial best: f(x) = 27.216160
Iter 1 | Best: 27.216160 | Curr: 408.060597 | Rate: 0.00 | Evals: 2.1%
Iter 2 | Best: 27.216160 | Curr: 32.079643 | Rate: 0.00 | Evals: 2.2%
Iter 3 | Best: 27.216160 | Curr: 614.515969 | Rate: 0.00 | Evals: 2.3%
Iter 4 | Best: 27.216160 | Curr: 31.070892 | Rate: 0.00 | Evals: 2.4%
Iter 5 | Best: 27.216160 | Curr: 73.689906 | Rate: 0.00 | Evals: 2.5%
Iter 6 | Best: 27.216160 | Curr: 36.703689 | Rate: 0.00 | Evals: 2.6%
Iter 7 | Best: 27.216160 | Curr: 48.850756 | Rate: 0.00 | Evals: 2.7%
Iter 8 | Best: 27.216160 | Curr: 4744.563192 | Rate: 0.00 | Evals: 2.8%
Iter 9 | Best: 27.216160 | Curr: 42.207683 | Rate: 0.00 | Evals: 2.9%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.38927077 0.97957357 0.90732985 0.58830303 0.64708486 0.08593081
 0.93532031 0.61690417 0.87214839 0.01080509]
  Internal scale: [0.38927077 0.97957357 0.90732985 0.58830303 0.64708486 0.08593081
 0.93532031 0.61690417 0.87214839 0.01080509]
Restart injection: Using best found point so far as starting point (f(x)=27.216160).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 22.241614
Iter 1 | Best: 22.241614 | Curr: 75.871283 | Rate: 0.00 | Evals: 5.0%
Iter 2 | Best: 22.241614 | Curr: 4432.689681 | Rate: 0.00 | Evals: 5.1%
Iter 3 | Best: 22.241614 | Curr: 869.250605 | Rate: 0.00 | Evals: 5.2%
Iter 4 | Best: 22.241614 | Curr: 180.014620 | Rate: 0.00 | Evals: 5.3%
Iter 5 | Best: 22.241614 | Curr: 46.137751 | Rate: 0.00 | Evals: 5.4%
Iter 6 | Best: 22.241614 | Curr: 59.808810 | Rate: 0.00 | Evals: 5.5%
Iter 7 | Best: 22.241614 | Curr: 1591.892670 | Rate: 0.00 | Evals: 5.6%
Iter 8 | Best: 22.241614 | Curr: 144.936808 | Rate: 0.00 | Evals: 5.7%
Iter 9 | Best: 22.241614 | Curr: 79.019147 | Rate: 0.00 | Evals: 5.8%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.29261083 0.85319363 0.8781048  0.33083401 0.21351571 0.67235035
 0.253193   0.76098493 0.42603152 0.23118203]
  Internal scale: [0.29261083 0.85319363 0.8781048  0.33083401 0.21351571 0.67235035
 0.253193   0.76098493 0.42603152 0.23118203]
Restart injection: Using best found point so far as starting point (f(x)=22.241614).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 18.210089
Iter 1 | Best: 18.210089 | Curr: 1870.808970 | Rate: 0.00 | Evals: 7.9%
Iter 2 | Best: 18.210089 | Curr: 168.755773 | Rate: 0.00 | Evals: 8.0%
Iter 3 | Best: 18.210089 | Curr: 2587.745741 | Rate: 0.00 | Evals: 8.1%
Iter 4 | Best: 18.210089 | Curr: 198.415505 | Rate: 0.00 | Evals: 8.2%
Iter 5 | Best: 18.210089 | Curr: 177.535536 | Rate: 0.00 | Evals: 8.3%
Iter 6 | Best: 18.210089 | Curr: 178.226174 | Rate: 0.00 | Evals: 8.4%
Iter 7 | Best: 18.210089 | Curr: 150.286776 | Rate: 0.00 | Evals: 8.5%
Iter 8 | Best: 18.210089 | Curr: 130.824176 | Rate: 0.00 | Evals: 8.6%
Iter 9 | Best: 18.210089 | Curr: 219.072253 | Rate: 0.00 | Evals: 8.7%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
  Internal scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
Restart injection: Using best found point so far as starting point (f(x)=18.210089).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 18.210089
Iter 1 | Best: 18.210089 | Curr: 3357.417230 | Rate: 0.00 | Evals: 10.8%
Iter 2 | Best: 18.210089 | Curr: 4314.651618 | Rate: 0.00 | Evals: 10.9%
Iter 3 | Best: 18.210089 | Curr: 30.887547 | Rate: 0.00 | Evals: 11.0%
Iter 4 | Best: 18.210089 | Curr: 422.492760 | Rate: 0.00 | Evals: 11.1%
Iter 5 | Best: 18.210089 | Curr: 2341.400055 | Rate: 0.00 | Evals: 11.2%
Iter 6 | Best: 18.210089 | Curr: 57.200237 | Rate: 0.00 | Evals: 11.3%
Iter 7 | Best: 17.809128 | Rate: 0.14 | Evals: 11.4%
Iter 8 | Best: 17.809128 | Curr: 72.494594 | Rate: 0.12 | Evals: 11.5%
Iter 9 | Best: 17.809128 | Curr: 1852.680576 | Rate: 0.11 | Evals: 11.6%
Iter 10 | Best: 17.809128 | Curr: 2544.181276 | Rate: 0.10 | Evals: 11.7%
Iter 11 | Best: 17.809128 | Curr: 1222.883314 | Rate: 0.10 | Evals: 11.8%
Iter 12 | Best: 17.809128 | Curr: 117.873436 | Rate: 0.10 | Evals: 11.9%
Iter 13 | Best: 17.809128 | Curr: 4027.322403 | Rate: 0.10 | Evals: 12.0%
Iter 14 | Best: 17.809128 | Curr: 33.115804 | Rate: 0.10 | Evals: 12.1%
Iter 15 | Best: 17.809128 | Curr: 75.402057 | Rate: 0.10 | Evals: 12.2%
Iter 16 | Best: 17.809128 | Curr: 112.308263 | Rate: 0.10 | Evals: 12.3%
Iter 17 | Best: 17.809128 | Curr: 1072.349826 | Rate: 0.00 | Evals: 12.4%
Iter 18 | Best: 17.809128 | Curr: 118.184399 | Rate: 0.00 | Evals: 12.5%
Iter 19 | Best: 17.809128 | Curr: 35.703206 | Rate: 0.00 | Evals: 12.6%
Iter 20 | Best: 17.809128 | Curr: 26.226740 | Rate: 0.00 | Evals: 12.7%
Iter 21 | Best: 17.809128 | Curr: 4383.664345 | Rate: 0.00 | Evals: 12.8%
Iter 22 | Best: 17.809128 | Curr: 27.840447 | Rate: 0.00 | Evals: 12.9%
Iter 23 | Best: 17.809128 | Curr: 51.036189 | Rate: 0.00 | Evals: 13.0%
Iter 24 | Best: 17.809128 | Curr: 41.351436 | Rate: 0.00 | Evals: 13.1%
Iter 25 | Best: 17.809128 | Curr: 1516.161368 | Rate: 0.00 | Evals: 13.2%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49043147 0.82203103 0.24484649 0.01675079 0.2020707  0.57427482
 0.62822569 0.34848927 0.50130396 0.54523182]
  Internal scale: [0.49043147 0.82203103 0.24484649 0.01675079 0.2020707  0.57427482
 0.62822569 0.34848927 0.50130396 0.54523182]
Restart injection: Using best found point so far as starting point (f(x)=17.809128).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.809128
Iter 1 | Best: 17.809128 | Curr: 73.235622 | Rate: 0.00 | Evals: 15.3%
Iter 2 | Best: 17.809128 | Curr: 5703.732031 | Rate: 0.00 | Evals: 15.4%
Iter 3 | Best: 17.809128 | Curr: 3055.553762 | Rate: 0.00 | Evals: 15.5%
Iter 4 | Best: 17.809128 | Curr: 5239.974611 | Rate: 0.00 | Evals: 15.6%
Iter 5 | Best: 17.809128 | Curr: 2543.777839 | Rate: 0.00 | Evals: 15.7%
Iter 6 | Best: 17.809128 | Curr: 1590.262366 | Rate: 0.00 | Evals: 15.8%
Iter 7 | Best: 17.809128 | Curr: 62.314813 | Rate: 0.00 | Evals: 15.9%
Iter 8 | Best: 17.809128 | Curr: 164.673496 | Rate: 0.00 | Evals: 16.0%
Iter 9 | Best: 17.809128 | Curr: 816.495969 | Rate: 0.00 | Evals: 16.1%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49043147 0.82203103 0.24484649 0.01675079 0.2020707  0.57427482
 0.62822569 0.34848927 0.50130396 0.54523182]
  Internal scale: [0.49043147 0.82203103 0.24484649 0.01675079 0.2020707  0.57427482
 0.62822569 0.34848927 0.50130396 0.54523182]
Restart injection: Using best found point so far as starting point (f(x)=17.809128).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.809128
Iter 1 | Best: 17.809128 | Curr: 3691.048293 | Rate: 0.00 | Evals: 18.2%
Iter 2 | Best: 17.809128 | Curr: 64.667834 | Rate: 0.00 | Evals: 18.3%
Iter 3 | Best: 17.809128 | Curr: 51.879610 | Rate: 0.00 | Evals: 18.4%
Iter 4 | Best: 17.809128 | Curr: 4068.315286 | Rate: 0.00 | Evals: 18.5%
Iter 5 | Best: 17.809128 | Curr: 189.569247 | Rate: 0.00 | Evals: 18.6%
Iter 6 | Best: 17.809128 | Curr: 200.205839 | Rate: 0.00 | Evals: 18.7%
Iter 7 | Best: 17.809128 | Curr: 3648.160720 | Rate: 0.00 | Evals: 18.8%
Iter 8 | Best: 17.809128 | Curr: 1676.054265 | Rate: 0.00 | Evals: 18.9%
Iter 9 | Best: 17.809128 | Curr: 120.643467 | Rate: 0.00 | Evals: 19.0%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49043147 0.82203103 0.24484649 0.01675079 0.2020707  0.57427482
 0.62822569 0.34848927 0.50130396 0.54523182]
  Internal scale: [0.49043147 0.82203103 0.24484649 0.01675079 0.2020707  0.57427482
 0.62822569 0.34848927 0.50130396 0.54523182]
Restart injection: Using best found point so far as starting point (f(x)=17.809128).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.809128
Iter 1 | Best: 17.809128 | Curr: 3034.017302 | Rate: 0.00 | Evals: 21.1%
Iter 2 | Best: 17.809128 | Curr: 71.406341 | Rate: 0.00 | Evals: 21.2%
Iter 3 | Best: 17.809128 | Curr: 5787.748956 | Rate: 0.00 | Evals: 21.3%
Iter 4 | Best: 17.809128 | Curr: 189.318675 | Rate: 0.00 | Evals: 21.4%
Iter 5 | Best: 17.809128 | Curr: 239.297363 | Rate: 0.00 | Evals: 21.5%
Iter 6 | Best: 17.809128 | Curr: 188.662028 | Rate: 0.00 | Evals: 21.6%
Iter 7 | Best: 17.809128 | Curr: 171.473305 | Rate: 0.00 | Evals: 21.7%
Iter 8 | Best: 17.809128 | Curr: 194.177725 | Rate: 0.00 | Evals: 21.8%
Iter 9 | Best: 17.809128 | Curr: 197.097759 | Rate: 0.00 | Evals: 21.9%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49043147 0.82203103 0.24484649 0.01675079 0.2020707  0.57427482
 0.62822569 0.34848927 0.50130396 0.54523182]
  Internal scale: [0.49043147 0.82203103 0.24484649 0.01675079 0.2020707  0.57427482
 0.62822569 0.34848927 0.50130396 0.54523182]
Restart injection: Using best found point so far as starting point (f(x)=17.809128).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.809128
Iter 1 | Best: 17.809128 | Curr: 3202.986633 | Rate: 0.00 | Evals: 24.0%
Iter 2 | Best: 17.809128 | Curr: 65.096428 | Rate: 0.00 | Evals: 24.1%
Iter 3 | Best: 17.809128 | Curr: 2757.767836 | Rate: 0.00 | Evals: 24.2%
Iter 4 | Best: 17.809128 | Curr: 224.896572 | Rate: 0.00 | Evals: 24.3%
Iter 5 | Best: 17.809128 | Curr: 466.385693 | Rate: 0.00 | Evals: 24.4%
Iter 6 | Best: 17.809128 | Curr: 5189.737007 | Rate: 0.00 | Evals: 24.5%
Iter 7 | Best: 17.809128 | Curr: 1897.951577 | Rate: 0.00 | Evals: 24.6%
Iter 8 | Best: 17.809128 | Curr: 213.554877 | Rate: 0.00 | Evals: 24.7%
Iter 9 | Best: 17.809128 | Curr: 3470.872323 | Rate: 0.00 | Evals: 24.8%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49043147 0.82203103 0.24484649 0.01675079 0.2020707  0.57427482
 0.62822569 0.34848927 0.50130396 0.54523182]
  Internal scale: [0.49043147 0.82203103 0.24484649 0.01675079 0.2020707  0.57427482
 0.62822569 0.34848927 0.50130396 0.54523182]
Restart injection: Using best found point so far as starting point (f(x)=17.809128).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.809128
Iter 1 | Best: 17.809128 | Curr: 3410.096887 | Rate: 0.00 | Evals: 26.9%
Iter 2 | Best: 17.809128 | Curr: 77.667449 | Rate: 0.00 | Evals: 27.0%
Iter 3 | Best: 17.809128 | Curr: 1409.730848 | Rate: 0.00 | Evals: 27.1%
Iter 4 | Best: 17.801886 | Rate: 0.25 | Evals: 27.2%
Iter 5 | Best: 17.801886 | Curr: 128.409255 | Rate: 0.20 | Evals: 27.3%
Iter 6 | Best: 17.801886 | Curr: 151.686269 | Rate: 0.17 | Evals: 27.4%
Iter 7 | Best: 17.801886 | Curr: 148.986188 | Rate: 0.14 | Evals: 27.5%
Iter 8 | Best: 17.801886 | Curr: 147.903408 | Rate: 0.12 | Evals: 27.6%
Iter 9 | Best: 17.801886 | Curr: 123.791161 | Rate: 0.11 | Evals: 27.7%
Iter 10 | Best: 17.801886 | Curr: 158.020835 | Rate: 0.10 | Evals: 27.8%
Iter 11 | Best: 17.801886 | Curr: 145.628696 | Rate: 0.10 | Evals: 27.9%
Iter 12 | Best: 17.801886 | Curr: 184.461897 | Rate: 0.10 | Evals: 28.0%
Iter 13 | Best: 17.801886 | Curr: 166.614018 | Rate: 0.10 | Evals: 28.1%
Iter 14 | Best: 17.801886 | Curr: 173.973951 | Rate: 0.00 | Evals: 28.2%
Iter 15 | Best: 17.801886 | Curr: 145.281937 | Rate: 0.00 | Evals: 28.3%
Iter 16 | Best: 17.801886 | Curr: 100.801971 | Rate: 0.00 | Evals: 28.4%
Iter 17 | Best: 17.801886 | Curr: 83.309241 | Rate: 0.00 | Evals: 28.5%
Iter 18 | Best: 17.801886 | Curr: 55.822952 | Rate: 0.00 | Evals: 28.6%
Iter 19 | Best: 17.801886 | Curr: 113.194883 | Rate: 0.00 | Evals: 28.7%
Iter 20 | Best: 17.801886 | Curr: 79.540647 | Rate: 0.00 | Evals: 28.8%
Iter 21 | Best: 17.801886 | Curr: 5089.667369 | Rate: 0.00 | Evals: 28.9%
Iter 22 | Best: 17.801886 | Curr: 116.392257 | Rate: 0.00 | Evals: 29.0%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49043127 0.82206513 0.24483683 0.01670678 0.20204161 0.57428185
 0.62824803 0.34847773 0.50131217 0.54525043]
  Internal scale: [0.49043127 0.82206513 0.24483683 0.01670678 0.20204161 0.57428185
 0.62824803 0.34847773 0.50131217 0.54525043]
Restart injection: Using best found point so far as starting point (f(x)=17.801886).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.801886
Iter 1 | Best: 17.801886 | Curr: 113.116320 | Rate: 0.00 | Evals: 31.1%
Iter 2 | Best: 17.801886 | Curr: 180.213272 | Rate: 0.00 | Evals: 31.2%
Iter 3 | Best: 17.801886 | Curr: 129.117612 | Rate: 0.00 | Evals: 31.3%
Iter 4 | Best: 17.801886 | Curr: 43.248957 | Rate: 0.00 | Evals: 31.4%
Iter 5 | Best: 17.801886 | Curr: 53.572768 | Rate: 0.00 | Evals: 31.5%
Iter 6 | Best: 17.801886 | Curr: 66.883075 | Rate: 0.00 | Evals: 31.6%
Iter 7 | Best: 17.801886 | Curr: 54.641203 | Rate: 0.00 | Evals: 31.7%
Iter 8 | Best: 17.801886 | Curr: 312.851856 | Rate: 0.00 | Evals: 31.8%
Iter 9 | Best: 17.801886 | Curr: 65.824793 | Rate: 0.00 | Evals: 31.9%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49043127 0.82206513 0.24483683 0.01670678 0.20204161 0.57428185
 0.62824803 0.34847773 0.50131217 0.54525043]
  Internal scale: [0.49043127 0.82206513 0.24483683 0.01670678 0.20204161 0.57428185
 0.62824803 0.34847773 0.50131217 0.54525043]
Restart injection: Using best found point so far as starting point (f(x)=17.801886).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.801886
Iter 1 | Best: 17.801886 | Curr: 270.329516 | Rate: 0.00 | Evals: 34.0%
Iter 2 | Best: 17.801886 | Curr: 101.844153 | Rate: 0.00 | Evals: 34.1%
Iter 3 | Best: 17.801886 | Curr: 100.207933 | Rate: 0.00 | Evals: 34.2%
Iter 4 | Best: 17.801886 | Curr: 147.548191 | Rate: 0.00 | Evals: 34.3%
Iter 5 | Best: 17.801886 | Curr: 70.716794 | Rate: 0.00 | Evals: 34.4%
Iter 6 | Best: 17.801886 | Curr: 75.899989 | Rate: 0.00 | Evals: 34.5%
Iter 7 | Best: 17.801886 | Curr: 72.383219 | Rate: 0.00 | Evals: 34.6%
Iter 8 | Best: 17.801886 | Curr: 61.976289 | Rate: 0.00 | Evals: 34.7%
Iter 9 | Best: 17.801886 | Curr: 107.709653 | Rate: 0.00 | Evals: 34.8%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49043127 0.82206513 0.24483683 0.01670678 0.20204161 0.57428185
 0.62824803 0.34847773 0.50131217 0.54525043]
  Internal scale: [0.49043127 0.82206513 0.24483683 0.01670678 0.20204161 0.57428185
 0.62824803 0.34847773 0.50131217 0.54525043]
Restart injection: Using best found point so far as starting point (f(x)=17.801886).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.801886
Iter 1 | Best: 17.801886 | Curr: 2221.031495 | Rate: 0.00 | Evals: 36.9%
Iter 2 | Best: 17.801886 | Curr: 2160.061784 | Rate: 0.00 | Evals: 37.0%
Iter 3 | Best: 17.801886 | Curr: 202.871210 | Rate: 0.00 | Evals: 37.1%
Iter 4 | Best: 17.801886 | Curr: 210.268661 | Rate: 0.00 | Evals: 37.2%
Iter 5 | Best: 17.801886 | Curr: 57.228061 | Rate: 0.00 | Evals: 37.3%
Iter 6 | Best: 17.801886 | Curr: 52.942773 | Rate: 0.00 | Evals: 37.4%
Iter 7 | Best: 17.801886 | Curr: 180.092747 | Rate: 0.00 | Evals: 37.5%
Iter 8 | Best: 17.801886 | Curr: 154.502538 | Rate: 0.00 | Evals: 37.6%
Iter 9 | Best: 17.801886 | Curr: 654.843267 | Rate: 0.00 | Evals: 37.7%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49043127 0.82206513 0.24483683 0.01670678 0.20204161 0.57428185
 0.62824803 0.34847773 0.50131217 0.54525043]
  Internal scale: [0.49043127 0.82206513 0.24483683 0.01670678 0.20204161 0.57428185
 0.62824803 0.34847773 0.50131217 0.54525043]
Restart injection: Using best found point so far as starting point (f(x)=17.801886).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.801886
Iter 1 | Best: 17.801886 | Curr: 3518.814372 | Rate: 0.00 | Evals: 39.8%
Iter 2 | Best: 17.801886 | Curr: 25.909100 | Rate: 0.00 | Evals: 39.9%
Iter 3 | Best: 17.801886 | Curr: 31.233326 | Rate: 0.00 | Evals: 40.0%
Iter 4 | Best: 17.801886 | Curr: 41.357350 | Rate: 0.00 | Evals: 40.1%
Iter 5 | Best: 17.801886 | Curr: 131.279903 | Rate: 0.00 | Evals: 40.2%
Iter 6 | Best: 17.801886 | Curr: 53.479594 | Rate: 0.00 | Evals: 40.3%
Iter 7 | Best: 17.801886 | Curr: 23.108914 | Rate: 0.00 | Evals: 40.4%
Iter 8 | Best: 17.801886 | Curr: 25.710439 | Rate: 0.00 | Evals: 40.5%
Iter 9 | Best: 17.801886 | Curr: 43.514529 | Rate: 0.00 | Evals: 40.6%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49043127 0.82206513 0.24483683 0.01670678 0.20204161 0.57428185
 0.62824803 0.34847773 0.50131217 0.54525043]
  Internal scale: [0.49043127 0.82206513 0.24483683 0.01670678 0.20204161 0.57428185
 0.62824803 0.34847773 0.50131217 0.54525043]
Restart injection: Using best found point so far as starting point (f(x)=17.801886).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.801886
Iter 1 | Best: 17.801886 | Curr: 5296.134145 | Rate: 0.00 | Evals: 42.7%
Iter 2 | Best: 17.801886 | Curr: 5680.959887 | Rate: 0.00 | Evals: 42.8%
Iter 3 | Best: 17.801886 | Curr: 2361.191219 | Rate: 0.00 | Evals: 42.9%
Iter 4 | Best: 17.801886 | Curr: 2374.645740 | Rate: 0.00 | Evals: 43.0%
Iter 5 | Best: 17.801886 | Curr: 6539.829595 | Rate: 0.00 | Evals: 43.1%
Iter 6 | Best: 17.801886 | Curr: 396.214287 | Rate: 0.00 | Evals: 43.2%
Iter 7 | Best: 17.801886 | Curr: 2088.304935 | Rate: 0.00 | Evals: 43.3%
Iter 8 | Best: 17.801886 | Curr: 1524.560779 | Rate: 0.00 | Evals: 43.4%
Iter 9 | Best: 17.801886 | Curr: 200.987415 | Rate: 0.00 | Evals: 43.5%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49043127 0.82206513 0.24483683 0.01670678 0.20204161 0.57428185
 0.62824803 0.34847773 0.50131217 0.54525043]
  Internal scale: [0.49043127 0.82206513 0.24483683 0.01670678 0.20204161 0.57428185
 0.62824803 0.34847773 0.50131217 0.54525043]
Restart injection: Using best found point so far as starting point (f(x)=17.801886).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.424393
Iter 1 | Best: 17.424393 | Curr: 46.905646 | Rate: 0.00 | Evals: 45.6%
Iter 2 | Best: 17.424393 | Curr: 92.315031 | Rate: 0.00 | Evals: 45.7%
Iter 3 | Best: 17.424393 | Curr: 56.840351 | Rate: 0.00 | Evals: 45.8%
Iter 4 | Best: 17.424393 | Curr: 28.809262 | Rate: 0.00 | Evals: 45.9%
Iter 5 | Best: 17.424393 | Curr: 58.323057 | Rate: 0.00 | Evals: 46.0%
Iter 6 | Best: 17.424393 | Curr: 54.830915 | Rate: 0.00 | Evals: 46.1%
Iter 7 | Best: 17.424393 | Curr: 7051.579925 | Rate: 0.00 | Evals: 46.2%
Iter 8 | Best: 17.424393 | Curr: 128.946863 | Rate: 0.00 | Evals: 46.3%
Iter 9 | Best: 17.424393 | Curr: 50.345964 | Rate: 0.00 | Evals: 46.4%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
  Internal scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
Restart injection: Using best found point so far as starting point (f(x)=17.424393).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.424393
Iter 1 | Best: 17.424393 | Curr: 90.468071 | Rate: 0.00 | Evals: 48.5%
Iter 2 | Best: 17.424393 | Curr: 107.582087 | Rate: 0.00 | Evals: 48.6%
Iter 3 | Best: 17.424393 | Curr: 89.759656 | Rate: 0.00 | Evals: 48.7%
Iter 4 | Best: 17.424393 | Curr: 109.923161 | Rate: 0.00 | Evals: 48.8%
Iter 5 | Best: 17.424393 | Curr: 51.549568 | Rate: 0.00 | Evals: 48.9%
Iter 6 | Best: 17.424393 | Curr: 4779.399837 | Rate: 0.00 | Evals: 49.0%
Iter 7 | Best: 17.424393 | Curr: 51.494724 | Rate: 0.00 | Evals: 49.1%
Iter 8 | Best: 17.424393 | Curr: 157.155153 | Rate: 0.00 | Evals: 49.2%
Iter 9 | Best: 17.424393 | Curr: 37.111003 | Rate: 0.00 | Evals: 49.3%
Iter 10 | Best: 16.792077 | Rate: 0.10 | Evals: 49.4%
Iter 11 | Best: 16.792077 | Curr: 51.553329 | Rate: 0.10 | Evals: 49.5%
Iter 12 | Best: 16.792077 | Curr: 83.177860 | Rate: 0.10 | Evals: 49.6%
Iter 13 | Best: 16.792077 | Curr: 757.066410 | Rate: 0.10 | Evals: 49.7%
Iter 14 | Best: 16.792077 | Curr: 81.798104 | Rate: 0.10 | Evals: 49.8%
Iter 15 | Best: 16.792077 | Curr: 32.154617 | Rate: 0.10 | Evals: 49.9%
Iter 16 | Best: 16.792077 | Curr: 1714.066317 | Rate: 0.10 | Evals: 50.0%
Iter 17 | Best: 16.792077 | Curr: 173.770246 | Rate: 0.10 | Evals: 50.1%
Iter 18 | Best: 16.792077 | Curr: 74.167619 | Rate: 0.10 | Evals: 50.2%
Iter 19 | Best: 16.792077 | Curr: 104.577233 | Rate: 0.10 | Evals: 50.3%
Iter 20 | Best: 16.792077 | Curr: 79.164963 | Rate: 0.00 | Evals: 50.4%
Iter 21 | Best: 16.792077 | Curr: 75.480327 | Rate: 0.00 | Evals: 50.5%
Iter 22 | Best: 16.792077 | Curr: 165.511918 | Rate: 0.00 | Evals: 50.6%
Iter 23 | Best: 16.792077 | Curr: 130.446812 | Rate: 0.00 | Evals: 50.7%
Iter 24 | Best: 16.792077 | Curr: 56.660925 | Rate: 0.00 | Evals: 50.8%
Iter 25 | Best: 16.792077 | Curr: 28.532691 | Rate: 0.00 | Evals: 50.9%
Iter 26 | Best: 16.792077 | Curr: 90.661075 | Rate: 0.00 | Evals: 51.0%
Iter 27 | Best: 16.792077 | Curr: 53.843935 | Rate: 0.00 | Evals: 51.1%
Iter 28 | Best: 16.792077 | Curr: 51.111731 | Rate: 0.00 | Evals: 51.2%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.41778886 0.05522899 0.2657875  0.77881433 0.99947306 0.77367191
 0.46685151 0.23501268 0.00945252 0.97570051]
  Internal scale: [0.41778886 0.05522899 0.2657875  0.77881433 0.99947306 0.77367191
 0.46685151 0.23501268 0.00945252 0.97570051]
Restart injection: Using best found point so far as starting point (f(x)=16.792077).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 16.792077
Iter 1 | Best: 16.792077 | Curr: 3694.478040 | Rate: 0.00 | Evals: 53.3%
Iter 2 | Best: 16.792077 | Curr: 189.353885 | Rate: 0.00 | Evals: 53.4%
Iter 3 | Best: 16.792077 | Curr: 1670.360482 | Rate: 0.00 | Evals: 53.5%
Iter 4 | Best: 16.792077 | Curr: 145.018475 | Rate: 0.00 | Evals: 53.6%
Iter 5 | Best: 16.792077 | Curr: 161.432111 | Rate: 0.00 | Evals: 53.7%
Iter 6 | Best: 16.792077 | Curr: 180.940535 | Rate: 0.00 | Evals: 53.8%
Iter 7 | Best: 16.792077 | Curr: 163.959573 | Rate: 0.00 | Evals: 53.9%
Iter 8 | Best: 16.792077 | Curr: 60.491765 | Rate: 0.00 | Evals: 54.0%
Iter 9 | Best: 16.792077 | Curr: 180.271299 | Rate: 0.00 | Evals: 54.1%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.41778886 0.05522899 0.2657875  0.77881433 0.99947306 0.77367191
 0.46685151 0.23501268 0.00945252 0.97570051]
  Internal scale: [0.41778886 0.05522899 0.2657875  0.77881433 0.99947306 0.77367191
 0.46685151 0.23501268 0.00945252 0.97570051]
Restart injection: Using best found point so far as starting point (f(x)=16.792077).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 16.792077
Iter 1 | Best: 16.792077 | Curr: 44.036271 | Rate: 0.00 | Evals: 56.2%
Iter 2 | Best: 16.792077 | Curr: 207.544711 | Rate: 0.00 | Evals: 56.3%
Iter 3 | Best: 16.792077 | Curr: 191.864009 | Rate: 0.00 | Evals: 56.4%
Iter 4 | Best: 16.792077 | Curr: 91.242270 | Rate: 0.00 | Evals: 56.5%
Iter 5 | Best: 16.792077 | Curr: 8149.351520 | Rate: 0.00 | Evals: 56.6%
Iter 6 | Best: 16.792077 | Curr: 91.933495 | Rate: 0.00 | Evals: 56.7%
Iter 7 | Best: 16.792077 | Curr: 72.189147 | Rate: 0.00 | Evals: 56.8%
Iter 8 | Best: 16.792077 | Curr: 117.829842 | Rate: 0.00 | Evals: 56.9%
Iter 9 | Best: 16.792077 | Curr: 197.459067 | Rate: 0.00 | Evals: 57.0%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.41778886 0.05522899 0.2657875  0.77881433 0.99947306 0.77367191
 0.46685151 0.23501268 0.00945252 0.97570051]
  Internal scale: [0.41778886 0.05522899 0.2657875  0.77881433 0.99947306 0.77367191
 0.46685151 0.23501268 0.00945252 0.97570051]
Restart injection: Using best found point so far as starting point (f(x)=16.792077).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 16.792077
Iter 1 | Best: 16.792077 | Curr: 187.520091 | Rate: 0.00 | Evals: 59.1%
Iter 2 | Best: 16.792077 | Curr: 1222.622379 | Rate: 0.00 | Evals: 59.2%
Iter 3 | Best: 16.792077 | Curr: 677.198138 | Rate: 0.00 | Evals: 59.3%
Iter 4 | Best: 16.792077 | Curr: 172.909302 | Rate: 0.00 | Evals: 59.4%
Iter 5 | Best: 16.792077 | Curr: 46.374058 | Rate: 0.00 | Evals: 59.5%
Iter 6 | Best: 16.792077 | Curr: 982.433165 | Rate: 0.00 | Evals: 59.6%
Iter 7 | Best: 16.792077 | Curr: 183.669041 | Rate: 0.00 | Evals: 59.7%
Iter 8 | Best: 16.792077 | Curr: 217.986954 | Rate: 0.00 | Evals: 59.8%
Iter 9 | Best: 16.792077 | Curr: 188.489106 | Rate: 0.00 | Evals: 59.9%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.41778886 0.05522899 0.2657875  0.77881433 0.99947306 0.77367191
 0.46685151 0.23501268 0.00945252 0.97570051]
  Internal scale: [0.41778886 0.05522899 0.2657875  0.77881433 0.99947306 0.77367191
 0.46685151 0.23501268 0.00945252 0.97570051]
Restart injection: Using best found point so far as starting point (f(x)=16.792077).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 16.792077
Iter 1 | Best: 16.792077 | Curr: 136.280733 | Rate: 0.00 | Evals: 62.0%
Iter 2 | Best: 16.792077 | Curr: 112.944174 | Rate: 0.00 | Evals: 62.1%
Iter 3 | Best: 16.792077 | Curr: 150.344719 | Rate: 0.00 | Evals: 62.2%
Iter 4 | Best: 16.792077 | Curr: 145.641312 | Rate: 0.00 | Evals: 62.3%
Iter 5 | Best: 16.792077 | Curr: 98.467814 | Rate: 0.00 | Evals: 62.4%
Iter 6 | Best: 16.792077 | Curr: 116.339282 | Rate: 0.00 | Evals: 62.5%
Iter 7 | Best: 16.792077 | Curr: 98.743678 | Rate: 0.00 | Evals: 62.6%
Iter 8 | Best: 16.792077 | Curr: 104.439999 | Rate: 0.00 | Evals: 62.7%
Iter 9 | Best: 16.792077 | Curr: 41.323490 | Rate: 0.00 | Evals: 62.8%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.41778886 0.05522899 0.2657875  0.77881433 0.99947306 0.77367191
 0.46685151 0.23501268 0.00945252 0.97570051]
  Internal scale: [0.41778886 0.05522899 0.2657875  0.77881433 0.99947306 0.77367191
 0.46685151 0.23501268 0.00945252 0.97570051]
Restart injection: Using best found point so far as starting point (f(x)=16.792077).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 16.792077
Iter 1 | Best: 16.792077 | Curr: 5427.703422 | Rate: 0.00 | Evals: 64.9%
Iter 2 | Best: 16.792077 | Curr: 172.533689 | Rate: 0.00 | Evals: 65.0%
Iter 3 | Best: 16.792077 | Curr: 329.226996 | Rate: 0.00 | Evals: 65.1%
Iter 4 | Best: 16.792077 | Curr: 144.663779 | Rate: 0.00 | Evals: 65.2%
Iter 5 | Best: 16.792077 | Curr: 169.815251 | Rate: 0.00 | Evals: 65.3%
Iter 6 | Best: 16.792077 | Curr: 187.687588 | Rate: 0.00 | Evals: 65.4%
Iter 7 | Best: 16.792077 | Curr: 173.580563 | Rate: 0.00 | Evals: 65.5%
Iter 8 | Best: 16.792077 | Curr: 166.875832 | Rate: 0.00 | Evals: 65.6%
Iter 9 | Best: 16.792077 | Curr: 137.101458 | Rate: 0.00 | Evals: 65.7%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.41778886 0.05522899 0.2657875  0.77881433 0.99947306 0.77367191
 0.46685151 0.23501268 0.00945252 0.97570051]
  Internal scale: [0.41778886 0.05522899 0.2657875  0.77881433 0.99947306 0.77367191
 0.46685151 0.23501268 0.00945252 0.97570051]
Restart injection: Using best found point so far as starting point (f(x)=16.792077).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 16.792077
Iter 1 | Best: 16.792077 | Curr: 1869.597694 | Rate: 0.00 | Evals: 67.8%
Iter 2 | Best: 16.792077 | Curr: 39.876541 | Rate: 0.00 | Evals: 67.9%
Iter 3 | Best: 16.792077 | Curr: 3428.415655 | Rate: 0.00 | Evals: 68.0%
Iter 4 | Best: 16.792077 | Curr: 209.587505 | Rate: 0.00 | Evals: 68.1%
Iter 5 | Best: 16.792077 | Curr: 184.530558 | Rate: 0.00 | Evals: 68.2%
Iter 6 | Best: 16.792077 | Curr: 180.884085 | Rate: 0.00 | Evals: 68.3%
Iter 7 | Best: 16.792077 | Curr: 201.670636 | Rate: 0.00 | Evals: 68.4%
Iter 8 | Best: 16.792077 | Curr: 133.534857 | Rate: 0.00 | Evals: 68.5%
Iter 9 | Best: 16.792077 | Curr: 177.914303 | Rate: 0.00 | Evals: 68.6%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.41778886 0.05522899 0.2657875  0.77881433 0.99947306 0.77367191
 0.46685151 0.23501268 0.00945252 0.97570051]
  Internal scale: [0.41778886 0.05522899 0.2657875  0.77881433 0.99947306 0.77367191
 0.46685151 0.23501268 0.00945252 0.97570051]
Restart injection: Using best found point so far as starting point (f(x)=16.792077).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 16.792077
Iter 1 | Best: 16.792077 | Curr: 1552.891513 | Rate: 0.00 | Evals: 70.7%
Iter 2 | Best: 16.792077 | Curr: 1386.589973 | Rate: 0.00 | Evals: 70.8%
Iter 3 | Best: 16.792077 | Curr: 195.262269 | Rate: 0.00 | Evals: 70.9%
Iter 4 | Best: 16.792077 | Curr: 156.883832 | Rate: 0.00 | Evals: 71.0%
Iter 5 | Best: 16.792077 | Curr: 180.795001 | Rate: 0.00 | Evals: 71.1%
Iter 6 | Best: 16.792077 | Curr: 124.246825 | Rate: 0.00 | Evals: 71.2%
Iter 7 | Best: 16.792077 | Curr: 180.084686 | Rate: 0.00 | Evals: 71.3%
Iter 8 | Best: 16.792077 | Curr: 186.114877 | Rate: 0.00 | Evals: 71.4%
Iter 9 | Best: 16.792077 | Curr: 219.752265 | Rate: 0.00 | Evals: 71.5%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.41778886 0.05522899 0.2657875  0.77881433 0.99947306 0.77367191
 0.46685151 0.23501268 0.00945252 0.97570051]
  Internal scale: [0.41778886 0.05522899 0.2657875  0.77881433 0.99947306 0.77367191
 0.46685151 0.23501268 0.00945252 0.97570051]
Restart injection: Using best found point so far as starting point (f(x)=16.792077).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 11.422734
Iter 1 | Best: 11.422734 | Curr: 1500.236886 | Rate: 0.00 | Evals: 73.6%
Iter 2 | Best: 11.422734 | Curr: 655.283450 | Rate: 0.00 | Evals: 73.7%
Iter 3 | Best: 11.422734 | Curr: 3340.959982 | Rate: 0.00 | Evals: 73.8%
Iter 4 | Best: 11.422734 | Curr: 2632.744904 | Rate: 0.00 | Evals: 73.9%
Iter 5 | Best: 11.422734 | Curr: 188.560534 | Rate: 0.00 | Evals: 74.0%
Iter 6 | Best: 11.422734 | Curr: 3005.213199 | Rate: 0.00 | Evals: 74.1%
Iter 7 | Best: 11.422734 | Curr: 130.932566 | Rate: 0.00 | Evals: 74.2%
Iter 8 | Best: 11.422734 | Curr: 193.948392 | Rate: 0.00 | Evals: 74.3%
Iter 9 | Best: 11.422734 | Curr: 2105.572601 | Rate: 0.00 | Evals: 74.4%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
  Internal scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
Restart injection: Using best found point so far as starting point (f(x)=11.422734).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 11.422734
Iter 1 | Best: 11.422734 | Curr: 177.095952 | Rate: 0.00 | Evals: 76.5%
Iter 2 | Best: 11.422734 | Curr: 583.991827 | Rate: 0.00 | Evals: 76.6%
Iter 3 | Best: 11.422734 | Curr: 268.927574 | Rate: 0.00 | Evals: 76.7%
Iter 4 | Best: 11.422734 | Curr: 177.241856 | Rate: 0.00 | Evals: 76.8%
Iter 5 | Best: 11.422734 | Curr: 171.458336 | Rate: 0.00 | Evals: 76.9%
Iter 6 | Best: 11.422734 | Curr: 125.533759 | Rate: 0.00 | Evals: 77.0%
Iter 7 | Best: 11.422734 | Curr: 215.463762 | Rate: 0.00 | Evals: 77.1%
Iter 8 | Best: 11.422734 | Curr: 1176.157981 | Rate: 0.00 | Evals: 77.2%
Iter 9 | Best: 11.422734 | Curr: 217.227462 | Rate: 0.00 | Evals: 77.3%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
  Internal scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
Restart injection: Using best found point so far as starting point (f(x)=11.422734).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 11.422734
Iter 1 | Best: 11.422734 | Curr: 2593.379155 | Rate: 0.00 | Evals: 79.4%
Iter 2 | Best: 11.422734 | Curr: 1559.639569 | Rate: 0.00 | Evals: 79.5%
Iter 3 | Best: 11.422734 | Curr: 6268.536111 | Rate: 0.00 | Evals: 79.6%
Iter 4 | Best: 5.444802 | Rate: 0.25 | Evals: 79.7%
Iter 5 | Best: 5.444802 | Curr: 1791.813070 | Rate: 0.20 | Evals: 79.8%
Iter 6 | Best: 5.444802 | Curr: 13.860606 | Rate: 0.17 | Evals: 79.9%
Iter 7 | Best: 5.444802 | Curr: 2485.500580 | Rate: 0.14 | Evals: 80.0%
Iter 8 | Best: 5.444802 | Curr: 314.459586 | Rate: 0.12 | Evals: 80.1%
Iter 9 | Best: 5.444802 | Curr: 67.409407 | Rate: 0.11 | Evals: 80.2%
Iter 10 | Best: 5.444802 | Curr: 4256.669247 | Rate: 0.10 | Evals: 80.3%
Iter 11 | Best: 5.444802 | Curr: 2242.084094 | Rate: 0.10 | Evals: 80.4%
Iter 12 | Best: 5.444802 | Curr: 171.273165 | Rate: 0.10 | Evals: 80.5%
Iter 13 | Best: 5.444802 | Curr: 84.743342 | Rate: 0.10 | Evals: 80.6%
Iter 14 | Best: 5.444802 | Curr: 62.037792 | Rate: 0.00 | Evals: 80.7%
Iter 15 | Best: 5.444802 | Curr: 1740.847897 | Rate: 0.00 | Evals: 80.8%
Iter 16 | Best: 5.444802 | Curr: 58.658966 | Rate: 0.00 | Evals: 80.9%
Iter 17 | Best: 5.444802 | Curr: 60.243377 | Rate: 0.00 | Evals: 81.0%
Iter 18 | Best: 5.444802 | Curr: 2524.999238 | Rate: 0.00 | Evals: 81.1%
Iter 19 | Best: 5.444802 | Curr: 177.495115 | Rate: 0.00 | Evals: 81.2%
Iter 20 | Best: 5.444802 | Curr: 6.336354 | Rate: 0.00 | Evals: 81.3%
Iter 21 | Best: 5.214140 | Rate: 0.10 | Evals: 81.4%
Iter 22 | Best: 5.214140 | Curr: 113.325673 | Rate: 0.10 | Evals: 81.5%
Iter 23 | Best: 5.214140 | Curr: 60.642049 | Rate: 0.10 | Evals: 81.6%
Iter 24 | Best: 5.214140 | Curr: 56.766173 | Rate: 0.10 | Evals: 81.7%
Iter 25 | Best: 5.214140 | Curr: 24.913043 | Rate: 0.10 | Evals: 81.8%
Iter 26 | Best: 5.214140 | Curr: 448.867354 | Rate: 0.10 | Evals: 81.9%
Iter 27 | Best: 5.214140 | Curr: 2285.743779 | Rate: 0.10 | Evals: 82.0%
Iter 28 | Best: 5.214140 | Curr: 626.763065 | Rate: 0.10 | Evals: 82.1%
Iter 29 | Best: 5.214140 | Curr: 171.768811 | Rate: 0.10 | Evals: 82.2%
Iter 30 | Best: 5.214140 | Curr: 177.753378 | Rate: 0.10 | Evals: 82.3%
Iter 31 | Best: 5.214140 | Curr: 1561.697306 | Rate: 0.00 | Evals: 82.4%
Iter 32 | Best: 5.214140 | Curr: 26.554547 | Rate: 0.00 | Evals: 82.5%
Iter 33 | Best: 5.214140 | Curr: 648.116999 | Rate: 0.00 | Evals: 82.6%
Iter 34 | Best: 5.214140 | Curr: 155.586677 | Rate: 0.00 | Evals: 82.7%
Iter 35 | Best: 5.214140 | Curr: 2190.528925 | Rate: 0.00 | Evals: 82.8%
Iter 36 | Best: 5.214140 | Curr: 59.627127 | Rate: 0.00 | Evals: 82.9%
Iter 37 | Best: 5.214140 | Curr: 210.719323 | Rate: 0.00 | Evals: 83.0%
Iter 38 | Best: 5.214140 | Curr: 48.048395 | Rate: 0.00 | Evals: 83.1%
Iter 39 | Best: 5.214140 | Curr: 5192.111574 | Rate: 0.00 | Evals: 83.2%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.69600931 0.39895903 0.69026024 0.47315754 0.35587218 0.6098225
 0.39906493 0.48994665 0.43417916 0.41559084]
  Internal scale: [0.69600931 0.39895903 0.69026024 0.47315754 0.35587218 0.6098225
 0.39906493 0.48994665 0.43417916 0.41559084]
Restart injection: Using best found point so far as starting point (f(x)=5.214140).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 5.214140
Iter 1 | Best: 5.214140 | Curr: 2037.408838 | Rate: 0.00 | Evals: 85.3%
Iter 2 | Best: 5.214140 | Curr: 1438.606442 | Rate: 0.00 | Evals: 85.4%
Iter 3 | Best: 5.214140 | Curr: 2910.821931 | Rate: 0.00 | Evals: 85.5%
Iter 4 | Best: 5.214140 | Curr: 2322.350001 | Rate: 0.00 | Evals: 85.6%
Iter 5 | Best: 5.214140 | Curr: 500.895794 | Rate: 0.00 | Evals: 85.7%
Iter 6 | Best: 5.214140 | Curr: 91.860363 | Rate: 0.00 | Evals: 85.8%
Iter 7 | Best: 5.214140 | Curr: 138.050767 | Rate: 0.00 | Evals: 85.9%
Iter 8 | Best: 5.214140 | Curr: 174.346494 | Rate: 0.00 | Evals: 86.0%
Iter 9 | Best: 5.214140 | Curr: 90.633416 | Rate: 0.00 | Evals: 86.1%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.69600931 0.39895903 0.69026024 0.47315754 0.35587218 0.6098225
 0.39906493 0.48994665 0.43417916 0.41559084]
  Internal scale: [0.69600931 0.39895903 0.69026024 0.47315754 0.35587218 0.6098225
 0.39906493 0.48994665 0.43417916 0.41559084]
Restart injection: Using best found point so far as starting point (f(x)=5.214140).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 5.214140
Iter 1 | Best: 5.214140 | Curr: 84.198299 | Rate: 0.00 | Evals: 88.2%
Iter 2 | Best: 5.214140 | Curr: 2705.855775 | Rate: 0.00 | Evals: 88.3%
Iter 3 | Best: 5.214140 | Curr: 64.272932 | Rate: 0.00 | Evals: 88.4%
Iter 4 | Best: 5.214140 | Curr: 176.140250 | Rate: 0.00 | Evals: 88.5%
Iter 5 | Best: 5.214140 | Curr: 499.343910 | Rate: 0.00 | Evals: 88.6%
Iter 6 | Best: 5.214140 | Curr: 135.796354 | Rate: 0.00 | Evals: 88.7%
Iter 7 | Best: 5.214140 | Curr: 826.284474 | Rate: 0.00 | Evals: 88.8%
Iter 8 | Best: 5.214140 | Curr: 78.456030 | Rate: 0.00 | Evals: 88.9%
Iter 9 | Best: 5.214140 | Curr: 143.912906 | Rate: 0.00 | Evals: 89.0%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.69600931 0.39895903 0.69026024 0.47315754 0.35587218 0.6098225
 0.39906493 0.48994665 0.43417916 0.41559084]
  Internal scale: [0.69600931 0.39895903 0.69026024 0.47315754 0.35587218 0.6098225
 0.39906493 0.48994665 0.43417916 0.41559084]
Restart injection: Using best found point so far as starting point (f(x)=5.214140).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 5.214140
Iter 1 | Best: 5.214140 | Curr: 1278.283975 | Rate: 0.00 | Evals: 91.1%
Iter 2 | Best: 5.214140 | Curr: 59.338086 | Rate: 0.00 | Evals: 91.2%
Iter 3 | Best: 5.214140 | Curr: 2120.662187 | Rate: 0.00 | Evals: 91.3%
Iter 4 | Best: 5.214140 | Curr: 12.437767 | Rate: 0.00 | Evals: 91.4%
Iter 5 | Best: 5.214140 | Curr: 509.947420 | Rate: 0.00 | Evals: 91.5%
Iter 6 | Best: 5.214140 | Curr: 84.756684 | Rate: 0.00 | Evals: 91.6%
Iter 7 | Best: 5.214140 | Curr: 98.834653 | Rate: 0.00 | Evals: 91.7%
Iter 8 | Best: 5.214140 | Curr: 160.282289 | Rate: 0.00 | Evals: 91.8%
Iter 9 | Best: 5.214140 | Curr: 125.113800 | Rate: 0.00 | Evals: 91.9%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.69600931 0.39895903 0.69026024 0.47315754 0.35587218 0.6098225
 0.39906493 0.48994665 0.43417916 0.41559084]
  Internal scale: [0.69600931 0.39895903 0.69026024 0.47315754 0.35587218 0.6098225
 0.39906493 0.48994665 0.43417916 0.41559084]
Restart injection: Using best found point so far as starting point (f(x)=5.214140).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 5.214140
Iter 1 | Best: 5.214140 | Curr: 10637.143430 | Rate: 0.00 | Evals: 94.0%
Iter 2 | Best: 5.214140 | Curr: 55.254505 | Rate: 0.00 | Evals: 94.1%
Iter 3 | Best: 5.214140 | Curr: 84.411980 | Rate: 0.00 | Evals: 94.2%
Iter 4 | Best: 5.214140 | Curr: 89.327270 | Rate: 0.00 | Evals: 94.3%
Iter 5 | Best: 5.214140 | Curr: 69.603754 | Rate: 0.00 | Evals: 94.4%
Iter 6 | Best: 5.214140 | Curr: 43.025917 | Rate: 0.00 | Evals: 94.5%
Iter 7 | Best: 5.214140 | Curr: 1564.301510 | Rate: 0.00 | Evals: 94.6%
Iter 8 | Best: 5.214140 | Curr: 122.902919 | Rate: 0.00 | Evals: 94.7%
Iter 9 | Best: 5.214140 | Curr: 99.123242 | Rate: 0.00 | Evals: 94.8%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.69600931 0.39895903 0.69026024 0.47315754 0.35587218 0.6098225
 0.39906493 0.48994665 0.43417916 0.41559084]
  Internal scale: [0.69600931 0.39895903 0.69026024 0.47315754 0.35587218 0.6098225
 0.39906493 0.48994665 0.43417916 0.41559084]
Restart injection: Using best found point so far as starting point (f(x)=5.214140).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 5.214140
Iter 1 | Best: 5.214140 | Curr: 4080.027133 | Rate: 0.00 | Evals: 96.9%
Iter 2 | Best: 5.214140 | Curr: 2040.566809 | Rate: 0.00 | Evals: 97.0%
Iter 3 | Best: 5.214140 | Curr: 1171.211856 | Rate: 0.00 | Evals: 97.1%
Iter 4 | Best: 5.214140 | Curr: 3588.827613 | Rate: 0.00 | Evals: 97.2%
Iter 5 | Best: 5.214140 | Curr: 524.735990 | Rate: 0.00 | Evals: 97.3%
Iter 6 | Best: 5.214140 | Curr: 88.706866 | Rate: 0.00 | Evals: 97.4%
Iter 7 | Best: 5.214140 | Curr: 54.531182 | Rate: 0.00 | Evals: 97.5%
Iter 8 | Best: 5.214140 | Curr: 480.919750 | Rate: 0.00 | Evals: 97.6%
Iter 9 | Best: 5.214140 | Curr: 86.324934 | Rate: 0.00 | Evals: 97.7%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.69600931 0.39895903 0.69026024 0.47315754 0.35587218 0.6098225
 0.39906493 0.48994665 0.43417916 0.41559084]
  Internal scale: [0.69600931 0.39895903 0.69026024 0.47315754 0.35587218 0.6098225
 0.39906493 0.48994665 0.43417916 0.41559084]
Restart injection: Using best found point so far as starting point (f(x)=5.214140).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 5.214140
Iter 1 | Best: 5.214140 | Curr: 1624.163699 | Rate: 0.00 | Evals: 99.8%
Iter 2 | Best: 5.214140 | Curr: 3357.479897 | Rate: 0.00 | Evals: 99.9%
Iter 3 | Best: 5.214140 | Curr: 198.520612 | Rate: 0.00 | Evals: 100.0%
   Best y (Multi-Surrogate): 5.214140
2. Running Default SpotOptim (GP only)...
TensorBoard logging disabled
Initial best: f(x) = 27.216160
Iter 1 | Best: 27.216160 | Curr: 408.060597 | Rate: 0.00 | Evals: 2.1%
Iter 2 | Best: 27.216160 | Curr: 1042.477829 | Rate: 0.00 | Evals: 2.2%
Iter 3 | Best: 27.216160 | Curr: 431.862591 | Rate: 0.00 | Evals: 2.3%
Iter 4 | Best: 27.216160 | Curr: 1869.165545 | Rate: 0.00 | Evals: 2.4%
Iter 5 | Best: 27.216160 | Curr: 12827.853095 | Rate: 0.00 | Evals: 2.5%
Iter 6 | Best: 27.216160 | Curr: 879.074727 | Rate: 0.00 | Evals: 2.6%
Iter 7 | Best: 27.216160 | Curr: 56.278018 | Rate: 0.00 | Evals: 2.7%
Iter 8 | Best: 27.216160 | Curr: 683.584693 | Rate: 0.00 | Evals: 2.8%
Iter 9 | Best: 27.216160 | Curr: 702.443729 | Rate: 0.00 | Evals: 2.9%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.38927077 0.97957357 0.90732985 0.58830303 0.64708486 0.08593081
 0.93532031 0.61690417 0.87214839 0.01080509]
  Internal scale: [0.38927077 0.97957357 0.90732985 0.58830303 0.64708486 0.08593081
 0.93532031 0.61690417 0.87214839 0.01080509]
Restart injection: Using best found point so far as starting point (f(x)=27.216160).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 22.241614
Iter 1 | Best: 22.241614 | Curr: 4626.676233 | Rate: 0.00 | Evals: 5.0%
Iter 2 | Best: 22.241614 | Curr: 3058.223569 | Rate: 0.00 | Evals: 5.1%
Iter 3 | Best: 22.241614 | Curr: 821.578740 | Rate: 0.00 | Evals: 5.2%
Iter 4 | Best: 22.241614 | Curr: 51.633659 | Rate: 0.00 | Evals: 5.3%
Iter 5 | Best: 22.241614 | Curr: 200.751830 | Rate: 0.00 | Evals: 5.4%
Iter 6 | Best: 22.241614 | Curr: 4537.718842 | Rate: 0.00 | Evals: 5.5%
Iter 7 | Best: 22.241614 | Curr: 52.890180 | Rate: 0.00 | Evals: 5.6%
Iter 8 | Best: 22.241614 | Curr: 2916.812364 | Rate: 0.00 | Evals: 5.7%
Iter 9 | Best: 22.241614 | Curr: 59.397778 | Rate: 0.00 | Evals: 5.8%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.29261083 0.85319363 0.8781048  0.33083401 0.21351571 0.67235035
 0.253193   0.76098493 0.42603152 0.23118203]
  Internal scale: [0.29261083 0.85319363 0.8781048  0.33083401 0.21351571 0.67235035
 0.253193   0.76098493 0.42603152 0.23118203]
Restart injection: Using best found point so far as starting point (f(x)=22.241614).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 18.210089
Iter 1 | Best: 18.210089 | Curr: 1870.808970 | Rate: 0.00 | Evals: 7.9%
Iter 2 | Best: 18.210089 | Curr: 1367.014173 | Rate: 0.00 | Evals: 8.0%
Iter 3 | Best: 18.210089 | Curr: 47.617095 | Rate: 0.00 | Evals: 8.1%
Iter 4 | Best: 18.210089 | Curr: 3226.249893 | Rate: 0.00 | Evals: 8.2%
Iter 5 | Best: 18.210089 | Curr: 112.627242 | Rate: 0.00 | Evals: 8.3%
Iter 6 | Best: 18.210089 | Curr: 493.722865 | Rate: 0.00 | Evals: 8.4%
Iter 7 | Best: 18.210089 | Curr: 92.687506 | Rate: 0.00 | Evals: 8.5%
Iter 8 | Best: 18.210089 | Curr: 131.036583 | Rate: 0.00 | Evals: 8.6%
Iter 9 | Best: 18.210089 | Curr: 147.448492 | Rate: 0.00 | Evals: 8.7%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
  Internal scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
Restart injection: Using best found point so far as starting point (f(x)=18.210089).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 18.210089
Iter 1 | Best: 18.210089 | Curr: 98.131070 | Rate: 0.00 | Evals: 10.8%
Iter 2 | Best: 18.210089 | Curr: 23.029428 | Rate: 0.00 | Evals: 10.9%
Iter 3 | Best: 18.210089 | Curr: 80.937481 | Rate: 0.00 | Evals: 11.0%
Iter 4 | Best: 18.210089 | Curr: 2746.579822 | Rate: 0.00 | Evals: 11.1%
Iter 5 | Best: 18.210089 | Curr: 77.463699 | Rate: 0.00 | Evals: 11.2%
Iter 6 | Best: 18.210089 | Curr: 86.118184 | Rate: 0.00 | Evals: 11.3%
Iter 7 | Best: 18.210089 | Curr: 1033.895849 | Rate: 0.00 | Evals: 11.4%
Iter 8 | Best: 18.210089 | Curr: 89.445722 | Rate: 0.00 | Evals: 11.5%
Iter 9 | Best: 18.210089 | Curr: 22.577329 | Rate: 0.00 | Evals: 11.6%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
  Internal scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
Restart injection: Using best found point so far as starting point (f(x)=18.210089).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 18.210089
Iter 1 | Best: 18.210089 | Curr: 2614.367087 | Rate: 0.00 | Evals: 13.7%
Iter 2 | Best: 18.210089 | Curr: 1809.860465 | Rate: 0.00 | Evals: 13.8%
Iter 3 | Best: 18.210089 | Curr: 1490.525251 | Rate: 0.00 | Evals: 13.9%
Iter 4 | Best: 18.210089 | Curr: 1039.400159 | Rate: 0.00 | Evals: 14.0%
Iter 5 | Best: 18.210089 | Curr: 2215.479491 | Rate: 0.00 | Evals: 14.1%
Iter 6 | Best: 18.210089 | Curr: 1277.049359 | Rate: 0.00 | Evals: 14.2%
Iter 7 | Best: 18.210089 | Curr: 2256.404364 | Rate: 0.00 | Evals: 14.3%
Iter 8 | Best: 18.210089 | Curr: 863.585774 | Rate: 0.00 | Evals: 14.4%
Iter 9 | Best: 18.210089 | Curr: 6231.041989 | Rate: 0.00 | Evals: 14.5%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
  Internal scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
Restart injection: Using best found point so far as starting point (f(x)=18.210089).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 18.210089
Iter 1 | Best: 18.210089 | Curr: 3694.654533 | Rate: 0.00 | Evals: 16.6%
Iter 2 | Best: 18.210089 | Curr: 181.785698 | Rate: 0.00 | Evals: 16.7%
Iter 3 | Best: 18.210089 | Curr: 2096.590482 | Rate: 0.00 | Evals: 16.8%
Iter 4 | Best: 18.210089 | Curr: 217.296035 | Rate: 0.00 | Evals: 16.9%
Iter 5 | Best: 18.210089 | Curr: 167.652004 | Rate: 0.00 | Evals: 17.0%
Iter 6 | Best: 18.210089 | Curr: 182.780137 | Rate: 0.00 | Evals: 17.1%
Iter 7 | Best: 18.210089 | Curr: 39.412108 | Rate: 0.00 | Evals: 17.2%
Iter 8 | Best: 18.210089 | Curr: 20.128827 | Rate: 0.00 | Evals: 17.3%
Iter 9 | Best: 18.210089 | Curr: 181.065076 | Rate: 0.00 | Evals: 17.4%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
  Internal scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
Restart injection: Using best found point so far as starting point (f(x)=18.210089).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 18.210089
Optimizer candidate 1/3 was duplicate/invalid.
Iter 1 | Best: 18.210089 | Curr: 65.740721 | Rate: 0.00 | Evals: 19.5%
Iter 2 | Best: 18.210089 | Curr: 2008.951594 | Rate: 0.00 | Evals: 19.6%
Iter 3 | Best: 18.210089 | Curr: 679.687477 | Rate: 0.00 | Evals: 19.7%
Iter 4 | Best: 18.210089 | Curr: 5545.238192 | Rate: 0.00 | Evals: 19.8%
Iter 5 | Best: 18.210089 | Curr: 865.498029 | Rate: 0.00 | Evals: 19.9%
Iter 6 | Best: 18.210089 | Curr: 1134.133560 | Rate: 0.00 | Evals: 20.0%
Iter 7 | Best: 18.210089 | Curr: 194.762833 | Rate: 0.00 | Evals: 20.1%
Iter 8 | Best: 18.210089 | Curr: 165.558372 | Rate: 0.00 | Evals: 20.2%
Iter 9 | Best: 18.210089 | Curr: 152.829163 | Rate: 0.00 | Evals: 20.3%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
  Internal scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
Restart injection: Using best found point so far as starting point (f(x)=18.210089).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 18.210089
Iter 1 | Best: 18.210089 | Curr: 641.725396 | Rate: 0.00 | Evals: 22.4%
Iter 2 | Best: 18.210089 | Curr: 2927.430083 | Rate: 0.00 | Evals: 22.5%
Iter 3 | Best: 18.210089 | Curr: 119.505325 | Rate: 0.00 | Evals: 22.6%
Iter 4 | Best: 18.210089 | Curr: 1689.660352 | Rate: 0.00 | Evals: 22.7%
Iter 5 | Best: 18.210089 | Curr: 174.381782 | Rate: 0.00 | Evals: 22.8%
Iter 6 | Best: 18.210089 | Curr: 214.928398 | Rate: 0.00 | Evals: 22.9%
Iter 7 | Best: 18.210089 | Curr: 1444.160990 | Rate: 0.00 | Evals: 23.0%
Iter 8 | Best: 18.210089 | Curr: 4666.578165 | Rate: 0.00 | Evals: 23.1%
Iter 9 | Best: 18.210089 | Curr: 64.003951 | Rate: 0.00 | Evals: 23.2%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
  Internal scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
Restart injection: Using best found point so far as starting point (f(x)=18.210089).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 18.210089
Iter 1 | Best: 18.210089 | Curr: 3415.930176 | Rate: 0.00 | Evals: 25.3%
Iter 2 | Best: 18.210089 | Curr: 1726.842937 | Rate: 0.00 | Evals: 25.4%
Iter 3 | Best: 18.210089 | Curr: 94.873237 | Rate: 0.00 | Evals: 25.5%
Iter 4 | Best: 18.210089 | Curr: 4767.771522 | Rate: 0.00 | Evals: 25.6%
Iter 5 | Best: 18.210089 | Curr: 1851.997977 | Rate: 0.00 | Evals: 25.7%
Iter 6 | Best: 18.210089 | Curr: 758.239958 | Rate: 0.00 | Evals: 25.8%
Iter 7 | Best: 18.210089 | Curr: 1949.829032 | Rate: 0.00 | Evals: 25.9%
Iter 8 | Best: 18.210089 | Curr: 121.287622 | Rate: 0.00 | Evals: 26.0%
Iter 9 | Best: 18.210089 | Curr: 119.364893 | Rate: 0.00 | Evals: 26.1%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
  Internal scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
Restart injection: Using best found point so far as starting point (f(x)=18.210089).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 18.210089
Optimizer candidate 1/3 was duplicate/invalid.
Iter 1 | Best: 18.210089 | Curr: 84.549833 | Rate: 0.00 | Evals: 28.2%
Iter 2 | Best: 18.210089 | Curr: 245.207511 | Rate: 0.00 | Evals: 28.3%
Iter 3 | Best: 18.210089 | Curr: 211.655809 | Rate: 0.00 | Evals: 28.4%
Iter 4 | Best: 18.210089 | Curr: 231.886437 | Rate: 0.00 | Evals: 28.5%
Iter 5 | Best: 18.210089 | Curr: 188.612624 | Rate: 0.00 | Evals: 28.6%
Iter 6 | Best: 18.210089 | Curr: 207.123132 | Rate: 0.00 | Evals: 28.7%
Iter 7 | Best: 18.210089 | Curr: 771.960865 | Rate: 0.00 | Evals: 28.8%
Iter 8 | Best: 18.210089 | Curr: 257.226874 | Rate: 0.00 | Evals: 28.9%
Iter 9 | Best: 18.210089 | Curr: 191.239343 | Rate: 0.00 | Evals: 29.0%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
  Internal scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
Restart injection: Using best found point so far as starting point (f(x)=18.210089).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 18.210089
Iter 1 | Best: 18.210089 | Curr: 30.931982 | Rate: 0.00 | Evals: 31.1%
Iter 2 | Best: 18.210089 | Curr: 273.831589 | Rate: 0.00 | Evals: 31.2%
Iter 3 | Best: 18.210089 | Curr: 7803.734732 | Rate: 0.00 | Evals: 31.3%
Iter 4 | Best: 18.210089 | Curr: 135.435983 | Rate: 0.00 | Evals: 31.4%
Iter 5 | Best: 18.210089 | Curr: 127.459850 | Rate: 0.00 | Evals: 31.5%
Iter 6 | Best: 18.210089 | Curr: 122.209034 | Rate: 0.00 | Evals: 31.6%
Iter 7 | Best: 18.210089 | Curr: 68.917181 | Rate: 0.00 | Evals: 31.7%
Iter 8 | Best: 18.210089 | Curr: 126.549066 | Rate: 0.00 | Evals: 31.8%
Iter 9 | Best: 18.210089 | Curr: 163.458341 | Rate: 0.00 | Evals: 31.9%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
  Internal scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
Restart injection: Using best found point so far as starting point (f(x)=18.210089).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 18.210089
Iter 1 | Best: 18.210089 | Curr: 2218.862609 | Rate: 0.00 | Evals: 34.0%
Iter 2 | Best: 18.210089 | Curr: 13354.757455 | Rate: 0.00 | Evals: 34.1%
Iter 3 | Best: 18.210089 | Curr: 5189.990049 | Rate: 0.00 | Evals: 34.2%
Iter 4 | Best: 18.210089 | Curr: 9352.833678 | Rate: 0.00 | Evals: 34.3%
Optimizer candidate 1/3 was duplicate/invalid.
Iter 5 | Best: 18.210089 | Curr: 78.310674 | Rate: 0.00 | Evals: 34.4%
Iter 6 | Best: 18.210089 | Curr: 186.336305 | Rate: 0.00 | Evals: 34.5%
Iter 7 | Best: 18.210089 | Curr: 196.295692 | Rate: 0.00 | Evals: 34.6%
Iter 8 | Best: 18.210089 | Curr: 196.340971 | Rate: 0.00 | Evals: 34.7%
Iter 9 | Best: 18.210089 | Curr: 1272.820268 | Rate: 0.00 | Evals: 34.8%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
  Internal scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
Restart injection: Using best found point so far as starting point (f(x)=18.210089).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 18.210089
Iter 1 | Best: 18.210089 | Curr: 1530.532644 | Rate: 0.00 | Evals: 36.9%
Iter 2 | Best: 18.210089 | Curr: 3512.935017 | Rate: 0.00 | Evals: 37.0%
Iter 3 | Best: 18.210089 | Curr: 2932.850771 | Rate: 0.00 | Evals: 37.1%
Iter 4 | Best: 18.210089 | Curr: 62.385369 | Rate: 0.00 | Evals: 37.2%
Iter 5 | Best: 18.210089 | Curr: 203.629690 | Rate: 0.00 | Evals: 37.3%
Iter 6 | Best: 18.210089 | Curr: 174.510173 | Rate: 0.00 | Evals: 37.4%
Iter 7 | Best: 18.210089 | Curr: 211.799516 | Rate: 0.00 | Evals: 37.5%
Iter 8 | Best: 18.210089 | Curr: 170.289685 | Rate: 0.00 | Evals: 37.6%
Iter 9 | Best: 18.210089 | Curr: 213.086122 | Rate: 0.00 | Evals: 37.7%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
  Internal scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
Restart injection: Using best found point so far as starting point (f(x)=18.210089).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 18.210089
Iter 1 | Best: 18.210089 | Curr: 5265.480348 | Rate: 0.00 | Evals: 39.8%
Iter 2 | Best: 18.210089 | Curr: 3683.907485 | Rate: 0.00 | Evals: 39.9%
Iter 3 | Best: 18.210089 | Curr: 843.929550 | Rate: 0.00 | Evals: 40.0%
Iter 4 | Best: 18.210089 | Curr: 3166.819635 | Rate: 0.00 | Evals: 40.1%
Iter 5 | Best: 18.210089 | Curr: 996.482504 | Rate: 0.00 | Evals: 40.2%
Iter 6 | Best: 18.210089 | Curr: 3755.618018 | Rate: 0.00 | Evals: 40.3%
Iter 7 | Best: 18.210089 | Curr: 6809.464992 | Rate: 0.00 | Evals: 40.4%
Iter 8 | Best: 18.210089 | Curr: 5945.011358 | Rate: 0.00 | Evals: 40.5%
Iter 9 | Best: 18.210089 | Curr: 443.750710 | Rate: 0.00 | Evals: 40.6%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
  Internal scale: [0.49074103 0.82189146 0.24490529 0.01735389 0.20223253 0.5743634
 0.62835138 0.34820786 0.50201127 0.54484987]
Restart injection: Using best found point so far as starting point (f(x)=18.210089).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.424393
Iter 1 | Best: 17.424393 | Curr: 78.785446 | Rate: 0.00 | Evals: 42.7%
Iter 2 | Best: 17.424393 | Curr: 2930.293780 | Rate: 0.00 | Evals: 42.8%
Iter 3 | Best: 17.424393 | Curr: 1019.601658 | Rate: 0.00 | Evals: 42.9%
Iter 4 | Best: 17.424393 | Curr: 1159.359392 | Rate: 0.00 | Evals: 43.0%
Iter 5 | Best: 17.424393 | Curr: 1175.212214 | Rate: 0.00 | Evals: 43.1%
Iter 6 | Best: 17.424393 | Curr: 17.765027 | Rate: 0.00 | Evals: 43.2%
Iter 7 | Best: 17.424393 | Curr: 81.888076 | Rate: 0.00 | Evals: 43.3%
Iter 8 | Best: 17.424393 | Curr: 103.684628 | Rate: 0.00 | Evals: 43.4%
Iter 9 | Best: 17.424393 | Curr: 81.205872 | Rate: 0.00 | Evals: 43.5%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
  Internal scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
Restart injection: Using best found point so far as starting point (f(x)=17.424393).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.424393
Iter 1 | Best: 17.424393 | Curr: 148.157196 | Rate: 0.00 | Evals: 45.6%
Iter 2 | Best: 17.424393 | Curr: 180.290280 | Rate: 0.00 | Evals: 45.7%
Iter 3 | Best: 17.424393 | Curr: 147.786236 | Rate: 0.00 | Evals: 45.8%
Iter 4 | Best: 17.424393 | Curr: 114.433238 | Rate: 0.00 | Evals: 45.9%
Iter 5 | Best: 17.424393 | Curr: 185.316541 | Rate: 0.00 | Evals: 46.0%
Iter 6 | Best: 17.424393 | Curr: 189.585739 | Rate: 0.00 | Evals: 46.1%
Iter 7 | Best: 17.424393 | Curr: 89.152317 | Rate: 0.00 | Evals: 46.2%
Iter 8 | Best: 17.424393 | Curr: 141.686163 | Rate: 0.00 | Evals: 46.3%
Iter 9 | Best: 17.424393 | Curr: 70.744411 | Rate: 0.00 | Evals: 46.4%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
  Internal scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
Restart injection: Using best found point so far as starting point (f(x)=17.424393).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.424393
Iter 1 | Best: 17.424393 | Curr: 1851.369759 | Rate: 0.00 | Evals: 48.5%
Iter 2 | Best: 17.424393 | Curr: 233.917240 | Rate: 0.00 | Evals: 48.6%
Iter 3 | Best: 17.424393 | Curr: 88.460575 | Rate: 0.00 | Evals: 48.7%
Iter 4 | Best: 17.424393 | Curr: 4327.986835 | Rate: 0.00 | Evals: 48.8%
Iter 5 | Best: 17.424393 | Curr: 70.918162 | Rate: 0.00 | Evals: 48.9%
Iter 6 | Best: 17.424393 | Curr: 4377.381221 | Rate: 0.00 | Evals: 49.0%
Iter 7 | Best: 17.424393 | Curr: 91.492553 | Rate: 0.00 | Evals: 49.1%
Iter 8 | Best: 17.424393 | Curr: 121.274804 | Rate: 0.00 | Evals: 49.2%
Iter 9 | Best: 17.424393 | Curr: 177.409323 | Rate: 0.00 | Evals: 49.3%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
  Internal scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
Restart injection: Using best found point so far as starting point (f(x)=17.424393).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.424393
Iter 1 | Best: 17.424393 | Curr: 2527.986745 | Rate: 0.00 | Evals: 51.4%
Iter 2 | Best: 17.424393 | Curr: 78.447275 | Rate: 0.00 | Evals: 51.5%
Iter 3 | Best: 17.424393 | Curr: 4397.539129 | Rate: 0.00 | Evals: 51.6%
Iter 4 | Best: 17.424393 | Curr: 83.374489 | Rate: 0.00 | Evals: 51.7%
Iter 5 | Best: 17.424393 | Curr: 225.119400 | Rate: 0.00 | Evals: 51.8%
Iter 6 | Best: 17.424393 | Curr: 53.554946 | Rate: 0.00 | Evals: 51.9%
Iter 7 | Best: 17.424393 | Curr: 87.430760 | Rate: 0.00 | Evals: 52.0%
Iter 8 | Best: 17.424393 | Curr: 152.423328 | Rate: 0.00 | Evals: 52.1%
Iter 9 | Best: 17.424393 | Curr: 205.446899 | Rate: 0.00 | Evals: 52.2%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
  Internal scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
Restart injection: Using best found point so far as starting point (f(x)=17.424393).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.424393
Iter 1 | Best: 17.424393 | Curr: 1017.321156 | Rate: 0.00 | Evals: 54.3%
Iter 2 | Best: 17.424393 | Curr: 2316.631753 | Rate: 0.00 | Evals: 54.4%
Iter 3 | Best: 17.424393 | Curr: 30.817371 | Rate: 0.00 | Evals: 54.5%
Iter 4 | Best: 17.424393 | Curr: 650.059611 | Rate: 0.00 | Evals: 54.6%
Iter 5 | Best: 17.424393 | Curr: 130.071133 | Rate: 0.00 | Evals: 54.7%
Iter 6 | Best: 17.424393 | Curr: 119.757840 | Rate: 0.00 | Evals: 54.8%
Iter 7 | Best: 17.424393 | Curr: 74.584553 | Rate: 0.00 | Evals: 54.9%
Iter 8 | Best: 17.424393 | Curr: 185.421724 | Rate: 0.00 | Evals: 55.0%
Iter 9 | Best: 17.424393 | Curr: 166.941000 | Rate: 0.00 | Evals: 55.1%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
  Internal scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
Restart injection: Using best found point so far as starting point (f(x)=17.424393).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.424393
Iter 1 | Best: 17.424393 | Curr: 2925.813924 | Rate: 0.00 | Evals: 57.2%
Iter 2 | Best: 17.424393 | Curr: 3232.953907 | Rate: 0.00 | Evals: 57.3%
Iter 3 | Best: 17.424393 | Curr: 106.452703 | Rate: 0.00 | Evals: 57.4%
Iter 4 | Best: 17.424393 | Curr: 1211.042583 | Rate: 0.00 | Evals: 57.5%
Iter 5 | Best: 17.424393 | Curr: 129.039181 | Rate: 0.00 | Evals: 57.6%
Iter 6 | Best: 17.424393 | Curr: 88.432594 | Rate: 0.00 | Evals: 57.7%
Iter 7 | Best: 17.424393 | Curr: 44.128935 | Rate: 0.00 | Evals: 57.8%
Iter 8 | Best: 17.424393 | Curr: 116.463344 | Rate: 0.00 | Evals: 57.9%
Iter 9 | Best: 17.424393 | Curr: 4870.056662 | Rate: 0.00 | Evals: 58.0%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
  Internal scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
Restart injection: Using best found point so far as starting point (f(x)=17.424393).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.424393
Iter 1 | Best: 17.424393 | Curr: 2010.663152 | Rate: 0.00 | Evals: 60.1%
Iter 2 | Best: 17.424393 | Curr: 2828.062557 | Rate: 0.00 | Evals: 60.2%
Iter 3 | Best: 17.424393 | Curr: 3287.546808 | Rate: 0.00 | Evals: 60.3%
Iter 4 | Best: 17.424393 | Curr: 2070.715758 | Rate: 0.00 | Evals: 60.4%
Iter 5 | Best: 17.424393 | Curr: 2469.811390 | Rate: 0.00 | Evals: 60.5%
Iter 6 | Best: 17.424393 | Curr: 165.546839 | Rate: 0.00 | Evals: 60.6%
Iter 7 | Best: 17.424393 | Curr: 172.109997 | Rate: 0.00 | Evals: 60.7%
Iter 8 | Best: 17.424393 | Curr: 122.168325 | Rate: 0.00 | Evals: 60.8%
Iter 9 | Best: 17.424393 | Curr: 155.443186 | Rate: 0.00 | Evals: 60.9%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
  Internal scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
Restart injection: Using best found point so far as starting point (f(x)=17.424393).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.424393
Iter 1 | Best: 17.424393 | Curr: 1554.738173 | Rate: 0.00 | Evals: 63.0%
Iter 2 | Best: 17.424393 | Curr: 3033.933273 | Rate: 0.00 | Evals: 63.1%
Iter 3 | Best: 17.424393 | Curr: 970.676715 | Rate: 0.00 | Evals: 63.2%
Iter 4 | Best: 17.424393 | Curr: 21.837472 | Rate: 0.00 | Evals: 63.3%
Iter 5 | Best: 17.424393 | Curr: 2147.622138 | Rate: 0.00 | Evals: 63.4%
Iter 6 | Best: 17.424393 | Curr: 1823.164138 | Rate: 0.00 | Evals: 63.5%
Iter 7 | Best: 17.424393 | Curr: 3346.891730 | Rate: 0.00 | Evals: 63.6%
Iter 8 | Best: 17.424393 | Curr: 472.233405 | Rate: 0.00 | Evals: 63.7%
Iter 9 | Best: 17.424393 | Curr: 911.874064 | Rate: 0.00 | Evals: 63.8%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
  Internal scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
Restart injection: Using best found point so far as starting point (f(x)=17.424393).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 17.424393
Iter 1 | Best: 17.424393 | Curr: 1156.778000 | Rate: 0.00 | Evals: 65.9%
Iter 2 | Best: 17.424393 | Curr: 2268.879869 | Rate: 0.00 | Evals: 66.0%
Iter 3 | Best: 17.424393 | Curr: 1763.020002 | Rate: 0.00 | Evals: 66.1%
Iter 4 | Best: 17.424393 | Curr: 725.479527 | Rate: 0.00 | Evals: 66.2%
Iter 5 | Best: 17.424393 | Curr: 3193.751580 | Rate: 0.00 | Evals: 66.3%
Iter 6 | Best: 17.424393 | Curr: 3946.804199 | Rate: 0.00 | Evals: 66.4%
Iter 7 | Best: 17.424393 | Curr: 63.005761 | Rate: 0.00 | Evals: 66.5%
Iter 8 | Best: 17.424393 | Curr: 2797.214621 | Rate: 0.00 | Evals: 66.6%
Iter 9 | Best: 17.424393 | Curr: 1056.292872 | Rate: 0.00 | Evals: 66.7%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
  Internal scale: [0.42481218 0.05390178 0.2746338  0.77807965 0.98637557 0.76705916
 0.47151775 0.23928273 0.01452935 0.96779566]
Restart injection: Using best found point so far as starting point (f(x)=17.424393).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 11.422734
Iter 1 | Best: 11.422734 | Curr: 2047.473473 | Rate: 0.00 | Evals: 68.8%
Iter 2 | Best: 11.422734 | Curr: 682.484050 | Rate: 0.00 | Evals: 68.9%
Iter 3 | Best: 11.422734 | Curr: 562.327116 | Rate: 0.00 | Evals: 69.0%
Iter 4 | Best: 11.422734 | Curr: 827.163578 | Rate: 0.00 | Evals: 69.1%
Iter 5 | Best: 11.422734 | Curr: 49.203668 | Rate: 0.00 | Evals: 69.2%
Iter 6 | Best: 11.422734 | Curr: 1389.062130 | Rate: 0.00 | Evals: 69.3%
Iter 7 | Best: 11.422734 | Curr: 1711.447974 | Rate: 0.00 | Evals: 69.4%
Iter 8 | Best: 11.422734 | Curr: 1574.062919 | Rate: 0.00 | Evals: 69.5%
Iter 9 | Best: 11.422734 | Curr: 1110.807632 | Rate: 0.00 | Evals: 69.6%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
  Internal scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
Restart injection: Using best found point so far as starting point (f(x)=11.422734).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 11.422734
Iter 1 | Best: 11.422734 | Curr: 177.095952 | Rate: 0.00 | Evals: 71.7%
Iter 2 | Best: 11.422734 | Curr: 135.699364 | Rate: 0.00 | Evals: 71.8%
Iter 3 | Best: 11.422734 | Curr: 215.355856 | Rate: 0.00 | Evals: 71.9%
Iter 4 | Best: 11.422734 | Curr: 533.443518 | Rate: 0.00 | Evals: 72.0%
Iter 5 | Best: 11.422734 | Curr: 207.264416 | Rate: 0.00 | Evals: 72.1%
Iter 6 | Best: 11.422734 | Curr: 241.616412 | Rate: 0.00 | Evals: 72.2%
Iter 7 | Best: 11.422734 | Curr: 212.080824 | Rate: 0.00 | Evals: 72.3%
Iter 8 | Best: 11.422734 | Curr: 164.827853 | Rate: 0.00 | Evals: 72.4%
Iter 9 | Best: 11.422734 | Curr: 204.328602 | Rate: 0.00 | Evals: 72.5%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
  Internal scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
Restart injection: Using best found point so far as starting point (f(x)=11.422734).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 11.422734
Iter 1 | Best: 11.422734 | Curr: 2593.379155 | Rate: 0.00 | Evals: 74.6%
Iter 2 | Best: 11.422734 | Curr: 2203.258356 | Rate: 0.00 | Evals: 74.7%
Iter 3 | Best: 11.422734 | Curr: 1660.063430 | Rate: 0.00 | Evals: 74.8%
Iter 4 | Best: 11.422734 | Curr: 5427.673188 | Rate: 0.00 | Evals: 74.9%
Iter 5 | Best: 11.422734 | Curr: 3681.819557 | Rate: 0.00 | Evals: 75.0%
Iter 6 | Best: 11.422734 | Curr: 4649.081598 | Rate: 0.00 | Evals: 75.1%
Iter 7 | Best: 11.422734 | Curr: 1235.781427 | Rate: 0.00 | Evals: 75.2%
Iter 8 | Best: 11.422734 | Curr: 50.337174 | Rate: 0.00 | Evals: 75.3%
Iter 9 | Best: 11.422734 | Curr: 29.504173 | Rate: 0.00 | Evals: 75.4%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
  Internal scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
Restart injection: Using best found point so far as starting point (f(x)=11.422734).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 11.422734
Iter 1 | Best: 11.422734 | Curr: 26.987808 | Rate: 0.00 | Evals: 77.5%
Iter 2 | Best: 11.422734 | Curr: 1900.939895 | Rate: 0.00 | Evals: 77.6%
Iter 3 | Best: 11.422734 | Curr: 81.423969 | Rate: 0.00 | Evals: 77.7%
Iter 4 | Best: 11.422734 | Curr: 3277.019183 | Rate: 0.00 | Evals: 77.8%
Iter 5 | Best: 11.422734 | Curr: 1977.990649 | Rate: 0.00 | Evals: 77.9%
Iter 6 | Best: 11.422734 | Curr: 855.923275 | Rate: 0.00 | Evals: 78.0%
Iter 7 | Best: 11.422734 | Curr: 2929.146696 | Rate: 0.00 | Evals: 78.1%
Iter 8 | Best: 11.422734 | Curr: 1648.687976 | Rate: 0.00 | Evals: 78.2%
Iter 9 | Best: 11.422734 | Curr: 3302.646274 | Rate: 0.00 | Evals: 78.3%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
  Internal scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
Restart injection: Using best found point so far as starting point (f(x)=11.422734).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 11.422734
Iter 1 | Best: 11.422734 | Curr: 2247.577937 | Rate: 0.00 | Evals: 80.4%
Iter 2 | Best: 11.422734 | Curr: 3426.248747 | Rate: 0.00 | Evals: 80.5%
Iter 3 | Best: 11.422734 | Curr: 2490.502768 | Rate: 0.00 | Evals: 80.6%
Iter 4 | Best: 11.422734 | Curr: 2744.112078 | Rate: 0.00 | Evals: 80.7%
Iter 5 | Best: 11.422734 | Curr: 1904.553720 | Rate: 0.00 | Evals: 80.8%
Iter 6 | Best: 11.422734 | Curr: 1310.239713 | Rate: 0.00 | Evals: 80.9%
Iter 7 | Best: 11.422734 | Curr: 91.472506 | Rate: 0.00 | Evals: 81.0%
Iter 8 | Best: 11.422734 | Curr: 1106.971154 | Rate: 0.00 | Evals: 81.1%
Iter 9 | Best: 11.422734 | Curr: 185.476581 | Rate: 0.00 | Evals: 81.2%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
  Internal scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
Restart injection: Using best found point so far as starting point (f(x)=11.422734).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 11.422734
Iter 1 | Best: 11.422734 | Curr: 66.974944 | Rate: 0.00 | Evals: 83.3%
Iter 2 | Best: 11.422734 | Curr: 3533.156947 | Rate: 0.00 | Evals: 83.4%
Iter 3 | Best: 11.422734 | Curr: 601.910219 | Rate: 0.00 | Evals: 83.5%
Iter 4 | Best: 11.422734 | Curr: 937.508246 | Rate: 0.00 | Evals: 83.6%
Iter 5 | Best: 11.422734 | Curr: 51.986588 | Rate: 0.00 | Evals: 83.7%
Iter 6 | Best: 11.422734 | Curr: 3586.019149 | Rate: 0.00 | Evals: 83.8%
Iter 7 | Best: 11.422734 | Curr: 6798.555226 | Rate: 0.00 | Evals: 83.9%
Iter 8 | Best: 11.422734 | Curr: 1009.342620 | Rate: 0.00 | Evals: 84.0%
Iter 9 | Best: 11.422734 | Curr: 3545.751852 | Rate: 0.00 | Evals: 84.1%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
  Internal scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
Restart injection: Using best found point so far as starting point (f(x)=11.422734).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 11.422734
Iter 1 | Best: 11.422734 | Curr: 1053.568498 | Rate: 0.00 | Evals: 86.2%
Iter 2 | Best: 11.422734 | Curr: 2618.866021 | Rate: 0.00 | Evals: 86.3%
Iter 3 | Best: 11.422734 | Curr: 2796.490411 | Rate: 0.00 | Evals: 86.4%
Iter 4 | Best: 11.422734 | Curr: 1952.005480 | Rate: 0.00 | Evals: 86.5%
Iter 5 | Best: 11.422734 | Curr: 3069.808994 | Rate: 0.00 | Evals: 86.6%
Iter 6 | Best: 11.422734 | Curr: 2530.061421 | Rate: 0.00 | Evals: 86.7%
Iter 7 | Best: 11.422734 | Curr: 6909.742140 | Rate: 0.00 | Evals: 86.8%
Iter 8 | Best: 11.422734 | Curr: 1163.915947 | Rate: 0.00 | Evals: 86.9%
Iter 9 | Best: 11.422734 | Curr: 4466.596321 | Rate: 0.00 | Evals: 87.0%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
  Internal scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
Restart injection: Using best found point so far as starting point (f(x)=11.422734).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 11.422734
Iter 1 | Best: 11.422734 | Curr: 2613.246849 | Rate: 0.00 | Evals: 89.1%
Iter 2 | Best: 11.422734 | Curr: 197.955265 | Rate: 0.00 | Evals: 89.2%
Iter 3 | Best: 11.422734 | Curr: 48.046767 | Rate: 0.00 | Evals: 89.3%
Iter 4 | Best: 11.422734 | Curr: 1757.559872 | Rate: 0.00 | Evals: 89.4%
Iter 5 | Best: 11.422734 | Curr: 74.604610 | Rate: 0.00 | Evals: 89.5%
Iter 6 | Best: 11.422734 | Curr: 2614.221056 | Rate: 0.00 | Evals: 89.6%
Iter 7 | Best: 11.422734 | Curr: 112.282536 | Rate: 0.00 | Evals: 89.7%
Iter 8 | Best: 11.422734 | Curr: 1344.679387 | Rate: 0.00 | Evals: 89.8%
Iter 9 | Best: 11.422734 | Curr: 2445.191293 | Rate: 0.00 | Evals: 89.9%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
  Internal scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
Restart injection: Using best found point so far as starting point (f(x)=11.422734).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 11.422734
Iter 1 | Best: 11.422734 | Curr: 404.142590 | Rate: 0.00 | Evals: 92.0%
Iter 2 | Best: 11.422734 | Curr: 2653.164065 | Rate: 0.00 | Evals: 92.1%
Iter 3 | Best: 11.422734 | Curr: 3969.088527 | Rate: 0.00 | Evals: 92.2%
Iter 4 | Best: 11.422734 | Curr: 239.645592 | Rate: 0.00 | Evals: 92.3%
Iter 5 | Best: 11.422734 | Curr: 172.953318 | Rate: 0.00 | Evals: 92.4%
Iter 6 | Best: 11.422734 | Curr: 3023.335622 | Rate: 0.00 | Evals: 92.5%
Iter 7 | Best: 11.422734 | Curr: 191.717718 | Rate: 0.00 | Evals: 92.6%
Iter 8 | Best: 11.422734 | Curr: 342.253335 | Rate: 0.00 | Evals: 92.7%
Iter 9 | Best: 11.422734 | Curr: 186.730851 | Rate: 0.00 | Evals: 92.8%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
  Internal scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
Restart injection: Using best found point so far as starting point (f(x)=11.422734).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 11.422734
Iter 1 | Best: 11.422734 | Curr: 465.468721 | Rate: 0.00 | Evals: 94.9%
Iter 2 | Best: 11.422734 | Curr: 58.034301 | Rate: 0.00 | Evals: 95.0%
Iter 3 | Best: 11.422734 | Curr: 135.915130 | Rate: 0.00 | Evals: 95.1%
Iter 4 | Best: 11.422734 | Curr: 143.878298 | Rate: 0.00 | Evals: 95.2%
Iter 5 | Best: 11.422734 | Curr: 1252.068235 | Rate: 0.00 | Evals: 95.3%
Iter 6 | Best: 11.422734 | Curr: 5686.136574 | Rate: 0.00 | Evals: 95.4%
Iter 7 | Best: 11.422734 | Curr: 185.303381 | Rate: 0.00 | Evals: 95.5%
Iter 8 | Best: 11.422734 | Curr: 181.009718 | Rate: 0.00 | Evals: 95.6%
Iter 9 | Best: 11.422734 | Curr: 191.321480 | Rate: 0.00 | Evals: 95.7%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
  Internal scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
Restart injection: Using best found point so far as starting point (f(x)=11.422734).
Including 1 starting points from x0 in initial design.
Skipping re-evaluation of injected best point.
Initial best: f(x) = 11.422734
Iter 1 | Best: 11.422734 | Curr: 492.535074 | Rate: 0.00 | Evals: 97.8%
Iter 2 | Best: 11.422734 | Curr: 55.878904 | Rate: 0.00 | Evals: 97.9%
Iter 3 | Best: 11.422734 | Curr: 2071.725970 | Rate: 0.00 | Evals: 98.0%
Iter 4 | Best: 11.422734 | Curr: 502.915418 | Rate: 0.00 | Evals: 98.1%
Iter 5 | Best: 11.422734 | Curr: 1013.183857 | Rate: 0.00 | Evals: 98.2%
Iter 6 | Best: 11.422734 | Curr: 561.838886 | Rate: 0.00 | Evals: 98.3%
Iter 7 | Best: 11.422734 | Curr: 2868.968184 | Rate: 0.00 | Evals: 98.4%
Iter 8 | Best: 11.422734 | Curr: 2812.621572 | Rate: 0.00 | Evals: 98.5%
Iter 9 | Best: 11.422734 | Curr: 5911.894771 | Rate: 0.00 | Evals: 98.6%
Restarting optimization: success_rate 0 for 10 iterations.
Starting point x0 validated and processed successfully.
  Original scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
  Internal scale: [0.8729275  0.0177692  0.93560849 0.25940766 0.45715759 0.68663914
 0.15485909 0.80756336 0.52877566 0.40454999]
Restart injection: Using best found point so far as starting point (f(x)=11.422734).
Global budget exhausted. Stopping restarts.
   Best y (Default): 11.422734

--- Results Summary ---
Method                    | Best Objective 
----------------------------------------
Default (GP Only)         | 11.422734      
Multi-Surrogate (GP+MLP)  | 5.214140