surrogate.kriging.Kriging

surrogate.kriging.Kriging(
    noise=None,
    penalty=10000.0,
    method='regression',
    var_type=None,
    name='Kriging',
    seed=124,
    model_fun_evals=None,
    n_theta=None,
    min_theta=-3.0,
    max_theta=2.0,
    theta_init_zero=False,
    n_p=1,
    optim_p=False,
    min_Lambda=-9.0,
    max_Lambda=0.0,
    lambda_scale='log10',
    metric_factorial='hamming',
    isotropic=False,
    theta=None,
    **kwargs,
)

A scikit-learn compatible Kriging model class for regression tasks.

This class provides Ordinary Kriging with support for: - Mixed variable types (continuous, integer, factor) - Gaussian/RBF correlation function - Three fitting methods (Forrester (2008), Section 6): - Isotropic or anisotropic length scales

Compatible with SpotOptim’s variable type conventions: - ‘float’: continuous numeric variables - ‘int’: integer variables - ‘factor’: categorical/unordered variables

Parameters

Name	Type	Description	Default
noise	float	Small regularization term for numerical stability (nugget effect). If None, defaults to sqrt(machine epsilon). Only used for “interpolation” method. For “regression” and “reinterpolation”, this is replaced by the Lambda parameter. Defaults to None.	None
penalty	float	Large negative log-likelihood assigned if correlation matrix is not positive-definite. Defaults to 1e4.	10000.0
method	str	Fitting method (Forrester (2008), Section 6). Options: - “interpolation”: Pure Kriging interpolation (Eq 2.X). Fits exact data points. Uses a small noise (nugget) for numerical stability. - “regression”: Regression Kriging (Section 6.2). Optimizes a regularization parameter Lambda (nugget) along with theta. Suitable for noisy data. - “reinterpolation”: Re-interpolation (Section 6.3). Fits hyperparameters using regression (with Lambda), but predicts using the “noise-free” correlation matrix (removing Lambda). This creates a surrogate that glosses over noise but passes closer to the underlying trend (interpolating the regression model). Defaults to “regression”.	'regression'
var_type	List[str]	Variable types for each dimension. SpotOptim uses: ‘float’ (continuous), ‘int’ (integer), ‘factor’ (categorical). Defaults to [“float”].	None
name	str	Name of the Kriging instance. Defaults to “Kriging”.	'Kriging'
seed	int	Random seed for reproducibility. Defaults to 124.	124
model_fun_evals	int	Maximum function evaluations for hyperparameter optimization. Defaults to 100.	None
n_theta	int	Number of theta parameters. If None, set during fit. Defaults to None.	None
min_theta	float	Minimum log10(theta) bound. Defaults to -3.0.	-3.0
max_theta	float	Maximum log10(theta) bound. Defaults to 2.0.	2.0
theta_init_zero	bool	Initialize theta to zero. Defaults to False.	False
n_p	int	Number of p parameters (currently not optimized). Defaults to 1.	1
optim_p	bool	Optimize p parameters (currently not supported). Defaults to False.	False
min_Lambda	float	Minimum log10(Lambda) bound; the admissible nugget range is [10min_Lambda, 10max_Lambda] regardless of lambda_scale. Defaults to -9.0.	-9.0
max_Lambda	float	Maximum log10(Lambda) bound. Defaults to 0.0.	0.0
lambda_scale	str	Search parametrization of the nugget Lambda within [10min_Lambda, 10max_Lambda]. Options: - “log10”: the optimizer searches the exponent log10(Lambda) uniformly, emphasizing small nuggets (near-interpolation). - “linear”: the optimizer searches Lambda itself uniformly, emphasizing larger nuggets (more smoothing on noisy data). Defaults to “log10”.	'log10'
metric_factorial	str	Distance metric for factor variables. Defaults to "hamming". Hamming is a true nominal (order-agnostic) metric; canberra distance on integer level indices is order-dependent and singles out index 0. Any scipy distance metric remains selectable via this kwarg.	'hamming'
isotropic	bool	Use single theta for all dimensions. Defaults to False.	False
theta	np.ndarray	Initial theta values (log10 scale). Defaults to None.	None

Attributes

Name	Type	Description
X_	ndarray	Training data, shape (n_samples, n_features).
y_	ndarray	Training targets, shape (n_samples,).
theta_	ndarray	Optimized log10(theta) parameters.
Lambda_	float or None	Optimized Lambda parameter in the search scale (the log10 exponent for lambda_scale="log10", the linear value for lambda_scale="linear") for regression methods.
mu_	float	Mean of Kriging predictor.
sigma2_	float	Variance of Kriging predictor.
U_	ndarray	Cholesky factor of correlation matrix.
Psi_	ndarray	Correlation matrix.
negLnLike	float	Negative log-likelihood value.

Examples

Basic usage with SpotOptim:

>>> import numpy as np
>>> from spotoptim import SpotOptim
>>> from spotoptim.surrogate import Kriging
>>>
>>> # Define objective
>>> def objective(X):
...     return np.sum(X**2, axis=1)
>>>
>>> # Create Kriging surrogate
>>> kriging = Kriging(
...     noise=1e-10,
...     method='regression',
...     min_theta=-3.0,
...     max_theta=2.0,
...     seed=42
... )
>>>
>>> # Use with SpotOptim
>>> opt = SpotOptim(
...     fun=objective,
...     bounds=[(-5, 5), (-5, 5)],
...     surrogate=kriging,
...     max_iter=30,
...     n_initial=10,
...     seed=42
... )
>>> result = opt.optimize()

Direct usage (scikit-learn compatible):

>>> from spotoptim.surrogate import Kriging
>>> import numpy as np
>>>
>>> # Training data
>>> X_train = np.array([[0.0, 0.0], [0.5, 0.5], [1.0, 1.0]])
>>> y_train = np.array([0.1, 0.2, 0.3])
>>>
>>> # Fit model
>>> model = Kriging(method='regression', seed=42)
>>> model.fit(X_train, y_train)
>>>
>>> # Predict
>>> X_test = np.array([[0.25, 0.25], [0.75, 0.75]])
>>> y_pred = model.predict(X_test)
>>>
>>> # Predict with uncertainty
>>> y_pred, std = model.predict(X_test, return_std=True)

Methods

Name	Description
build_correlation_matrix	Build correlation matrix from training data.
build_psi_vector	Build correlation vector between x and training points.
fit	Fit the Kriging model to training data.
get_params	Get parameters for this estimator (scikit-learn compatibility).
likelihood	Compute negative concentrated log-likelihood.
objective	Objective function for hyperparameter optimization.
predict	Predict at new points.
predict_single	Predict at a single point.
set_params	Set parameters (scikit-learn compatibility).

build_correlation_matrix

surrogate.kriging.Kriging.build_correlation_matrix()

Build correlation matrix from training data.

Returns

Name	Type	Description
	np.ndarray	np.ndarray: Upper triangle of correlation matrix.

Examples

>>> import numpy as np
>>> from spotoptim.surrogate import Kriging
>>> X = np.array([[0.], [1.]])
>>> y = np.array([0., 1.])
>>> k = Kriging(seed=42).fit(X, y)
>>> Psi_upper = k.build_correlation_matrix()
>>> print(Psi_upper.shape)
(2, 2)

build_psi_vector

surrogate.kriging.Kriging.build_psi_vector(x)

Build correlation vector between x and training points.

Parameters

Name	Type	Description	Default
x	np.ndarray	Single test point, shape (n_features,).	required

Returns

Name	Type	Description
	np.ndarray	np.ndarray: Correlation vector, shape (n_samples,).

Reference

Calculates the vector small psi from Eq 2.15 in Forrester (2008).

Examples

>>> import numpy as np
>>> from spotoptim.surrogate import Kriging
>>> X = np.array([[0.], [1.]])
>>> y = np.array([0., 1.])
>>> k = Kriging(seed=42).fit(X, y)
>>> x_new = np.array([0.5])
>>> psi = k.build_psi_vector(x_new)
>>> print(psi.shape)
(2,)

fit

surrogate.kriging.Kriging.fit(X, y)

Fit the Kriging model to training data.

Optimizes hyperparameters (theta, Lambda) by maximizing the concentrated log-likelihood using differential evolution.

Parameters

Name	Type	Description	Default
X	np.ndarray	Training inputs, shape (n_samples, n_features).	required
y	np.ndarray	Training targets, shape (n_samples,).	required

Returns

Name	Type	Description
Kriging	Kriging	Fitted model instance (self).

get_params

surrogate.kriging.Kriging.get_params(deep=True)

Get parameters for this estimator (scikit-learn compatibility).

Parameters

Name	Type	Description	Default
deep	bool	Ignored, for compatibility.	True

Returns

Name	Type	Description
dict	Dict	Parameter names mapped to their values.

likelihood

surrogate.kriging.Kriging.likelihood(params)

Compute negative concentrated log-likelihood.

Parameters

Name	Type	Description	Default
params	np.ndarray	Hyperparameters [log10(theta), log10(Lambda)].	required

Returns

Name	Type	Description
tuple	Tuple[float, np.ndarray, np.ndarray]	(negLnLike, Psi, U) where U is Cholesky factor.

Reference

Forrester et al. (2008), Section 2.4. Matches implementation in likelihood.m from the book’s code. Concentrated Log-Likelihood approx: ln(L) ~ -(n/2)ln(sigma^2) - (1/2)ln|R|

Examples

>>> import numpy as np
>>> from spotoptim.surrogate import Kriging
>>> X = np.array([[0.], [1.]])
>>> y = np.array([0., 1.])
>>> k = Kriging(seed=42).fit(X, y)
>>> params = np.concatenate([k.theta_, [k.Lambda_]])
>>> nll, _, _ = k.likelihood(params)

objective

surrogate.kriging.Kriging.objective(params)

Objective function for hyperparameter optimization.

Parameters

Name	Type	Description	Default
params	np.ndarray	Hyperparameters to evaluate.	required

Returns

Name	Type	Description
float	float	Negative concentrated log-likelihood.

Examples

>>> import numpy as np
>>> from spotoptim.surrogate import Kriging
>>> X = np.array([[0.], [1.]])
>>> y = np.array([0., 1.])
>>> k = Kriging(seed=42).fit(X, y)
>>> # Evaluate objective at optimal parameters
>>> val = k.objective(np.concatenate([k.theta_, [k.Lambda_]]))

predict

surrogate.kriging.Kriging.predict(X, return_std=False)

Predict at new points.

Parameters

Name	Type	Description	Default
X	np.ndarray	Test points, shape (n_samples, n_features).	required
return_std	bool	Return standard deviations. Defaults to False.	False

Returns

Name	Type	Description
	np.ndarray	np.ndarray: Predictions, shape (n_samples,).
tuple	np.ndarray	(predictions, std_devs) if return_std=True.

predict_single

surrogate.kriging.Kriging.predict_single(x)

Predict at a single point.

Parameters

Name	Type	Description	Default
x	np.ndarray	Test point, shape (n_features,).	required

Returns

Name	Type	Description
tuple	Tuple[float, float]	(prediction, std_dev).

Reference

Forrester et al. (2008), Eq 2.15 (Predictor) and Eq 2.19 (Error). Matches implementation in pred.m from the book’s code.

Examples

>>> import numpy as np
>>> from spotoptim.surrogate import Kriging
>>> X = np.array([[0.], [1.]])
>>> y = np.array([0., 1.])
>>> k = Kriging(seed=42).fit(X, y)
>>> x_new = np.array([0.5])
>>> y_pred, y_std = k.predict_single(x_new)

set_params

surrogate.kriging.Kriging.set_params(**params)

Set parameters (scikit-learn compatibility).

Parameters

Name	Type	Description	Default
**params	typing.Any	Parameter names and values.	{}

Returns

Name	Type	Description
Kriging	Kriging	Self.