surrogate.mlp_surrogate

surrogate.mlp_surrogate

MLP Surrogate model for SpotOptim.

This module implements a standard Multi-Layer Perceptron (MLP) surrogate that enables uncertainty estimation via Monte Carlo Dropout (MC Dropout). It is designed to be a drop-in alternative to the Kriging surrogate within the SpotOptim framework.

Classes

Name	Description
MLPSurrogate	A scikit-learn compatible MLP surrogate model with uncertainty estimation.

MLPSurrogate

surrogate.mlp_surrogate.MLPSurrogate(
    in_channels=None,
    hidden_channels=None,
    l1=128,
    num_hidden_layers=3,
    dropout=0.0,
    activation='relu',
    lr=0.001,
    epochs=200,
    batch_size=32,
    optimizer_name='AdamWScheduleFree',
    mc_dropout_passes=30,
    seed=42,
    var_type=None,
    name='MLPSurrogate',
    verbose=False,
)

A scikit-learn compatible MLP surrogate model with uncertainty estimation.

This class wraps a PyTorch MLP (from spotoptim.nn.mlp) and provides the standard fit/predict interface required by SpotOptim. It uses Monte Carlo (MC) Dropout during prediction to estimate uncertainty (standard deviation) which is crucial for acquisition functions.

Compatible with SpotOptim’s variable type conventions: - ‘float’: continuous numeric variables - ‘int’: integer variables - ‘factor’: categorical/unordered variables

Note: All input variables are currently treated as numeric and standardized. For best performance with categorical variables, this simple treatment may be suboptimal compared to embedding processing, but ensures compatibility.

Parameters

Name	Type	Description	Default
in_channels	int	Input dimension. If None, inferred during fit.	None
hidden_channels	List[int]	Explicit list of hidden layer sizes.	None
l1	int	Neurons per hidden layer (used if hidden_channels is None). Defaults to 64.	128
num_hidden_layers	int	Number of hidden layers (used if hidden_channels is None). Defaults to 2.	3
dropout	float	Dropout probability. Crucial for uncertainty estimation. Defaults to 0.1.	0.0
lr	float	Learning rate. Defaults to 1e-3.	0.001
activation	str	Activation function. Defaults to “relu”.	'relu'
epochs	int	Number of training epochs. Defaults to 200.	200
batch_size	int	Training batch size. Defaults to 32.	32
optimizer_name	str	Name of PyTorch optimizer (“Adam”, “SGD”, etc.) or “AdamWScheduleFree”. Defaults to “AdamWScheduleFree”.	'AdamWScheduleFree'
mc_dropout_passes	int	Number of forward passes for MC Dropout uncertainty estimation. Defaults to 30.	30
seed	int	Random seed for reproducibility. Defaults to 42.	42
var_type	List[str]	Variable types for each dimension. Defaults to None.	None
name	str	Name of the surrogate. Defaults to “MLPSurrogate”.	'MLPSurrogate'
verbose	bool	Whether to print training progress. Defaults to False.	False

Methods

Name	Description
fit	Fit the MLP model to the training data.
predict	Predict targets for X.

fit

surrogate.mlp_surrogate.MLPSurrogate.fit(X, y)

Fit the MLP model to the training data.

Parameters

Name	Type	Description	Default
X	np.ndarray	Training inputs, shape (n_samples, n_features).	required
y	np.ndarray	Training targets, shape (n_samples,).	required

Returns

Name	Type	Description
MLPSurrogate	MLPSurrogate	The fitted model.

predict

surrogate.mlp_surrogate.MLPSurrogate.predict(X, return_std=False)

Predict targets for X.

Parameters

Name	Type	Description	Default
X	np.ndarray	Input data, shape (n_samples, n_features).	required
return_std	bool	Whether to return the standard deviation (uncertainty) along with the mean prediction. Defaults to False.	False

Returns

Name	Type	Description
	Union[np.ndarray, Tuple[np.ndarray, np.ndarray]]	np.ndarray: Predicted mean values, shape (n_samples,).
tuple	Union[np.ndarray, Tuple[np.ndarray, np.ndarray]]	(mean, std) if return_std is True.