function.sequence_cv_objective

function.sequence_cv_objective

Leave-one-sequence-out cross-validation objective for sequence regression.

Ported from the training path used by spotpython.fun.hyperlight.HyperLight with fun_control["hacky"]=True (spotpython.light.trainmodel), without Lightning: for every sequence of a spotoptim.data.manydataset.ManyToManyDataset, a fresh model is trained on all remaining sequences for the full number of epochs and evaluated on the held-out sequence; the objective value is the mean of the held-out losses. As in spotPython, training runs the complete epoch budget (the patience hyperparameter is accepted but has no effect, because the reference implementation never attached a validation loader during fitting) and the loss is computed on zero-padded batches without masking.

Requires the torch optional extra (pip install 'spotoptim[torch]').

Classes

Name	Description
SequenceCVObjective	Callable SpotOptim objective running leave-one-sequence-out CV.

SequenceCVObjective

function.sequence_cv_objective.SequenceCVObjective(
    experiment,
    seed=None,
    collate_fn=None,
    param_mappers=None,
)

Callable SpotOptim objective running leave-one-sequence-out CV.

The experiment’s dataset must be a spotoptim.core.data.SpotDataFromTorchDataset whose training dataset yields variable-length (features, targets) sequence pairs (e.g. a spotoptim.data.manydataset.ManyToManyDataset). For each hyperparameter configuration, every sequence is held out once: a fresh model is trained on the remaining sequences and scored on the held-out one, and the mean held-out loss is returned. A configuration whose training fails (e.g. an optimizer incompatible with dense gradients) evaluates to np.nan, to be repaired by SpotOptim’s penalty handling.

The tuned hyperparameters follow the spotPython ManyToManyRNNRegressor convention: epochs, batch_size, optimizer, and lr_mult drive training via spotoptim.nn.optimizer.optimizer_handler and a MultiStepLR schedule (three milestones at 1/4, 2/4, and 3/4 of the epoch budget, decay factor 0.1); all remaining parameters are passed to the model constructor.

Parameters

Name	Type	Description	Default
experiment	ExperimentControl	Experiment configuration; its `dataset` wraps the sequence dataset and its `model_class` is instantiated once per fold.	required
seed	Optional[int]	Random seed applied before each configuration evaluation. Falls back to `experiment.seed` when None. Defaults to None.	`None`
collate_fn	Optional[Callable]	Batch collate producing `(padded_x, lengths, padded_y)`. Defaults to `spotoptim.data.manydataset.PadSequenceManyToMany`.	`None`
param_mappers	Optional[Dict[str, Callable]]	Per-parameter functions applied to the decoded hyperparameter values before training, e.g. `{"epochs": lambda v: 2 ** int(v)}` to reproduce spotPython’s `transform_power_2_int`. Defaults to None.	`None`

Examples

import numpy as np
import pandas as pd
from spotoptim.core.data import SpotDataFromTorchDataset
from spotoptim.core.experiment import ExperimentControl
from spotoptim.data.manydataset import ManyToManyDataset
from spotoptim.function.sequence_cv_objective import SequenceCVObjective
from spotoptim.hyperparameters import ParameterSet
from spotoptim.nn.many_to_many_rnn import ManyToManyRNNRegressor

rng = np.random.default_rng(0)
frames = [
    pd.DataFrame({"x": rng.random(4), "y": rng.random(4)})
    for _ in range(3)
]
ds = ManyToManyDataset(frames, target="y")
params = ParameterSet()
params.add_int("epochs", 1, 2)
params.add_float("lr_mult", 0.1, 1.0)
exp = ExperimentControl(
    dataset=SpotDataFromTorchDataset(ds, input_dim=1, output_dim=1),
    model_class=ManyToManyRNNRegressor,
    hyperparameters=params,
    seed=42,
)
objective = SequenceCVObjective(exp)
y = objective(np.array([[2, 1.0]]))
print(y.shape)

(1, 1)

Methods

Name	Description
decode_params	Decode one parameter vector and apply the configured mappers.

decode_params

function.sequence_cv_objective.SequenceCVObjective.decode_params(X_row)

Decode one parameter vector and apply the configured mappers.

Parameters

Name	Type	Description	Default
X_row	np.ndarray	One row of the optimizer’s input array, in natural scale with factor levels as strings.	required

Returns

Name	Type	Description
	Dict[str, Any]	Dict[str, Any]: Hyperparameter dictionary ready for training.