utils.eval.mo_cv_models

utils.eval.mo_cv_models(X, y, model_define_func, cv=5, scores=None)

Performs cross-validation for separate models for each target in a multi-output problem.

Parameters

Name	Type	Description	Default
X	`pd`.`DataFrame` or np.ndarray	Feature matrix.	required
y	`pd`.`DataFrame` or np.ndarray	Target matrix with multiple columns.	required
model_define_func	`Callable`	Function that returns a fresh model or pipeline instance.	required
cv	int or cross-validation generator, default=5	Number of folds or CV object.	`5`
scores	`Union`[`Dict`[str, str / `Callable`], str, `Callable`, None]	Scoring metric(s). - If None (default): Uses default scorer of the estimator. Returns List[np.ndarray]. - If str/Callable: Single scorer. Returns List[np.ndarray]. - If Dict: Dictionary of {name: scorer}. Returns Dict[str, List[np.ndarray]]. Note: Unlike mo_eval_models which takes raw functions, cross_val_score expects strings (e.g. ‘neg_mean_squared_error’) or make_scorer callables, or raw callables that fit the sklearn signature. To maintain similarity with mo_eval_models, we rely on cross_val_score’s flexibility.	`None`

Returns

Name	Type	Description
scores	`Union`[`List`[np.ndarray], `Dict`[str, `List`[np.ndarray]]]	- List of arrays (one array of CV scores per target) if `scores` is None or single. - Dictionary of {name: List[np.ndarray]} if `scores` is a dictionary.

Examples

>>> import pandas as pd
>>> import numpy as np
>>> from spotoptim.utils.eval import mo_cv_models

>>> # Generate dummy data
>>> np.random.seed(42)
>>> X = pd.DataFrame(np.random.rand(100, 5), columns=[f'x{i}' for i in range(5)])
>>> y = pd.DataFrame(np.random.rand(100, 3), columns=[f'y{i}' for i in range(3)])

>>> # Example 1: Default behavior (Default scorer, e.g. R2)
>>> def make_model():
...     from sklearn.linear_model import Ridge
...     return Ridge()
>>> cv_scores = mo_cv_models(X, y, make_model, cv=3)
Cross-validating target 1/3...
Cross-validating target 2/3...
Cross-validating target 3/3...
CV Scores Mean: ['-0.14', '-0.07', '-0.20']

>>> # Example 2: Custom single score (NMSE - Negative Mean Squared Error)
>>> # Note: sklearn scoring strings are preferred for cross_val_score
>>> nmse_scores = mo_cv_models(X, y, make_model, cv=3, scores='neg_mean_squared_error')
Cross-validating target 1/3...
Cross-validating target 2/3...
Cross-validating target 3/3...
CV Scores Mean: ['-0.09', '-0.08', '-0.10']

>>> # Example 3: Multiple custom scores
>>> my_scores = {'R2': 'r2', 'NMSE': 'neg_mean_squared_error'}
>>> all_cv_scores = mo_cv_models(X, y, make_model, cv=3, scores=my_scores)
Cross-validating target 1/3...
Cross-validating target 2/3...
Cross-validating target 3/3...
CV Scores Mean:
  R2: ['-0.14', '-0.07', '-0.20']
  NMSE: ['-0.09', '-0.08', '-0.10']