nn.training.evaluate_sequences

nn.training.evaluate_sequences(
    model,
    val_loader,
    device='cpu',
    metrics_only=False,
)

Evaluate a sequence model with per-batch MAPE and RMSE.

Metrics are the torchmetrics MeanAbsolutePercentageError and MeanSquaredError(squared=False) between model(x, lengths).squeeze() and the squeezed target batch, as in the schu25a original. Inputs, predictions, and targets are additionally returned as nested Python lists for plotting.

Parameters

Name	Type	Description	Default
model	`nn`.`Module`	Model called as `model(x, lengths)`.	required
val_loader	`DataLoader`	Loader yielding `(x, lengths, y)` batches. Batches are converted to NumPy, so `device` should be “cpu” unless `metrics_only=True`.	required
device	str	Evaluation device. Defaults to “cpu”.	`'cpu'`
metrics_only	bool	Return only `(mean_mape, mean_rmse)`. Defaults to False.	`False`

Returns

Name	Type	Description
tuple	Union[Tuple[float, float], Tuple[list, list, list, List[float], List[float]]]	`(x_ls, y_hat_ls, y_ls, mape_loss_ls, rmse_loss_ls)` with one entry per batch, or `(mean_mape, mean_rmse)` if `metrics_only`.

Examples

import pandas as pd
import torch
from torch.utils.data import DataLoader
from spotoptim.data.manydataset import ManyToManyDataset, PadSequenceManyToMany
from spotoptim.nn.many_to_many_rnn import ManyToManyRNN
from spotoptim.nn.training import evaluate_sequences
from spotoptim.utils.seed import seed_everything

seed_everything(42)
frames = [pd.DataFrame({"x": [0.1, 0.2, 0.3], "y": [1.0, 2.0, 3.0]})]
ds = ManyToManyDataset(frames, target="y")
dl = DataLoader(ds, batch_size=1, shuffle=False, collate_fn=PadSequenceManyToMany())
model = ManyToManyRNN(input_size=1, rnn_units=8, fc_units=8)
x, y_hat, y, mape, rmse = evaluate_sequences(model, dl)
print(len(y_hat), len(mape), len(rmse))

1 1 1