nn.optimizer.optimizer_handler

nn.optimizer.optimizer_handler(optimizer_name, params, lr_mult=1.0, **kwargs)

Return a torch optimizer with spotPython’s base learning-rate convention.

Parameters

Name	Type	Description	Default
optimizer_name	str	Name of the optimizer. Options: - “Adadelta”: base lr 1.0 (times lr_mult) - “Adagrad”: base lr 0.01 (times lr_mult) - “Adam”: base lr 0.001 (times lr_mult) - “AdamW”: base lr 0.001 (times lr_mult) - “SparseAdam”: fixed lr 0.001 (lr_mult ignored) - “Adamax”: base lr 0.002 (times lr_mult) - “ASGD”: base lr 0.01 (times lr_mult) - “LBFGS”: base lr 1.0 (times lr_mult) - “NAdam”: base lr 0.002 (times lr_mult) - “RAdam”: fixed lr 0.001 (lr_mult ignored) - “RMSprop”: base lr 0.01 (times lr_mult) - “Rprop”: base lr 0.01 (times lr_mult) - “SGD”: base lr 0.001 (times lr_mult)	required
params	Union[Iterable, list, torch.Tensor]	The parameters to optimize, e.g. model.parameters().	required
lr_mult	float	Multiplier for the optimizer’s base learning rate. Defaults to 1.0.	1.0
**kwargs	Any	Ignored. Accepted for call-site compatibility.	{}

Returns

Name	Type	Description
	torch.optim.Optimizer	torch.optim.Optimizer: An instance of the specified optimizer.

Raises

Name	Type	Description
	ValueError	If optimizer_name is not one of the supported names.

Examples

import torch
from spotoptim.nn.optimizer import optimizer_handler

weights = [torch.nn.Parameter(torch.zeros(3))]
opt = optimizer_handler("Adam", weights, lr_mult=0.5)
print(type(opt).__name__, opt.param_groups[0]["lr"])

Adam 0.0005