import numpy as np
from math import inf
from spotpython.fun.objectivefunctions import Analytical
from spotpython.spot import Spot
from spotpython.utils.init import fun_control_init, surrogate_control_init
="015" PREFIX
20 Kriging with Varying Correlation-p
This chapter illustrates the difference between Kriging models with varying p. The difference is illustrated with the help of the spotpython
package.
20.1 Example: Spot
Surrogate and the 2-dim Sphere Function
20.1.1 The Objective Function: 2-dim Sphere
- The
spotpython
package provides several classes of objective functions. - We will use an analytical objective function, i.e., a function that can be described by a (closed) formula: \[f(x, y) = x^2 + y^2\]
- The size of the
lower
bound vector determines the problem dimension. - Here we will use
np.array([-1, -1])
, i.e., a two-dim function.
= Analytical().fun_sphere
fun = fun_control_init(PREFIX=PREFIX,
fun_control = np.array([-1, -1]),
lower = np.array([1, 1])) upper
- Although the default
spot
surrogate model is an isotropic Kriging model, we will explicitly set thetheta
parameter to a value of1
for both dimensions. This is done to illustrate the difference between isotropic and anisotropic Kriging models.
=surrogate_control_init(n_p=1,
surrogate_control=2.0,) p_val
= Spot(fun=fun,
spot_2 =fun_control,
fun_control=surrogate_control)
surrogate_control
spot_2.run()
spotpython tuning: 7.295426096197904e-06 [#######---] 73.33%
spotpython tuning: 7.295426096197904e-06 [########--] 80.00%
spotpython tuning: 7.295426096197904e-06 [#########-] 86.67%
spotpython tuning: 7.295426096197904e-06 [#########-] 93.33%
spotpython tuning: 7.295426096197904e-06 [##########] 100.00% Done...
Experiment saved to 015_res.pkl
<spotpython.spot.spot.Spot at 0x121ef76e0>
20.1.2 Results
spot_2.print_results()
min y: 7.295426096197904e-06
x0: 0.0005343129105654898
x1: 0.002647628336795204
[['x0', np.float64(0.0005343129105654898)],
['x1', np.float64(0.002647628336795204)]]
=True) spot_2.plot_progress(log_y
spot_2.surrogate.plot()
20.2 Example With Modified p
- We can use set
p_val
to a value other than2
to obtain a different Kriging model.
= surrogate_control_init(n_p=1,
surrogate_control =1.0)
p_val= Spot(fun=fun,
spot_2_p1=fun_control,
fun_control=surrogate_control)
surrogate_control spot_2_p1.run()
spotpython tuning: 0.008151340444524795 [#######---] 73.33%
spotpython tuning: 0.008151338321477783 [########--] 80.00%
spotpython tuning: 0.008151338321477783 [#########-] 86.67%
spotpython tuning: 0.008151329471958513 [#########-] 93.33%
spotpython tuning: 0.008151328637051841 [##########] 100.00% Done...
Experiment saved to 015_res.pkl
<spotpython.spot.spot.Spot at 0x1552b8560>
- The search progress of the optimization with the anisotropic model can be visualized:
=True) spot_2_p1.plot_progress(log_y
spot_2_p1.print_results()
min y: 0.008151328637051841
x0: 0.051706566088337895
x1: 0.07401188864232683
[['x0', np.float64(0.051706566088337895)],
['x1', np.float64(0.07401188864232683)]]
spot_2_p1.surrogate.plot()
20.2.1 Taking a Look at the p_val
Values
20.2.1.1 p_val
Values from the spot
Model
- We can check, which
p_val
values thespot
model has used: - The
p_val
values from the surrogate can be printed as follows:
spot_2_p1.surrogate.p_val
1.0
- Since the surrogate from the isotropic setting was stored as
spot_2
, we can also take a look at thetheta
value from this model:
spot_2.surrogate.p_val
2.0
20.3 Optimization of the p_val
Values
= surrogate_control_init(n_p=1,
surrogate_control =True)
optim_p= Spot(fun=fun,
spot_2_pm=fun_control,
fun_control=surrogate_control)
surrogate_control spot_2_pm.run()
spotpython tuning: 7.476853710610035e-06 [#######---] 73.33%
spotpython tuning: 7.476853710610035e-06 [########--] 80.00%
spotpython tuning: 7.476853710610035e-06 [#########-] 86.67%
spotpython tuning: 6.058492032872421e-06 [#########-] 93.33%
spotpython tuning: 6.058492032872421e-06 [##########] 100.00% Done...
Experiment saved to 015_res.pkl
<spotpython.spot.spot.Spot at 0x156f69070>
=True) spot_2_pm.plot_progress(log_y
spot_2_pm.print_results()
min y: 6.058492032872421e-06
x0: 0.0008753496178213194
x1: 0.0023004901824290602
[['x0', np.float64(0.0008753496178213194)],
['x1', np.float64(0.0023004901824290602)]]
spot_2_pm.surrogate.plot()
spot_2_pm.surrogate.p_val
array([1.99943293])
20.4 Optimization of Multiple p_val
Values
= surrogate_control_init(n_p=2,
surrogate_control =True)
optim_p= Spot(fun=fun,
spot_2_pmo=fun_control,
fun_control=surrogate_control)
surrogate_control spot_2_pmo.run()
spotpython tuning: 1.0859280773037223e-05 [#######---] 73.33%
spotpython tuning: 1.0859280773037223e-05 [########--] 80.00%
spotpython tuning: 1.0859280773037223e-05 [#########-] 86.67%
spotpython tuning: 1.0859280773037223e-05 [#########-] 93.33%
spotpython tuning: 1.0859280773037223e-05 [##########] 100.00% Done...
Experiment saved to 015_res.pkl
<spotpython.spot.spot.Spot at 0x156ea6cc0>
=True) spot_2_pmo.plot_progress(log_y
spot_2_pmo.print_results()
min y: 1.0859280773037223e-05
x0: 0.0007730756995786265
x1: 0.0032033786438318904
[['x0', np.float64(0.0007730756995786265)],
['x1', np.float64(0.0032033786438318904)]]
spot_2_pmo.surrogate.plot()
spot_2_pmo.surrogate.p_val
array([2. , 1.99967559])
20.5 Exercises
20.5.1 fun_branin
- Describe the function.
- The input dimension is
2
. The search range is \(-5 \leq x_1 \leq 10\) and \(0 \leq x_2 \leq 15\).
- The input dimension is
- Compare the results from
spotpython
runs with different options forp_val
. - Modify the termination criterion: instead of the number of evaluations (which is specified via
fun_evals
), the time should be used as the termination criterion. This can be done as follows (max_time=1
specifies a run time of one minute):
=inf,
fun_evals=1, max_time
20.5.2 fun_sin_cos
- Describe the function.
- The input dimension is
2
. The search range is \(-2\pi \leq x_1 \leq 2\pi\) and \(-2\pi \leq x_2 \leq 2\pi\).
- The input dimension is
- Compare the results from
spotpython
run a) with isotropic and b) anisotropic surrogate models. - Modify the termination criterion (
max_time
instead offun_evals
) as described forfun_branin
.
20.5.3 fun_runge
- Describe the function.
- The input dimension is
2
. The search range is \(-5 \leq x_1 \leq 5\) and \(-5 \leq x_2 \leq 5\).
- The input dimension is
- Compare the results from
spotpython
runs with different options forp_val
. - Modify the termination criterion (
max_time
instead offun_evals
) as described forfun_branin
.
20.5.4 fun_wingwt
- Describe the function.
- The input dimension is
10
. The search ranges are between 0 and 1 (values are mapped internally to their natural bounds).
- The input dimension is
- Compare the results from
spotpython
runs with different options forp_val
. - Modify the termination criterion (
max_time
instead offun_evals
) as described forfun_branin
.
20.6 Jupyter Notebook
Note
- The Jupyter-Notebook of this lecture is available on GitHub in the Hyperparameter-Tuning-Cookbook Repository