57  Hyperparameter Tuning with spotpython and PyTorch Lightning Using a CondNet Model

• We use the Diabetes dataset to illustrate the hyperparameter tuning process of a CondNet model using the spotpython package.
• The CondNet model is a conditional neural network that can be used to model conditional distributions [LINK].

Note, the divergence_threshold is set to 5,000, which is based on some pre-experiments with the Diabetes data set.

from spotpython.data.diabetes import Diabetes
from spotpython.hyperdict.light_hyper_dict import LightHyperDict
from spotpython.fun.hyperlight import HyperLight
from spotpython.utils.init import (fun_control_init, surrogate_control_init, design_control_init)
from spotpython.utils.eda import print_exp_table
from spotpython.spot import Spot
from spotpython.utils.file import get_experiment_filename
from math import inf
from spotpython.hyperparameters.values import set_hyperparameter

PREFIX="CondNet_01"

data_set = Diabetes()
input_dim = 10
output_dim = 1
cond_dim = 2

fun_control = fun_control_init(
    PREFIX=PREFIX,
    fun_evals=inf,
    max_time=1,
    data_set = data_set,
    core_model_name="light.regression.NNCondNetRegressor",
    hyperdict=LightHyperDict,
    divergence_threshold=5_000,
    _L_in=input_dim - cond_dim,
    _L_out=1,
    _L_cond=cond_dim,)

fun = HyperLight().fun


set_hyperparameter(fun_control, "optimizer", [ "Adadelta", "Adam", "Adamax"])
set_hyperparameter(fun_control, "l1", [3,4])
set_hyperparameter(fun_control, "epochs", [3,7])
set_hyperparameter(fun_control, "batch_size", [4,5])
set_hyperparameter(fun_control, "dropout_prob", [0.0, 0.025])
set_hyperparameter(fun_control, "patience", [2,3])
set_hyperparameter(fun_control, "lr_mult", [0.1, 20.0])

design_control = design_control_init(init_size=10)

print_exp_table(fun_control)

module_name: light
submodule_name: regression
model_name: NNCondNetRegressor
| name           | type   | default   |   lower |   upper | transform             |
|----------------|--------|-----------|---------|---------|-----------------------|
| l1             | int    | 3         |     3   |   4     | transform_power_2_int |
| epochs         | int    | 4         |     3   |   7     | transform_power_2_int |
| batch_size     | int    | 4         |     4   |   5     | transform_power_2_int |
| act_fn         | factor | ReLU      |     0   |   5     | None                  |
| optimizer      | factor | SGD       |     0   |   2     | None                  |
| dropout_prob   | float  | 0.01      |     0   |   0.025 | None                  |
| lr_mult        | float  | 1.0       |     0.1 |  20     | None                  |
| patience       | int    | 2         |     2   |   3     | transform_power_2_int |
| batch_norm     | factor | 0         |     0   |   1     | None                  |
| initialization | factor | Default   |     0   |   4     | None                  |

spot_tuner = Spot(fun=fun,fun_control=fun_control, design_control=design_control)
res = spot_tuner.run()

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    587 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 779                                                                                              
Non-trainable params: 0                                                                                            
Total params: 779                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 26                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 24266.927734375, 'hp_metric': 24266.927734375}

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 2.6 K │ [[16, 8], [16, 2]] │   [16, 8] │
│ 1 │ layers     │ Sequential       │    153 │ train │ 4.2 K │            [16, 8] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 249                                                                                              
Non-trainable params: 0                                                                                            
Total params: 249                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 18                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 6.7 K                                                                                                 

train_model result: {'val_loss': 24001.298828125, 'hp_metric': 24001.298828125}

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 21142.806640625, 'hp_metric': 21142.806640625}

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 5.1 K │ [[32, 8], [32, 2]] │   [32, 8] │
│ 1 │ layers     │ Sequential       │    197 │ train │ 8.3 K │            [32, 8] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 293                                                                                              
Non-trainable params: 0                                                                                            
Total params: 293                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 24                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 13.4 K                                                                                                

train_model result: {'val_loss': 24029.455078125, 'hp_metric': 24029.455078125}

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 23382.876953125, 'hp_metric': 23382.876953125}

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    587 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 779                                                                                              
Non-trainable params: 0                                                                                            
Total params: 779                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 26                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 24010.201171875, 'hp_metric': 24010.201171875}

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 5.1 K │ [[32, 8], [32, 2]] │   [32, 8] │
│ 1 │ layers     │ Sequential       │    153 │ train │ 8.3 K │            [32, 8] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 249                                                                                              
Non-trainable params: 0                                                                                            
Total params: 249                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 18                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 13.4 K                                                                                                

train_model result: {'val_loss': 23698.623046875, 'hp_metric': 23698.623046875}

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 2.6 K │ [[16, 8], [16, 2]] │   [16, 8] │
│ 1 │ layers     │ Sequential       │    197 │ train │ 4.2 K │            [16, 8] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 293                                                                                              
Non-trainable params: 0                                                                                            
Total params: 293                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 24                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 6.7 K                                                                                                 

train_model result: {'val_loss': 23921.322265625, 'hp_metric': 23921.322265625}

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 2.6 K │ [[16, 8], [16, 2]] │   [16, 8] │
│ 1 │ layers     │ Sequential       │    153 │ train │ 4.2 K │            [16, 8] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 249                                                                                              
Non-trainable params: 0                                                                                            
Total params: 249                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 18                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 6.7 K                                                                                                 

train_model result: {'val_loss': 23699.216796875, 'hp_metric': 23699.216796875}

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 24038.666015625, 'hp_metric': 24038.666015625}
Anisotropic model: n_theta set to 10

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 22875.310546875, 'hp_metric': 22875.310546875}
Anisotropic model: n_theta set to 10
spotpython tuning: 21142.806640625 [----------] 0.71% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 23346.134765625, 'hp_metric': 23346.134765625}
Anisotropic model: n_theta set to 10
spotpython tuning: 21142.806640625 [----------] 1.40% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 21624.439453125, 'hp_metric': 21624.439453125}
Anisotropic model: n_theta set to 10
spotpython tuning: 21142.806640625 [----------] 2.27% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 24039.486328125, 'hp_metric': 24039.486328125}
Anisotropic model: n_theta set to 10
spotpython tuning: 21142.806640625 [----------] 3.28% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 20032.734375, 'hp_metric': 20032.734375}
Anisotropic model: n_theta set to 10
spotpython tuning: 20032.734375 [----------] 4.33% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 23277.66015625, 'hp_metric': 23277.66015625}
Anisotropic model: n_theta set to 10
spotpython tuning: 20032.734375 [#---------] 5.44% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 20655.541015625, 'hp_metric': 20655.541015625}
Anisotropic model: n_theta set to 10
spotpython tuning: 20032.734375 [#---------] 6.56% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 21750.794921875, 'hp_metric': 21750.794921875}
Anisotropic model: n_theta set to 10
spotpython tuning: 20032.734375 [#---------] 7.56% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 21496.064453125, 'hp_metric': 21496.064453125}
Anisotropic model: n_theta set to 10
spotpython tuning: 20032.734375 [#---------] 8.63% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 23725.314453125, 'hp_metric': 23725.314453125}
Anisotropic model: n_theta set to 10
spotpython tuning: 20032.734375 [#---------] 9.73% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 19829.45703125, 'hp_metric': 19829.45703125}
Anisotropic model: n_theta set to 10
spotpython tuning: 19829.45703125 [#---------] 10.76% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 21865.6171875, 'hp_metric': 21865.6171875}
Anisotropic model: n_theta set to 10
spotpython tuning: 19829.45703125 [#---------] 11.77% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 18811.375, 'hp_metric': 18811.375}
Anisotropic model: n_theta set to 10
spotpython tuning: 18811.375 [#---------] 12.80% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 22219.146484375, 'hp_metric': 22219.146484375}
Anisotropic model: n_theta set to 10
spotpython tuning: 18811.375 [#---------] 13.93% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 20606.806640625, 'hp_metric': 20606.806640625}
Anisotropic model: n_theta set to 10
spotpython tuning: 18811.375 [#---------] 14.98% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 23898.427734375, 'hp_metric': 23898.427734375}
Anisotropic model: n_theta set to 10
spotpython tuning: 18811.375 [##--------] 16.12% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 21360.95703125, 'hp_metric': 21360.95703125}
Anisotropic model: n_theta set to 10
spotpython tuning: 18811.375 [##--------] 17.31% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 22921.826171875, 'hp_metric': 22921.826171875}
Anisotropic model: n_theta set to 10
spotpython tuning: 18811.375 [##--------] 18.41% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 19732.291015625, 'hp_metric': 19732.291015625}
Anisotropic model: n_theta set to 10
spotpython tuning: 18811.375 [##--------] 19.62% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 19636.115234375, 'hp_metric': 19636.115234375}
Anisotropic model: n_theta set to 10
spotpython tuning: 18811.375 [##--------] 20.80% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 5.1 K │ [[32, 8], [32, 2]] │   [32, 8] │
│ 1 │ layers     │ Sequential       │    197 │ train │ 8.3 K │            [32, 8] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 293                                                                                              
Non-trainable params: 0                                                                                            
Total params: 293                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 24                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 13.4 K                                                                                                

train_model result: {'val_loss': 23986.65625, 'hp_metric': 23986.65625}
Anisotropic model: n_theta set to 10
spotpython tuning: 18811.375 [##--------] 22.13% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 22833.04296875, 'hp_metric': 22833.04296875}
Anisotropic model: n_theta set to 10
spotpython tuning: 18811.375 [##--------] 23.49% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 22749.46875, 'hp_metric': 22749.46875}
Anisotropic model: n_theta set to 10
spotpython tuning: 18811.375 [##--------] 24.97% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 19317.609375, 'hp_metric': 19317.609375}
Anisotropic model: n_theta set to 10
spotpython tuning: 18811.375 [###-------] 26.31% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 19508.140625, 'hp_metric': 19508.140625}
Anisotropic model: n_theta set to 10
spotpython tuning: 18811.375 [###-------] 27.61% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 2.6 K │ [[16, 8], [16, 2]] │   [16, 8] │
│ 1 │ layers     │ Sequential       │    197 │ train │ 4.2 K │            [16, 8] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 293                                                                                              
Non-trainable params: 0                                                                                            
Total params: 293                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 24                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 6.7 K                                                                                                 

train_model result: {'val_loss': 23646.12890625, 'hp_metric': 23646.12890625}
Anisotropic model: n_theta set to 10
spotpython tuning: 18811.375 [###-------] 28.96% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 18021.251953125, 'hp_metric': 18021.251953125}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [###-------] 30.18% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 18058.759765625, 'hp_metric': 18058.759765625}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [###-------] 31.24% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    587 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 779                                                                                              
Non-trainable params: 0                                                                                            
Total params: 779                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 26                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 23420.48046875, 'hp_metric': 23420.48046875}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [###-------] 32.20% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 23910.453125, 'hp_metric': 23910.453125}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [###-------] 33.50% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 24003.908203125, 'hp_metric': 24003.908203125}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [###-------] 34.70% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 20360.953125, 'hp_metric': 20360.953125}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [####------] 35.91% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 5.1 K │ [[32, 8], [32, 2]] │   [32, 8] │
│ 1 │ layers     │ Sequential       │    153 │ train │ 8.3 K │            [32, 8] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 249                                                                                              
Non-trainable params: 0                                                                                            
Total params: 249                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 18                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 13.4 K                                                                                                

train_model result: {'val_loss': 24040.212890625, 'hp_metric': 24040.212890625}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [####------] 36.79% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 23881.66015625, 'hp_metric': 23881.66015625}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [####------] 38.15% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 24051.560546875, 'hp_metric': 24051.560546875}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [####------] 39.18% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 23597.978515625, 'hp_metric': 23597.978515625}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [####------] 40.82% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 24148.033203125, 'hp_metric': 24148.033203125}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [####------] 42.39% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 24003.236328125, 'hp_metric': 24003.236328125}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [####------] 43.80% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 23903.51171875, 'hp_metric': 23903.51171875}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [#####-----] 45.06% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 23512.669921875, 'hp_metric': 23512.669921875}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [#####-----] 46.38% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 5.1 K │ [[32, 8], [32, 2]] │   [32, 8] │
│ 1 │ layers     │ Sequential       │    197 │ train │ 8.3 K │            [32, 8] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 293                                                                                              
Non-trainable params: 0                                                                                            
Total params: 293                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 24                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 13.4 K                                                                                                

train_model result: {'val_loss': 24030.25, 'hp_metric': 24030.25}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [#####-----] 47.66% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 23808.833984375, 'hp_metric': 23808.833984375}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [#####-----] 49.14% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    587 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 779                                                                                              
Non-trainable params: 0                                                                                            
Total params: 779                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 26                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 23566.736328125, 'hp_metric': 23566.736328125}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [#####-----] 50.26% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    587 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 779                                                                                              
Non-trainable params: 0                                                                                            
Total params: 779                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 26                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 23632.30859375, 'hp_metric': 23632.30859375}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [#####-----] 51.57% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 5.1 K │ [[32, 8], [32, 2]] │   [32, 8] │
│ 1 │ layers     │ Sequential       │    153 │ train │ 8.3 K │            [32, 8] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 249                                                                                              
Non-trainable params: 0                                                                                            
Total params: 249                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 18                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 13.4 K                                                                                                

train_model result: {'val_loss': 23915.96484375, 'hp_metric': 23915.96484375}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [#####-----] 52.57% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 22723.376953125, 'hp_metric': 22723.376953125}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [#####-----] 53.97% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 2.6 K │ [[16, 8], [16, 2]] │   [16, 8] │
│ 1 │ layers     │ Sequential       │    153 │ train │ 4.2 K │            [16, 8] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 249                                                                                              
Non-trainable params: 0                                                                                            
Total params: 249                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 18                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 6.7 K                                                                                                 

train_model result: {'val_loss': 23333.560546875, 'hp_metric': 23333.560546875}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [######----] 55.11% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 19659.193359375, 'hp_metric': 19659.193359375}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [######----] 56.02% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    587 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 779                                                                                              
Non-trainable params: 0                                                                                            
Total params: 779                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 26                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 23950.236328125, 'hp_metric': 23950.236328125}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [######----] 57.39% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    587 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 779                                                                                              
Non-trainable params: 0                                                                                            
Total params: 779                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 26                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 24033.064453125, 'hp_metric': 24033.064453125}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [######----] 58.25% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 23981.572265625, 'hp_metric': 23981.572265625}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [######----] 59.35% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 2.6 K │ [[16, 8], [16, 2]] │   [16, 8] │
│ 1 │ layers     │ Sequential       │    153 │ train │ 4.2 K │            [16, 8] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 249                                                                                              
Non-trainable params: 0                                                                                            
Total params: 249                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 18                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 6.7 K                                                                                                 

train_model result: {'val_loss': 23718.318359375, 'hp_metric': 23718.318359375}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [######----] 60.62% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    587 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 779                                                                                              
Non-trainable params: 0                                                                                            
Total params: 779                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 26                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 23007.353515625, 'hp_metric': 23007.353515625}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [######----] 61.74% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 24018.419921875, 'hp_metric': 24018.419921875}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [######----] 63.40% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 23720.45703125, 'hp_metric': 23720.45703125}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [######----] 64.54% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 2.6 K │ [[16, 8], [16, 2]] │   [16, 8] │
│ 1 │ layers     │ Sequential       │    197 │ train │ 4.2 K │            [16, 8] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 293                                                                                              
Non-trainable params: 0                                                                                            
Total params: 293                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 24                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 6.7 K                                                                                                 

train_model result: {'val_loss': 22730.6796875, 'hp_metric': 22730.6796875}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [#######---] 65.69% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 24064.5703125, 'hp_metric': 24064.5703125}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [#######---] 66.90% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 5.1 K │ [[32, 8], [32, 2]] │   [32, 8] │
│ 1 │ layers     │ Sequential       │    153 │ train │ 8.3 K │            [32, 8] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 249                                                                                              
Non-trainable params: 0                                                                                            
Total params: 249                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 18                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 13.4 K                                                                                                

train_model result: {'val_loss': 24041.13671875, 'hp_metric': 24041.13671875}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [#######---] 67.77% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 2.6 K │ [[16, 8], [16, 2]] │   [16, 8] │
│ 1 │ layers     │ Sequential       │    197 │ train │ 4.2 K │            [16, 8] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 293                                                                                              
Non-trainable params: 0                                                                                            
Total params: 293                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 24                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 6.7 K                                                                                                 

train_model result: {'val_loss': 22901.23046875, 'hp_metric': 22901.23046875}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [#######---] 69.09% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 23762.171875, 'hp_metric': 23762.171875}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [#######---] 70.42% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    587 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 779                                                                                              
Non-trainable params: 0                                                                                            
Total params: 779                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 26                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 23242.76171875, 'hp_metric': 23242.76171875}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [#######---] 71.43% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 5.1 K │ [[32, 8], [32, 2]] │   [32, 8] │
│ 1 │ layers     │ Sequential       │    197 │ train │ 8.3 K │            [32, 8] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 293                                                                                              
Non-trainable params: 0                                                                                            
Total params: 293                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 24                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 13.4 K                                                                                                

train_model result: {'val_loss': 21843.51953125, 'hp_metric': 21843.51953125}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [#######---] 72.67% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 18435.33203125, 'hp_metric': 18435.33203125}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [#######---] 73.86% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 2.6 K │ [[16, 8], [16, 2]] │   [16, 8] │
│ 1 │ layers     │ Sequential       │    153 │ train │ 4.2 K │            [16, 8] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 249                                                                                              
Non-trainable params: 0                                                                                            
Total params: 249                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 18                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 6.7 K                                                                                                 

train_model result: {'val_loss': 23414.376953125, 'hp_metric': 23414.376953125}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [#######---] 74.60% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 22943.310546875, 'hp_metric': 22943.310546875}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [########--] 76.96% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 21491.65234375, 'hp_metric': 21491.65234375}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [########--] 78.41% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 20338.62109375, 'hp_metric': 20338.62109375}
Anisotropic model: n_theta set to 10
spotpython tuning: 18021.251953125 [########--] 80.05% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 12682.9072265625, 'hp_metric': 12682.9072265625}
Anisotropic model: n_theta set to 10
spotpython tuning: 12682.9072265625 [########--] 81.08% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 10749.7138671875, 'hp_metric': 10749.7138671875}
Anisotropic model: n_theta set to 10
spotpython tuning: 10749.7138671875 [########--] 82.33% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 5.1 K │ [[32, 8], [32, 2]] │   [32, 8] │
│ 1 │ layers     │ Sequential       │    153 │ train │ 8.3 K │            [32, 8] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 249                                                                                              
Non-trainable params: 0                                                                                            
Total params: 249                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 18                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 13.4 K                                                                                                

train_model result: {'val_loss': 11404.779296875, 'hp_metric': 11404.779296875}
Anisotropic model: n_theta set to 10
spotpython tuning: 10749.7138671875 [########--] 83.90% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 5.1 K │ [[32, 8], [32, 2]] │   [32, 8] │
│ 1 │ layers     │ Sequential       │    153 │ train │ 8.3 K │            [32, 8] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 249                                                                                              
Non-trainable params: 0                                                                                            
Total params: 249                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 18                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 13.4 K                                                                                                

train_model result: {'val_loss': 16156.546875, 'hp_metric': 16156.546875}
Anisotropic model: n_theta set to 10
spotpython tuning: 10749.7138671875 [########--] 84.59% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    587 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 779                                                                                              
Non-trainable params: 0                                                                                            
Total params: 779                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 26                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': nan, 'hp_metric': nan}

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │  5.1 K │ [[16, 8], [16, 2]] │  [16, 16] │
│ 1 │ layers     │ Sequential       │    587 │ train │ 17.1 K │           [16, 16] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 779                                                                                              
Non-trainable params: 0                                                                                            
Total params: 779                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 26                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 22.2 K                                                                                                

train_model result: {'val_loss': 23719.162109375, 'hp_metric': 23719.162109375}
Anisotropic model: n_theta set to 10
spotpython tuning: 10749.7138671875 [#########-] 85.98% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 5.1 K │ [[32, 8], [32, 2]] │   [32, 8] │
│ 1 │ layers     │ Sequential       │    197 │ train │ 8.3 K │            [32, 8] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 293                                                                                              
Non-trainable params: 0                                                                                            
Total params: 293                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 24                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 13.4 K                                                                                                

train_model result: {'val_loss': 23277.46484375, 'hp_metric': 23277.46484375}
Anisotropic model: n_theta set to 10
spotpython tuning: 10749.7138671875 [#########-] 87.33% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    587 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 779                                                                                              
Non-trainable params: 0                                                                                            
Total params: 779                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 26                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 17525.4765625, 'hp_metric': 17525.4765625}
Anisotropic model: n_theta set to 10
spotpython tuning: 10749.7138671875 [#########-] 89.11% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 5.1 K │ [[32, 8], [32, 2]] │   [32, 8] │
│ 1 │ layers     │ Sequential       │    153 │ train │ 8.3 K │            [32, 8] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 249                                                                                              
Non-trainable params: 0                                                                                            
Total params: 249                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 18                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 13.4 K                                                                                                

train_model result: {'val_loss': 8332.513671875, 'hp_metric': 8332.513671875}
Anisotropic model: n_theta set to 10
spotpython tuning: 8332.513671875 [#########-] 90.59% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 5.1 K │ [[32, 8], [32, 2]] │   [32, 8] │
│ 1 │ layers     │ Sequential       │    153 │ train │ 8.3 K │            [32, 8] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 249                                                                                              
Non-trainable params: 0                                                                                            
Total params: 249                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 18                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 13.4 K                                                                                                

train_model result: {'val_loss': 14106.375, 'hp_metric': 14106.375}
Anisotropic model: n_theta set to 10
spotpython tuning: 8332.513671875 [#########-] 92.04% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 5.1 K │ [[32, 8], [32, 2]] │   [32, 8] │
│ 1 │ layers     │ Sequential       │    197 │ train │ 8.3 K │            [32, 8] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 293                                                                                              
Non-trainable params: 0                                                                                            
Total params: 293                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 24                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 13.4 K                                                                                                

train_model result: {'val_loss': 22904.21875, 'hp_metric': 22904.21875}
Anisotropic model: n_theta set to 10
spotpython tuning: 8332.513671875 [#########-] 94.03% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 5.1 K │ [[32, 8], [32, 2]] │   [32, 8] │
│ 1 │ layers     │ Sequential       │    197 │ train │ 8.3 K │            [32, 8] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 293                                                                                              
Non-trainable params: 0                                                                                            
Total params: 293                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 24                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 13.4 K                                                                                                

train_model result: {'val_loss': 21631.609375, 'hp_metric': 21631.609375}
Anisotropic model: n_theta set to 10
spotpython tuning: 8332.513671875 [##########] 95.75% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 5.1 K │ [[32, 8], [32, 2]] │   [32, 8] │
│ 1 │ layers     │ Sequential       │    197 │ train │ 8.3 K │            [32, 8] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 293                                                                                              
Non-trainable params: 0                                                                                            
Total params: 293                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 24                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 13.4 K                                                                                                

train_model result: {'val_loss': 22928.84765625, 'hp_metric': 22928.84765625}
Anisotropic model: n_theta set to 10
spotpython tuning: 8332.513671875 [##########] 97.35% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 5.1 K │ [[32, 8], [32, 2]] │   [32, 8] │
│ 1 │ layers     │ Sequential       │    153 │ train │ 8.3 K │            [32, 8] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 249                                                                                              
Non-trainable params: 0                                                                                            
Total params: 249                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 18                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 13.4 K                                                                                                

train_model result: {'val_loss': nan, 'hp_metric': nan}

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 2.6 K │ [[16, 8], [16, 2]] │   [16, 8] │
│ 1 │ layers     │ Sequential       │    197 │ train │ 4.2 K │            [16, 8] │   [16, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 293                                                                                              
Non-trainable params: 0                                                                                            
Total params: 293                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 24                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 6.7 K                                                                                                 

train_model result: {'val_loss': 23986.814453125, 'hp_metric': 23986.814453125}
Anisotropic model: n_theta set to 10
spotpython tuning: 8332.513671875 [##########] 98.22% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃ FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │     96 │ train │ 5.1 K │ [[32, 8], [32, 2]] │   [32, 8] │
│ 1 │ layers     │ Sequential       │    197 │ train │ 8.3 K │            [32, 8] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴───────┴────────────────────┴───────────┘

Trainable params: 293                                                                                              
Non-trainable params: 0                                                                                            
Total params: 293                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 24                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 13.4 K                                                                                                

train_model result: {'val_loss': 21144.810546875, 'hp_metric': 21144.810546875}
Anisotropic model: n_theta set to 10
spotpython tuning: 8332.513671875 [##########] 99.37% 

┏━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃   ┃ Name       ┃ Type             ┃ Params ┃ Mode  ┃  FLOPs ┃           In sizes ┃ Out sizes ┃
┡━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ 0 │ cond_layer │ ConditionalLayer │    192 │ train │ 10.2 K │ [[32, 8], [32, 2]] │  [32, 16] │
│ 1 │ layers     │ Sequential       │    691 │ train │ 34.2 K │           [32, 16] │   [32, 1] │
└───┴────────────┴──────────────────┴────────┴───────┴────────┴────────────────────┴───────────┘

Trainable params: 883                                                                                              
Non-trainable params: 0                                                                                            
Total params: 883                                                                                                  
Total estimated model params size (MB): 0                                                                          
Modules in train mode: 36                                                                                          
Modules in eval mode: 0                                                                                            
Total FLOPs: 44.4 K                                                                                                

train_model result: {'val_loss': 23853.578125, 'hp_metric': 23853.578125}
Anisotropic model: n_theta set to 10
spotpython tuning: 8332.513671875 [##########] 100.00% Done...

Experiment saved to CondNet_01_res.pkl

57.1 Looking at the Results

57.1.1 Tuning Progress

After the hyperparameter tuning run is finished, the progress of the hyperparameter tuning can be visualized with spotpython’s method plot_progress. The black points represent the performace values (score or metric) of hyperparameter configurations from the initial design, whereas the red points represents the hyperparameter configurations found by the surrogate model based optimization.

spot_tuner.plot_progress()

57.1.2 Tuned Hyperparameters and Their Importance

Results can be printed in tabular form.

from spotpython.utils.eda import print_res_table
print_res_table(spot_tuner)

| name           | type   | default   |   lower |   upper | tuned                | transform             |   importance | stars   |
|----------------|--------|-----------|---------|---------|----------------------|-----------------------|--------------|---------|
| l1             | int    | 3         |     3.0 |     4.0 | 3.0                  | transform_power_2_int |         0.25 | .       |
| epochs         | int    | 4         |     3.0 |     7.0 | 6.0                  | transform_power_2_int |        65.89 | **      |
| batch_size     | int    | 4         |     4.0 |     5.0 | 5.0                  | transform_power_2_int |        75.95 | **      |
| act_fn         | factor | ReLU      |     0.0 |     5.0 | ReLU                 | None                  |         0.25 | .       |
| optimizer      | factor | SGD       |     0.0 |     2.0 | Adadelta             | None                  |       100.00 | ***     |
| dropout_prob   | float  | 0.01      |     0.0 |   0.025 | 0.018183881133974802 | None                  |         5.61 | *       |
| lr_mult        | float  | 1.0       |     0.1 |    20.0 | 14.149741704784315   | None                  |         3.69 | *       |
| patience       | int    | 2         |     2.0 |     3.0 | 3.0                  | transform_power_2_int |         0.25 | .       |
| batch_norm     | factor | 0         |     0.0 |     1.0 | 0                    | None                  |        79.61 | **      |
| initialization | factor | Default   |     0.0 |     4.0 | Default              | None                  |        82.01 | **      |

A histogram can be used to visualize the most important hyperparameters.

spot_tuner.plot_importance(threshold=1.0)

spot_tuner.plot_important_hyperparameter_contour(max_imp=3)

l1:  0.24756627308334384
epochs:  65.89186481906664
batch_size:  75.95287834997562
act_fn:  0.24756627308334384
optimizer:  100.0
dropout_prob:  5.606977564360167
lr_mult:  3.6896098803603588
patience:  0.24756627308334384
batch_norm:  79.60935381759975
initialization:  82.00807794754971

57.1.3 Get the Tuned Architecture

import pprint
from spotpython.hyperparameters.values import get_tuned_architecture
config = get_tuned_architecture(spot_tuner)
pprint.pprint(config)

{'act_fn': ReLU(),
 'batch_norm': False,
 'batch_size': 32,
 'dropout_prob': 0.018183881133974802,
 'epochs': 64,
 'initialization': 'Default',
 'l1': 8,
 'lr_mult': 14.149741704784315,
 'optimizer': 'Adadelta',
 'patience': 8}