is learning rate schedule a hyperparameter?

Question

I believe term ‘learning rate schedule’ is a certain solution for tuning the learning rate. But at the same time, every parameter evaluating the parameter itself can be called a hyperparameter. So can the learning rate schedule be called a hyperparameter?

Answer 1

Hyper-parameters are essentially parameters that you have to tune manually and aren't learned.

The schedule can be considered a hyper-parameter, as well as all the scheduler's parameters (patience, decay, etc.) and even the decision to use or not a scheduler can be considered a hyperparameter.

Answer 2

Learning rate scheduling can be achieved through multiple ways e.g. 1 cycle, power, performance.

Selecting the approach is first Hyperparameter.

Then, selecting the appropriate parameter value for the approach will be another set of Hyperparameters. e.g. eta0, step size etc.