I am curious about what would happen to hyperparameters when they would be set by a neural network itself
In general this is not possible as many hyper-parameters are discrete, so they are not differentiable with respect to any objective. For example, this applies to layer sizes, number of layers, choices of transfer functions. This prevents using any form of gradient descent to tune them directly as learnable parameters.
In fact the separation between parameters and hyperparameters is exactly that hyperparameters are not learnable by the model type. This applies to other ML models, not just neural networks.
or by creating a neural network that encapsulates and influences the hyperparameters of the network it encapsulates.
This is more feasible. You could use one neural network to try and predict the results from another. Then prefer to run tests on target networks that look like they will do well. However, using a "meta" neural network like this has some major drawbacks:
Neural networks require a lot of training data. Getting enough samples to make good predictions would require that you train your primary neural network (a time-consuming process) many times
Neural networks are bad at extrapolating to data outside of areas already experienced, so not so great at making creative predictions of new parameters to try
Neural networks have a lot of hyper-parameters to tune. Would you need a "meta meta" neural network to predict the performance of your "meta" network?
either it has never been done before or the idea is just really dumb
This is a real issue that comes up repeatedly. In general the search for best hyper-parameters is a chore. It is an active area of research and experimentation to find efficient ways of automating it, or avoiding it by making some hyperparameters less important or not necessary.
The reason you are not finding neural networks that tune neural networks is due to the issues listed above. So the main areas of research focus on different approaches, that can work with limited data and don't have so many hyperparameters themselves. Or models that are robust to large ranges of hyperparameters, so precise tuning is not a big deal.
Here are a few pointers to help with automated searches:
You could use a variety of hyperparameter optimisation schemes, including random search, grid search, genetic algorithms, simple gradient methods etc.
Random searches, perhaps constrained by previous experience or second-hand knowledge from similar problems, can be reasonably effective.
The quality of any search is limited by the quality and amount of cross-validation data. There is not much point tuning the cv loss value to the point that you care about changes that are much less than the standard error in its estimate.
Response to hyperparameters is typically non-linear over the search space, which makes things harder.
Outside of automation, expert analysis is often a good starting point, especially if you want to assess success of regularisation. Typically you can look at learning curves for training and cross-validation data, and based on that you can make a reasonable guess as to whether to increase or reduce regularisation hyperparameters and/or learning rate, even from observing results from a single training run.
There have likely been attempts to automate some parts of reading learning curves, since sometimes it is relatively easy to detect over-fitting and under-fitting scenarios. However, I could not find any examples when searching just now.
