1

When I combine standardizing and normalizing my input data for my hybrid ANN model, it gives the best results.

But I can't find anywhere, why. I based it on a paper's approach but they don't justify their practice either. Anyone knows why?

Standardization of my input data gives R2 less than 0.71, a higher RMSE and less stable results, than both standardizing and normalizing my input data.

Random initialization: initial_weights = np.random.randn((inputs * hiddens) + (hiddens * outputs)) Xavier initialization: limit = np.sqrt(6 / (n_in + n_out)) For stand with random initialization, RMSE: 43.9268, R^2: 0.4678 For stand with xavier initialization, RMSE: 36.7307, R^2: 0.6279 For stand-norm with random initialization, RMSE: 32.0556, R^2: 0.7166. My model converges quicker, better and more stable with stand-norm with random initialization. R^2 used is coefficient of determination.

Mia May
  • 11
  • 2

1 Answers1

3

It really doesn't matter, since if mean-zero standardized features are input into the ANN, then there will be negative and positive values -- which is almost a requirement to get an ANN to run. You could also only run min-max normalization using input features with range $[0,1]$, but there won't be any negative values.

Min-max normalization only changes the range of a feature into [0,1]. So if you standardize an original feature or standardize a normalized variant of the same feature, you'll still get the same standardized feature. Look a the table below:

enter image description here

The effect of the difference of feature scaling on ANN learning based on mean-zero standardized vs min-max normalized features could be overcome during ANN training, and it also depends on which activation function you use in the hidden layer.

wjktrs
  • 358
  • 5