17

I'm new to data science and Neural Networks in general. Looking around, many people say it is better to normalize the data before doing anything with the NN. I understand how normalizing the input data can be useful.

However, I really don't see how normalizing the output data can help.

I've also tried both cases with an easy dataset, and I achieved the same results. The only difference is that in some weird problems, it is really hard to then re-convert the output back.

Can you give me some intuition on why we should also normalize the output?

Or maybe why it is indifferent?

Mauro Giliberti
  • 103
  • 4
Euler_Salter
  • 343
  • 1
  • 2
  • 7

1 Answers1

26

That paper gives a nice answer, where i quoted from. Search for Should I standardize the target variables (column vectors)? in that page.

Standardizing target variables is typically more a convenience for getting good initial weights than a necessity. However, if you have two or more target variables and your error function is scale-sensitive like the usual least (mean) squares error function, then the variability of each target relative to the others can effect how well the net learns that target. If one target has a range of 0 to 1, while another target has a range of 0 to 1,000,000, the net will expend most of its effort learning the second target to the possible exclusion of the first. So it is essential to rescale the targets so that their variability reflects their importance, or at least is not in inverse relation to their importance. If the targets are of equal importance, they should typically be standardized to the same range or the same standard deviation.

maniac
  • 103
  • 3
tomar__
  • 600
  • 6
  • 8