6

Is a NN with no hidden layer is behave like a regression? What we could say that NN without hidden layer can say us?

​

If we have for instance 20 input and 4 output and I have no true label, is it similar to regression?

If it is a regression then it will be a supervised learning however there is no true label!

What is wrong inside this clarification?

more clarification: Also, I have a dqn ( a deep learning structure inside q-learning approach)! Unfortunately I do not have the optimal answer of it! I am looking for a way to compare my results, Maybe I can compare by putting a un/supervised learning approach instead of DNN! For instance, you can put a DNN without hidden layer and as it behaves like the linear regression then you can put every supervised learning instead of it! As there is no True label I have problem with training section and estimate the DNN section with supervised learning! Also,as it is MDP, I can not estimate the DNN section with unsupervised learning!

user10296606
  • 1,906
  • 6
  • 18
  • 33

2 Answers2

4

You would then call it perception model. As in 1945 or so some researcher did this with no hidden layer. They came up with AND,OR,XOR problem where you can solve AND,OR with a single perception model but that model was unable to solve the XOR problem.

There are two kinds of regression: Linear and non linear. The activation functions in neural net makes them non linear regressor.

A Neural Net would be a linear regressor if you don't put any activation function.

Then it would be like : $Wx+b$ (which is $y=mx+b$ equivalent). But when you add activation function let it be sigmoid then it achieves a non linear space.Like $$\frac{1}{1+e^{-(wx+b)}}$$

"If we have for instance 20 input and 4 output and I have no true label, is it similar to regression?" --- In this you are just deleting back propagation thing. what as usual regression algorithms like linear,polynomial does is they just predict the function but never learns something like neural net learns proper function via optimizing weights. If you don't have true level that means you are not calculating the loss value between [true-predicted] so it would be a simple regression.

Stephen Rauch
  • 1,831
  • 11
  • 23
  • 34
Amin Pial
  • 165
  • 3
4

For your first problem, NN without hidden layer is simply linear regression. Of course there is an activation function, but you can use the inverse function of that activation function on your target set, then it's basically a linear regression.

Your second part of statement is confusing. You need to clarify what you want to do with your NN. Some unsupervised networks include Boltzmann Machine, auto-encoder, etc., for feature extraction and dimension reduction. However, we don't know if that's what you want.

If it's reinforcement learning, then it's reinforcement learning. RL is neither supervised nor unsupervised.

plpopk
  • 308
  • 1
  • 10