2

Below I have a model trained and the loss of both the training dataset (blue) and validation dataset (orange) are shown. From my understanding, the ideal case is that both validation and training loss should converge and stabilize in order to tell that the model does not underfit or overfit. But I am not sure about the model below. What you can tell from it's loss, please?

enter image description here

In addition, this is the accuracy of the model:

enter image description here

Update: after setting learning rate to 0.0001 as the answer suggested, I got the following loss:

enter image description here

And accuracy of the model:

enter image description here

Avv
  • 231
  • 1
  • 2
  • 10

1 Answers1

5

It seems like you're over fitting. There are tones of articles and blogs on how to avoid over fitting, but I mention some of them here anyway:

  • Reduce your learning rate to a very small number like 0.001 or even 0.0001.

  • Provide more data.

  • Set Dropout rates to a number like 0.2. Keep them uniform across the network. Another modern approach is the idea of using Batch Normalization instead of Dropout.

  • Try decreasing the batch size.

  • Using appropriate optimizer: You may need to experiment a bit on this. Use different optimizer on the same network, and select an optimizer which gives you the least loss.

  • Also try reducing the size of your model.

Amirhossein Rezaei
  • 623
  • 3
  • 14