Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [transfer-learning]

Transfer learning is the process of learning a set of characteristics from one data and applying this "knowledge" to another similar dataset (i.e. using the same model across datasets).

Transfer learning is the process of learning a set of characteristics from one data and applying this "knowledge" to another similar dataset (i.e. using the same model across datasets).

187 questions
13
votes
3 answers

What are the consequences of not freezing layers in transfer learning?

I am trying to fine tune some code from a Kaggle kernel. The model uses pretrained VGG16 weights (via 'imagenet') for transfer learning. However, I notice there is no layer freezing of layers as is recommended in a keras blog. One approach would be…
Borealis
  • 347
  • 2
  • 4
  • 16
9
votes
2 answers

Effect of Stop-Word Removal on Transformers for Text Classification

The domain here is essentially topic classification, so not necessarily a problem where stop-words have an impact on the analysis (as opposed to, say, sentiment analysis where structure can affect meaning). With respect to the positional encoding…
Andy
  • 650
  • 4
  • 13
9
votes
1 answer

How to arrange the dataset/images for CNN+LSTM

I am working on an image classification problem using Transfer Learning with Resnet50 as base model (in Keras) (For example Class A and Class B). There is a time factor involved in this classification. For example, I need sufficient evidence to make…
deepguy
  • 1,471
  • 8
  • 21
  • 39
8
votes
3 answers

Further Training a pre-trained LLM

My goal is to use the general knowledge and language understanding of a pre-trained LLM and to continue training on a smaller domain specific corpus to improve the model's knowledge on the domain. What is the best practice approach here without…
Arthuro
  • 111
  • 1
  • 3
7
votes
3 answers

meaning of fine-tuning in nlp task

There are two types of transfer learning model. One is feature extraction, where the weights of the pre-trained model are not changed while training on the actual task and other is the weights of the pre-trained model can be changed. According to…
sovon
  • 521
  • 1
  • 5
  • 8
7
votes
2 answers

Default value of learning rate in adam optimizer - Keras

I am working on a image classification problem using Transfer Learning. Parameters used given below: Adam optimizer with learning rate - 0.0001 adamOpti = Adam(lr = 0.0001) model.compile(optimizer = adamOpti, loss = "categorical_crossentropy,…
deepguy
  • 1,471
  • 8
  • 21
  • 39
7
votes
3 answers

Is there any proven disadvantage of transfer learning for CNNs?

Suppose I know that I want to use a ResNet-101 architecture for my specific problem. There are ReseNet-101 models trained on ImageNet. Is there any disadvantage of using those pre-trained models and just resetting the last (few) layers to match the…
Martin Thoma
  • 19,540
  • 36
  • 98
  • 170
6
votes
2 answers

Why is input preprocessing in VGG16 in Keras not 1/255.0

I am just trying to use pre-trained vgg16 to make prediction in Keras like this. from scipy import ndimage from keras.preprocessing import image from keras.applications.vgg16 import preprocess_input im =…
kawingkelvin
  • 221
  • 1
  • 2
  • 7
6
votes
2 answers

What is the purpose of untrainable weights in Keras

In this page it is mentioned that when trainable=false, the weight won't be updated and is used for optimization, too. But I still do not understand how it can be useful? (e.g., I want to find out the best number of neurons or the best drop out…
william007
  • 775
  • 1
  • 10
  • 21
5
votes
2 answers

Is Flatten() layer in keras necessary?

In CNN transfer learning, after applying convolution and pooling,is Flatten() layer necessary? I have seen an example where after removing top layer of a vgg16 ,first applied layer was GlobalAveragePooling2D() and then Dense(). Is this specific to…
Shiv
  • 719
  • 6
  • 20
5
votes
1 answer

How to use MNIST dataset to make predictions on similar images (colorblindness charts)?

I am trying to use the MNIST dataset to train a convolutional neural network to classify digits written in colorblindness charts. As some people have suggested, I have tried playing with the brightness and contrast, as well as converting to…
shurup
  • 151
  • 3
5
votes
2 answers

How to add a CNN layer on top of BERT?

I am just playing with bert (Bidirectional Encoder Representation from Transformer) Research Paper Suppose I want to add any other model or layers like Convolutional Neural Network layers (CNN), Non Linear (NL) layers on top of BERT model. How can…
dev
  • 51
  • 1
  • 2
5
votes
2 answers

Feeding 3 consecutive video frames to a CNN to track a tennis ball

I want to use CNN transfer learning to track a tennis ball from TV broadcasts of tennis matches. I used VGG annotating tool annotation tool link (use version 1 of the tool for compatibility with matterport code) and have about 200 frames annotated…
mLstudent33
  • 604
  • 1
  • 7
  • 20
5
votes
2 answers

Can neural networks be adapted without recreating them completely?

If I have, for example, a classification network which can tell if there is a dog or a cat in a picture, is it possible to adapt the network so it can also learn to detect a mouse? Without making a new one from scratch. In this case it doesn't make…
Sven Laschinski
  • 85
  • 5
4
votes
3 answers

Can CNNs detect features of different images?

In lecture, we talked about “parameter sharing” as a benefit of using convolutional networks. Which of the following statements about parameter sharing in ConvNets are true? (Check all that apply.) It allows parameters learned for one task to be…
Z Oscar
  • 41
  • 2
1
2 3
…
12 13