Here we do not have an overfitting concern, as the dataset is large. So, in this case, we can retrain the entire network:

• Remove the last fully connected layer and replace it with a fully connected layer that matches the number of classes in the target dataset
• Randomly initialize the weights of this newly added, fully connected layer
• Initialize the rest of the weights with pre-trained weights
• Train the entire network: