In a feature extraction approach, we train only the top level of the network; the rest of the network remains fixed. Consider a feature extraction approach when the new dataset is relatively small and similar to the original dataset. In such cases, the higher-level features learned from the original dataset should transfer well to the new dataset.

Consider a fine-tuning approach when the new dataset is large and similar to the original dataset. Altering the original weights should be safe because the network is unlikely to overfit the new, large dataset.

Let us consider a pre-trained convolutional neural network, as shown in the following diagram. Using this we can study how the transfer of knowledge can be used in different situations:

When should we use transfer learning? Transfer learning can be applied in the following situations, depending on:

• The size of the new (target) dataset
• Similarity between the original and target datasets

There are four main use cases:

• Case 1: New (target) dataset is small and is similar to the original training dataset
• Case 2: New (target) dataset is small but is different from the original training dataset
• Case 3: New (target) dataset is large and is similar to the original training dataset
• Case 4: New (target) dataset is large and is different from the original training dataset

Let us now walk through each case in detail in the following sections.