In this example, we will take a pre-trained VGGNet and use transfer learning to train a CNN classifier that predicts dog breeds, given a dog image. Keras contains many pre-trained models, along with the code that loads and visualizes them. Another is a flower dataset that can be downloaded here. The Dog breed dataset has 133 dog breed categories and 8,351 dog images. Download the Dog breed dataset here and copy it to your folder. VGGNet has 16 convolutional with pooling layers from beginning to end and three fully connected layers followed by a softmax function. Its main objective was to show how the depth of the network gives the best performance. It came from Visual Geometric Group (VGG) at Oxford. Their best performing network is VGG16. The Dog breed dataset is relatively small and has a little overlap with the imageNet dataset. So, we can remove the last fully connected layer after the convolutional layer and replace it with our own. The weights of the convolutional layer are kept constant. An input image is passed through the convolutional layer and stops at the 16th layer:

We will use the bottleneck features of a pre-trained VGG16 network – such a network has already learned features from the imageNet dataset. Because the imageNet dataset already contains a few images of dogs, the VGG16 network model has already learned key features for classification. Similarly, other pre-trained CNN architectures can also be considered as an exercise to solve other image classification tasks.

Download the bottleneck_features of VGG16 here, copy it to your own folder, and load it:

bottleneck_features = np.load('bottleneck_features/DogVGG16Data.npz')
train_vgg16 = bottleneck_features['train']
valid_vgg16 = bottleneck_features['valid']
test_vgg16 = bottleneck_features['test']

Now define the model architecture:

from keras.layers import GlobalAveragePooling2D

model = Sequential()
model.add(GlobalAveragePooling2D(input_shape=(7, 7, 512)))
model.add(Dense(133, activation='softmax'))
model.summary()

Layer (type)                     Output Shape          Param #     Connected to                     
=================================================================================================
globalaveragepooling2d_1 (Global (None, 512)           0           globalaveragepooling2d_input_1[0]
_________________________________________________________________________________________________
dense_2 (Dense)                  (None, 133)           68229       globalaveragepooling2d_1[0][0]   
=================================================================================================
Total params: 68,229
Trainable params: 68,229
Non-trainable params: 0
_________________________________________________________________________________________________

Compile the model and train it:

model.compile(loss='categorical_crossentropy', optimizer='rmsprop', 
                  metrics=['accuracy'])
from keras.callbacks import ModelCheckpoint 

# train the model
checkpointer = ModelCheckpoint(filepath='dogvgg16.weights.best.hdf5', verbose=1, 
                               save_best_only=True)
model.fit(train_vgg16, train_targets, nb_epoch=20, validation_data=(valid_vgg16, valid_targets), 
          callbacks=[checkpointer], verbose=1, shuffle=True)

Load the model and calculate the classification accuracy on the test set:

# load the weights that yielded the best validation accuracy
model.load_weights('dogvgg16.weights.best.hdf5')
# get index of predicted dog breed for each image in test set
vgg16_predictions = [np.argmax(model.predict(np.expand_dims(feature, axis=0))) 
                     for feature in test_vgg16]

# report test accuracy
test_accuracy = 100*np.sum(np.array(vgg16_predictions)==
                           np.argmax(test_targets, axis=1))/len(vgg16_predictions)
print('
Test accuracy: %.4f%%' % test_accuracy)