It's now time to build the core of our classification application, which is the computational graph of this CNN architecture, but to maximize the benefits of this implementation, we aren't going to use the TensorFlow layers API. Instead, we are going to use the TensorFlow neural network version of it.

So, let's start off by defining the model input placeholders which will input the images, target classes, and the keep probability parameter of the dropout layer (this helps us to reduce the complexity of the architecture by dropping some connections and hence reducing the chances of overfitting):

# Defining the model inputs
def images_input(img_shape):
 return tf.placeholder(tf.float32, (None, ) + img_shape, name="input_images")

def target_input(num_classes):
 
 target_input = tf.placeholder(tf.int32, (None, num_classes), name="input_images_target")
 return target_input

#define a function for the dropout layer keep probability
def keep_prob_input():
 return tf.placeholder(tf.float32, name="keep_prob")

Next up, we need to use the TensorFlow neural network implementation version to build up our convolution layers with max pooling:

# Applying a convolution operation to the input tensor followed by max pooling
def conv2d_layer(input_tensor, conv_layer_num_outputs, conv_kernel_size, conv_layer_strides, pool_kernel_size, pool_layer_strides):


 input_depth = input_tensor.get_shape()[3].value
 weight_shape = conv_kernel_size + (input_depth, conv_layer_num_outputs,)
 
 
 #Defining layer weights and biases
 weights = tf.Variable(tf.random_normal(weight_shape))
 biases = tf.Variable(tf.random_normal((conv_layer_num_outputs,)))
 
 #Considering the biase variable
 conv_strides = (1,) + conv_layer_strides + (1,)


 conv_layer = tf.nn.conv2d(input_tensor, weights, strides=conv_strides, padding='SAME')
 conv_layer = tf.nn.bias_add(conv_layer, biases)

 conv_kernel_size = (1,) + conv_kernel_size + (1,)

 pool_strides = (1,) + pool_layer_strides + (1,)
 pool_layer = tf.nn.max_pool(conv_layer, ksize=conv_kernel_size, strides=pool_strides, padding='SAME')
 return pool_layer

As you have probably seen in the previous chapter, the output of the max pooling operation is a 4D tensor, which is not compatible with the required input format for the fully connected layers. So, we need to implement a flattened layer to convert the output of the max pooling layer from 4D to 2D tensor:

#Flatten the output of max pooling layer to be fing to the fully connected layer which only accepts the output
# to be in 2D
def flatten_layer(input_tensor):
return tf.contrib.layers.flatten(input_tensor)

Next up, we need to define a helper function that will enable us to add a fully connected layer to our architecture:

#Define the fully connected layer that will use the flattened output of the stacked convolution layers
#to do the actuall classification
def fully_connected_layer(input_tensor, num_outputs):
 return tf.layers.dense(input_tensor, num_outputs)

Finally, before using these helper functions to create the entire architecture, we need to create another one that will take the output of the fully connected layer and produce 10 real-valued corresponding to the number of classes that we have in the dataset:

#Defining the output function
def output_layer(input_tensor, num_outputs):
    return  tf.layers.dense(input_tensor, num_outputs)

So, let's go ahead and define the function that will put all these bits and pieces together and create a CNN with three convolution layers. Each one of them is followed by max pooling operations. We'll also have two fully connected layers, where each one of them is followed by a dropout layer to reduce the model complexity and prevent overfitting. Finally, we'll have the output layer to produce 10 real-valued vectors, where each value represents a score for each class being the correct one:

def build_convolution_net(image_data, keep_prob):
 
 # Applying 3 convolution layers followed by max pooling layers
 conv_layer_1 = conv2d_layer(image_data, 32, (3,3), (1,1), (3,3), (3,3)) 
 conv_layer_2 = conv2d_layer(conv_layer_1, 64, (3,3), (1,1), (3,3), (3,3))
 conv_layer_3 = conv2d_layer(conv_layer_2, 128, (3,3), (1,1), (3,3), (3,3))

# Flatten the output from 4D to 2D to be fed to the fully connected layer
 flatten_output = flatten_layer(conv_layer_3)

# Applying 2 fully connected layers with drop out
 fully_connected_layer_1 = fully_connected_layer(flatten_output, 64)
 fully_connected_layer_1 = tf.nn.dropout(fully_connected_layer_1, keep_prob)
 fully_connected_layer_2 = fully_connected_layer(fully_connected_layer_1, 32)
 fully_connected_layer_2 = tf.nn.dropout(fully_connected_layer_2, keep_prob)
 
 #Applying the output layer while the output size will be the number of categories that we have
 #in CIFAR-10 dataset
 output_logits = output_layer(fully_connected_layer_2, 10)
 
 #returning output
 return output_logits

Let's call the preceding helper functions to build the network and define its loss and optimization criteria:

#Using the helper function above to build the network

#First off, let's remove all the previous inputs, weights, biases form the previous runs
tf.reset_default_graph()

# Defining the input placeholders to the convolution neural network
input_images = images_input((32, 32, 3))
input_images_target = target_input(10)
keep_prob = keep_prob_input()

# Building the models
logits_values = build_convolution_net(input_images, keep_prob)

# Name logits Tensor, so that is can be loaded from disk after training
logits_values = tf.identity(logits_values, name='logits')

# defining the model loss
model_cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits_values, labels=input_images_target))

# Defining the model optimizer
model_optimizer = tf.train.AdamOptimizer().minimize(model_cost)

# Calculating and averaging the model accuracy
correct_prediction = tf.equal(tf.argmax(logits_values, 1), tf.argmax(input_images_target, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32), name='model_accuracy')
tests.test_conv_net(build_convolution_net)

Now that we have built the computational architecture of this network, it's time to kick off the training process and see some results.