The second convolutional layers use the results of the first layer as input. Remember, the input to the first layer was the original bitmap. The output of the first layer actually represents the location in the 2D bitmap, where a specific primitive was seen. The features of the second layer are more comprehensive than the first. The second layer would have composite structures such as splines and curves. Here, we will describe the role of the neuron, and the computations necessary to force an output from a neuron.

The role of the neuron is to input the sum of all the weights entering it against the pixel values. In the following graph, we see the neuron accepting inputs from the previous layer in the form of weights and bitmap values. The role of the neuron is to sum the weights and values and force them through an activation function as input to the next layer:

The equation for the neuron function is:

This can be a very large matrix multiplication problem. The input image is flattened into a one-dimensional array. The bias provides a method to influence the output without interacting with the real data. In the following diagram, we see an example of a weight matrix multiplied by a flattened one-dimensional image, and with the added bias. Note that in actual CNN devices you can add the bias to the weight matrix, and add a single 1.0 value to the bottom of the bitmap vector as a form of optimization. Here, the second value in the result matrix of 29.6 is the value chosen:

The input values are multiplied by the weighting on each entering the neuron. This is a simple linear transform in matrix math. That value needs to pass through an activation function to determine if the neuron should fire or not. A digital system built on transistors takes voltages as input, and if the voltage meets a threshold value, the transistor switches on. The biological analog is the neuron that behaves nonlinearly to the input. Since we are modeling a neural network, we attempt to use nonlinear activation functions. Typical activation functions that can be chosen include:

• Logistic (sigmoid)
• tanH
• Rectified linear unit (ReLU)
• Exponential Linear Unit (ELU)
• sinusoidal

The sigmoid activation function is:

Without a sigmoid (or any type of activation function) layer, the system would be a linear transformation function and have substantially less accuracy for image or pattern recognition.