All weights in the network are randomly initialized. Let's consider a specific hidden layer node and call it V. Assume that the weights of the connections from the inputs to that node are w1, w2, and w3 (as shown).
The network then takes the first training samples as input (we know that for inputs 35 and 67, the probability of passing is 1):
- Input to the network = [35, 67]
- Desired output from the network (target) = [1, 0]
Then, output V from the node in consideration, which can be calculated as follows (f is an activation function such as sigmoid):
V = f (1*w1 + 35*w2 + 67*w3)
Similarly, outputs from the other node in the hidden layer are also calculated. The outputs of the two nodes in the hidden layer act as inputs to the two nodes in the output layer. This enables us to calculate output probabilities from the two nodes in the output layer.
Suppose the output probabilities from the two nodes in the output layer are 0.4 and 0.6, respectively (since the weights are randomly assigned, outputs will also be random). We can see that the calculated probabilities (0.4 and 0.6) are very far from the desired probabilities (1 and 0 respectively); hence the network is said to have an incorrect output.