Consider the feedforward neural network represented in the preceding figure, for example, it could represent the model for a XOR gate:

The following architectural representation is not very useful if you want to implement the network, because it does not consider the weight and bias vectors, which play a key role, in training phase, during the backpropagation algorithm. For this reason, it is convenient to represent the network as a computational graph, that is, in terms of data flowing through the network.

Just by way of explanation, let's focus on the following figure:

Here L3 denotes the output layer, L2 the hidden layer, and L1 the input layer. Similarly, θ2 denotes the weight vector between layer 2 and layer 3; θ1 denotes the weight vector between layer 1 and layer 2. The σ notation just refers to the sigmoid operation that takes place within those nodes (however, the outputs of those nodes will be referred to using the L notation, that is, L1, L2, and L3).

In this representation, each node does not represent a single neuron, but rather an operation, while the arrows are not connections but represent the flow of information along the network.

This computational graph shows us, step by step, which functions occur and the inputs those functions operated on.

The L2 layer function operates on two inputs: the outputs L1 layer (a vector) and the weight vector θ1, while the L2 and L3 function operates on θ2, and is our final output.

These considerations lead us to another peculiar aspect of TensorFlow, the programming model.