The third type of nodes are mathematical operations, and these are going to be our matrix multiplication (MatMul), addition (Add), and ReLU. All of these are nodes in your TensorFlow graph, and it's very similar to NumPy operations:

Let's see what this graph will look like in code.

We perform the following steps to produce the preceding graph:

1. Create weights W and b, including initialization. We can initialize the weight matrix W by sampling from uniform distribution W ~ Uniform(-1,1) and initialize b to be 0.
2. Create input placeholder x, which is going to have a shape of m * 784 input matrix.
3. Build a flow graph.

Let's go ahead and follow those steps to build the flow graph:

# import TensorFlow package
import tensorflow as tf
# build a TensorFlow variable b taking in initial zeros of size 100
# ( a vector of 100 values)
b  = tf.Variable(tf.zeros((100,)))
# TensorFlow variable uniformly distributed values between -1 and 1
# of shape 784 by 100
W = tf.Variable(tf.random_uniform((784, 100),-1,1))
# TensorFlow placeholder for our input data that doesn't take in
# any initial values, it just takes a data type 32 bit floats as
# well as its shape
x = tf.placeholder(tf.float32, (100, 784))
# express h as Tensorflow ReLU of the TensorFlow matrix
#Multiplication of x and W and we add b
h = tf.nn.relu(tf.matmul(x,W) + b )

As you can see from the preceding code, we are not actually manipulating any data with this code snippet. We are only building symbols inside our graph and you can not print off h and see its value until we run this graph. So, this code snippet is just for building a backbone for our model. If you try to print the value of W or b in the preceding code, you should get the following output in Python:

So far, we have defined our graph and now, we need to actually run it.