We can now move onto TensorBoard, by simply opening the terminal in the running folder, followed by this command:

$> Tensorboard --logdir = 'log_simple_stats_5_layers_relu_softmax'

Then open your browser at localhost .

In the following figure, we show the trend of the cost function as a function of the number of examples over the training set:

The cost function decreases with increasing iterations. This pattern is correct; it is exactly shown in the preceding figure. If this doesn't happen, it means that something went wrong: in the best case, it could simply be due to the fact that some parameters have not been set properly; at worst, it could be a problem in the constructed dataset, for example, too little information, poor quality images, or more. In this case, we must directly fix the dataset.