In this example, we'll define and train either a two-layer model or a convolutional model in the style of LeNet 5:

from six.moves import xrange   
import tensorflow as tf 
import prettytensor as pt 
from prettytensor.tutorial import data_utils 

tf.app.flags.DEFINE_string( 
    'save_path', None, 'Where to save the model checkpoints.') 
FLAGS = tf.app.flags.FLAGS 

BATCH_SIZE = 50 
EPOCH_SIZE = 60000 // BATCH_SIZE 
TEST_SIZE = 10000 // BATCH_SIZE

Since we are feeding our data as numpy arrays, we need to create placeholders in the graph. These must then be fed using the feed dict.

image_placeholder = tf.placeholder
                       (tf.float32, [BATCH_SIZE, 28, 28, 1])
labels_placeholder = tf.placeholder
                       (tf.float32, [BATCH_SIZE, 10])

tf.app.flags.DEFINE_string('model', 'full', 
                           'Choose one of the models, either            
                                              full or conv') 
FLAGS = tf.app.flags.FLAGS

We created the following function, multilayer_fully_connected. The first two layers are fully connected (100 neurons), and the final layer is a softmax result layer. Note that the chaining layer is a very simple operation:

def multilayer_fully_connected(images, labels): 
images = pt.wrap(images) 
  with pt.defaults_scope 
          (activation_fn=tf.nn.relu,l2loss=0.00001): 

    return (images.flatten(). 
            fully_connected(100). 
            fully_connected(100). 
            softmax_classifier(10, labels))

In the following, we'll build a multilayer convolutional network; the architecture is similar to that defined in LeNet 5. Please change this to experiment with other architectures:

def lenet5(images, labels): 
images = pt.wrap(images) 
  with pt.defaults_scope 
            (activation_fn=tf.nn.relu, l2loss=0.00001): 

    return (images.conv2d(5, 20). 
            max_pool(2, 2). 
            conv2d(5, 50). 
            max_pool(2, 2). 
            flatten(). 
            fully_connected(500). 
            softmax_classifier(10, labels))

Since we are feeding our data as numpy arrays, we need to create placeholders in the graph. These must then be fed using the feed dict:

def main(_=None): 
  image_placeholder = tf.placeholder 
                       (tf.float32, [BATCH_SIZE, 28, 28, 1]) 
  labels_placeholder = tf.placeholder 
                       (tf.float32, [BATCH_SIZE, 10])

Depending on FLAGS.model, we may have a two-layer classifier or a convolutional classifier, previously defined:

def main(_=None):

if FLAGS.model == 'full': 
    result = multilayer_fully_connected 
               (image_placeholder, labels_placeholder) 
  elif FLAGS.model == 'conv': 
    result = lenet5(image_placeholder, labels_placeholder) 
  else: 
    raise ValueError 
              ('model must be full or conv: %s' % FLAGS.model)

Then we define the accuracy function for the evaluated classifier:

accuracy = result.softmax.evaluate_classifier 
                   (labels_placeholder,phase=pt.Phase.test)

Next, we build the training and test sets:

  train_images, train_labels = data_utils.mnist(training=True) 
  test_images, test_labels = data_utils.mnist(training=False)

We will use a gradient descent optimizer procedure and apply it to the graph. The pt.apply_optimizer function adds regularization losses and sets up a step counter:

optimizer = tf.train.GradientDescentOptimizer(0.01) 
  train_op = pt.apply_optimizer 
                   (optimizer,losses=[result.loss])

We can set  save_path in the running session to automatically checkpoint every so often. Otherwise, at the end of the session, the model will be lost:

runner = pt.train.Runner(save_path=FLAGS.save_path) 
with tf.Session(): 
    for epoch in xrange(10):

Shuffle the training data:

        train_images, train_labels =  
                      data_utils.permute_data 
                      ((train_images, train_labels)) 

        runner.train_model(train_op,result. 
                           loss,EPOCH_SIZE, 
                           feed_vars=(image_placeholder, 
                                      labels_placeholder), 
                           feed_data=pt.train. 
                           feed_numpy(BATCH_SIZE, 
                                      train_images, 
                                      train_labels), 
                           print_every=100) 

        classification_accuracy = runner.evaluate_model 
                                  (accuracy, 
                                   TEST_SIZE, 
                                   feed_vars=(image_placeholder, 
                                              labels_placeholder), 
                                   feed_data=pt.train. 
                                   feed_numpy(BATCH_SIZE, 
                                              test_images, 
                                              test_labels))       
print('epoch' , epoch + 1) 
  print('accuracy', classification_accuracy ) 


if __name__ == '__main__': 
  tf.app.run()

Running this example provides the following output:

>>> 
Extracting /tmp/data/train-images-idx3-ubyte.gz 
Extracting tmp/data/train-labels-idx1-ubyte.gz 
Extracting /tmp/data/t10k-images-idx3-ubyte.gz 
Extracting /tmp/data/t10k-labels-idx1-ubyte.gz 
epoch =  1 
Accuracy [0.8994] 
epoch =  2 
Accuracy [0.91549999] 
epoch =  3 
Accuracy [0.92259997] 
epoch =  4 
Accuracy [0.92760003] 
epoch =  5 
Accuracy [0.9303] 
epoch =  6 
Accuracy [0.93870002] 
epoch =  7 
epoch =  8 
Accuracy [0.94700003] 
epoch =  9 
Accuracy [0.94910002] 
epoch =  10 
Accuracy [0.94980001]