In the previous section, we saw a high-level overview of our network model. In this section, we want to look at a Keras implementation of a generative conversational model that uses sequence-to-sequence learning. Before we get into the theory of this form of generative model, let's get the sample running, since it can take a while. The sample we will explore is the Keras reference sample for sequence-to-sequence machine translation. It is currently configured to do English-to-French translation. 

Open up the Chapter_4_1.py sample code listing and get it running using these steps:

1. Open up a shell or Anaconda window. Then run the following command:

python3 Chapter_4_1.py

2. This will run the sample, and it may take several hours to run. The sample can also consume a substantial amount of memory and this may force memory paging on lower memory systems. Paging memory to disk will take additional time to train, especially if you are not running an SSD. If you find that you are unable to complete training on this sample, reduce the number of epochs and/or num_samples parameters as follows:

batch_size = 64 # Batch size for training.
epochs = 100 # Number of epochs to train for.
latent_dim = 256 # Latent dimensionality of the encoding space.
num_samples = 10000 # Number of samples to train on.

3. Decrease the epochs or num_samples parameters if you are unable to train on the original values.

4. After the sample has completed training, it will run through a test set of data. As it does so, it will output the results and you can see how well it is translating from English to French.
5. Open the fra-eng folder located in the chapter source code.

6. Open the fra.txt file and the top few lines are as follows:

Go. Va !
Hi. Salut !
Run! Cours !
Run! Courez !
Wow! Ça alors !
Fire! Au feu !
Help! À l'aide !
Jump. Saute.
Stop! Ça suffit !
Stop! Stop !
Stop! Arrête-toi !
Wait! Attends !
Wait! Attendez !
Go on. Poursuis.
Go on. Continuez.
Go on. Poursuivez.
Hello! Bonjour !
Hello! Salut !

7. Notice how the training text (English/French) is split on punctuation and spaces. Also, note how the sequences vary in length. The sequences we input do not have to match the length of the output, and vice versa.

The sample we just looked at uses sequence-to-sequence character encoding to translate text from English to French. Typically, chat generation is done with word-to-word encoding, but this sample uses a finer-grained character-to-character model. This has an advantage in games because the language we attempt to generate may not always be human. Keep in mind that while we are only generating translated text in this sample, the text paired with an input could be any response you deem appropriate. In the next section, we will break down the code and understand in some detail how this sample works.