Chapter 2. Build your vocabulary (word tokenization)
Listing 2.1. Example Monticello sentence split into tokens
Listing 2.2. One-hot vector sequence for the Monticello sentence
Listing 2.3. Prettier one-hot vectors
Listing 2.4. Construct a DataFrame of bag-of-words vectors
Listing 2.5. Example dot product calculation
Listing 2.6. Overlap of word counts for two bag-of-words vectors
Listing 2.7. Tokenize the Monticello sentence with a regular expression
Chapter 3. Math with words (TF-IDF vectors)
Chapter 4. Finding meaning in word counts (semantic analysis)
Listing 4.1. The SMS spam dataset
Listing 4.2. Topic-word matrix for LSA on 16 short sentences about cats, dogs, and NYC
Chapter 5. Baby steps with neural networks (perceptrons and backpropagation)
Listing 5.2. Perceptron random guessing
Listing 5.3. Perceptron learning
Listing 5.4. XOR Keras network
Chapter 6. Reasoning with word vectors (Word2vec)
Listing 6.1. Compute nessvector
Listing 6.2. Parameters to control Word2vec model training
Listing 6.3. Instantiating a Word2vec model
Listing 6.4. Loading a saved Word2vec model
Listing 6.5. Load a pretrained Word2vec model using nlpia
Listing 6.6. Examine Word2vec vocabulary frequencies
Listing 6.7. Distance between “Illinois” and “Illini”
Listing 6.8. Some US city data
Listing 6.9. Some US state data
Listing 6.10. Augment city word vectors with US state word vectors
Listing 6.11. Bubble chart of US cities
Chapter 7. Getting words in order with convolutional neural networks (CNNs)
Listing 7.1. Keras network with one convolution layer
Listing 7.2. Import your Keras convolution tools
Listing 7.3. Preprocessor to load your documents
Listing 7.4. Vectorizer and tokenizer
Listing 7.8. Padding and truncating your token sequence
Listing 7.9. Gathering your augmented and truncated data
Listing 7.10. Construct a 1D CNN
Listing 7.11. Fully connected layer with dropout
Listing 7.14. Output layer for categorical variable (word)
Listing 7.16. Save your hard work
Chapter 8. Loopy (recurrent) neural networks (RNNs)
Listing 8.1. Import all the things
Listing 8.2. Data preprocessor
Listing 8.3. Data tokenizer + vectorizer
Listing 8.5. Load and prepare your data
Listing 8.6. Initialize your network parameters
Listing 8.7. Load your test and training data
Listing 8.8. Initialize an empty Keras network
Listing 8.9. Add a recurrent layer
Listing 8.10. Add a dropout layer
Listing 8.11. Compile your recurrent network
Listing 8.12. Train and save your model
Listing 8.13. Model parameters
Listing 8.14. Build a larger network
Listing 8.15. Train your larger network
Chapter 9. Improving retention with long short-term memory networks
Listing 9.1. LSTM layer in Keras
Listing 9.2. Load and prepare the IMDB data
Listing 9.3. Build a Keras LSTM network
Listing 9.4. Fit your LSTM model
Listing 9.5. Save it for later
Listing 9.6. Reload your LSTM model
Listing 9.7. Use the model to predict on a sample
Listing 9.8. Optimize the thought vector size
Listing 9.9. Optimize LSTM hyperparameters
Listing 9.10. A more optimally sized LSTM
Listing 9.11. Train a smaller LSTM
Listing 9.12. Prepare the data
Listing 9.13. Calculate the average sample length
Listing 9.14. Prepare the strings for a character-based model
Listing 9.15. Pad and truncated characters
Listing 9.16. Character-based model “vocabulary”
Listing 9.17. One-hot encoder for characters
Listing 9.18. Load and preprocess the IMDB data
Listing 9.19. Split dataset for training (80%) and testing (20%)
Listing 9.20. Build a character-based LSTM
Listing 9.21. Train a character-based LSTM
Listing 9.22. And save it for later
Listing 9.23. Import the Project Gutenberg dataset
Listing 9.24. Preprocess Shakespeare plays
Listing 9.25. Assemble a training set
Listing 9.26. One-hot encode the training examples
Listing 9.27. Assemble a character-based LSTM model for generating text
Listing 9.28. Train your Shakespearean chatbot
Listing 9.29. Sampler to generate character sequences
Listing 9.30. Generate three texts with three diversity levels
Chapter 10. Sequence-to-sequence models and attention
Listing 10.1. Thought encoder in Keras
Listing 10.2. Thought decoder in Keras
Listing 10.3. Keras functional API (Model())
Listing 10.4. Train a sequence-to-sequence model in Keras
Listing 10.5. Decoder for generating text using the generic Keras Model
Listing 10.6. Sequence generator for random thoughts
Listing 10.7. Simple decoder—next word prediction
Listing 10.8. Build character sequence-to-sequence training set
Listing 10.9. Character sequence-to-sequence model parameters
Listing 10.10. Construct character sequence encoder-decoder training set
Listing 10.11. Construct and train a character sequence encoder-decoder network
Chapter 11. Information extraction (named entity extraction and question answering)
Listing 11.1. Pattern hardcoded in Python
Listing 11.2. Brittle pattern-matching example
Listing 11.3. Regular expression for GPS coordinates
Listing 11.4. Regular expression for US dates
Listing 11.5. Structuring extracted dates
Listing 11.6. Basic context maintenance
Listing 11.7. Regular expression for European dates
Listing 11.8. Recognizing years
Listing 11.9. Recognizing month words with regular expressions
Listing 11.10. Combining information extraction regular expressions
Listing 11.11. Validating dates
Listing 11.12. POS tagging with spaCy
Listing 11.13. Visualize a dependency tree
Listing 11.14. Helper functions for spaCy tagged strings
Listing 11.15. Example spaCy POS pattern
Listing 11.16. Creating a POS pattern matcher with spaCy
Listing 11.17. Using a POS pattern matcher
Listing 11.18. Combining multiple patterns for a more robust pattern matcher
Chapter 12. Getting chatty (dialog engines)
Listing 12.1. nlpia/book/examples/greeting.v2.aiml
Listing 12.2. nlpia/nlpia/data/greeting_step1.aiml
Listing 12.3. nlpia/book/examples/ch12.py
Listing 12.4. nlpia/nlpia/book/examples/ch12.py
Listing 12.5. nlpia/data/greeting_step2.aiml
Listing 12.6. nlpia/nlpia/book/examples/ch12.py
Listing 12.7. nlpia/nlpia/data/greeting_step3.aiml
Listing 12.8. nlpia/nlpia/book/examples/ch12.py
Listing 12.9. ch12_retrieval.py
Listing 12.10. ch12_retrieval.py
Listing 12.11. ch12_retrieval.py
Listing 12.12. ch12_retrieval.py
Listing 12.13. ch12_retrieval.py
Listing 12.14. ch12_retrieval.py
Listing 12.15. ch12_retrieval.py
Chapter 13. Scaling up (optimization, parallelization, and batch processing)
Listing 13.1. Load word2vec vectors
Listing 13.2. Initialize 300D AnnoyIndex
Listing 13.3. Add each word vector to the AnnoyIndex
Listing 13.4. Build Euclidean distance index with 15 trees
Listing 13.5. Find Harry_Potter neighbors with AnnoyIndex
Listing 13.6. Top Harry_Potter neighbors with gensim.KeyedVectors index
Listing 13.7. Build a cosine distance index
Listing 13.8. Build a cosine distance index
Listing 13.9. Harry_Potter neighbors in a cosine distance world
Listing 13.10. Search results accuracy for top 10
Listing 13.11. MinMaxScaler for low-dimensional vectors
Listing 13.12. Error message if your training data exceeds the GPU’s memory
Listing 13.13. Generator for improved RAM efficiency
Listing 13.14. Convert an embedding into a TensorBoard projection
Appendix A. Your NLP tools
Listing A.1. Install Anaconda3
Listing A.2. Install nlpia source with conda
Listing A.3. Install developer tools with apt
Appendix B. Playful Python and regular expressions
Appendix C. Vectors and matrices (linear algebra fundamentals)
Appendix D. Machine learning tools and techniques
Listing D.1. A dropout layer in Keras reduces overfitting
Listing D.2. BatchNormalization
Listing D.3. Count what the model got right
Appendix E. Setting up your AWS GPU
Appendix F. Locality sensitive hashing