We have seen so far that neural networks show an amazing capability in learning through data in both supervised and unsupervised ways. In this chapter, we present an additional case of pattern recognition involving an example of Optical Character Recognition (OCR). Neural networks can be trained to strictly recognize text characters written in an image file. A brief review of classification and clusterization is covered prior to presenting the application itself. In this chapter, we will cover the following topics:
- Pattern Recognition
- Defined Classes
- Undefined Classes
- Neural Networks in Pattern Recognition
- Kohonen and MLP
- The OCR Problem
- Preprocessing and Class Definitions
- Implementation in Java
- Digit Recognition
Patterns are a bunch of data and elements that look similar to each other, and can occur systematically and repeat from time to time. Pattern recognition is a task that can be performed mainly by unsupervised learning using clusterization; however, when there is labeled data or defined classes of data, this task can be performed by supervised methods. We as humans perform this task more often than we can imagine. When we see objects and recognize them as belonging to a certain class, we are indeed recognizing a pattern. Also, when we analyze charts, discrete events, and time series, we might find evidence of some sequence of events that repeat systematically under certain conditions. In summary, patterns can be learned by data observations.
Examples of pattern recognition tasks include the following:
- Shape recognition
- Object classification
- Behavior clustering
- Voice recognition
- OCR
- Chemical reaction taxonomy
In a list of classes that has been predefined for a specific domain, each class is considered to be a pattern; therefore, every data record or occurrence is assigned one of these predefined classes.
One illustrated example for pattern recognition using defined classes is animal recognition by images, as shown in the following figure. The pattern recognizer however should be trained to catch all the characteristics that formally define the classes. In the example, eight figures of animals are shown, belonging to two classes: mammals and birds. Since this is a supervised mode of learning, the neural network should be provided with a sufficient number of images that allow it to properly classify new images.
Of course, sometimes, the classification may fail mainly due to similar hidden patterns in the images that neural networks may catch and due to the small nuances present in the shapes. For example, a dolphin has flippers, but it is still a mammal. Sometimes, in order to obtain a better classification, it is necessary to apply preprocessing and ensure that the neural network will receive the appropriate data that would allow for classification.
When data are unlabeled and there is no predefined set of classes, it is a scenario for unsupervised learning. Shape recognition is a good example since shapes may be flexible and have an infinite number of edges, vertices, or bindings, as shown in the following figure:
In the preceding image, we can see some types of shapes and we want to arrange them such that the similar ones can be grouped into the same cluster. Based on the shape information that is present in the images, it is likely for the pattern recognizer to classify the rectangle, the square, and the right triangle into the same group. However, if the information were presented to the pattern recognizer, not as an image, but as a graph with edges and vertices coordinates, the classification might have changed a little.
In summary, the pattern recognition task may use both supervised and unsupervised modes of learning, basically depending on the objective of recognition.
In some cases, there is already an expected result for clustering, as in the example of plant clustering. This is called external validation. One may apply a neural network with unsupervised learning to cluster data that is already assigned a value. The major difference against the classification lies in the fact that the target outputs are not considered, so the algorithm itself is expected to draw a borderline based only on the data.