In this chapter, we will show the learning process that neural networks perform in order to learn from data. We present the concepts of training, test, and validation, and show how to implement them in Java. We also show some methods for evaluating a neural network's performance in learning as well as learning algorithms' parameters. In summary, the following are the concepts addressed in this chapter:
- Learning process
- Learning algorithm
- Types of learning
- Supervised
- Unsupervised
- Training, test, and validation
- Error measurements
- Generalization
What is really amazing about neural networks is their capacity to learn from the environment, just like brain-gifted beings are able to. We, as humans, experience the learning process through observations and repetitions, until some task or concept is completely mastered. From the physiological point of view, the learning process in the human brain is a reconfiguration of the neural connections between the nodes (neurons), which results in a new thinking structure.
While the connectionist nature of neural networks distributes the learning process all over the entire structure, this feature makes this structure flexible enough to learn a wide variety of knowledge. As opposed to ordinary digital computers that can execute only those tasks that they are programmed to, neural systems are able to improve and perform new activities according to some satisfaction criteria. In other words, neural networks don't need to be programmed; they learn the program by themselves.
Considering that every task that requires solving solve may have a huge number of theoretically possible solutions, the learning process seeks to find an optimal solution that can produce a satisfying result. The use of structures like artificial neural networks (ANNs) is encouraged because of their ability to acquire knowledge of any type, strictly by receiving input stimuli, that is, data relevant to the task/problem. First, the ANN will produce a random result and an error, and based on this error, the ANN parameters will be adjusted.
Tip
We can then think of the ANN parameters (weights) as the components of a solution. Let's imagine that one single solution represents a single point in the solution hyperspace. Each single solution produces an error measure, which informs how far away that solution is from the optimal one. For each iteration, the learning algorithm seeks a solution that can yield a smaller error and therefore, be closer to the optimal one.