While using neural networks in regression problems (that include prediction), there is no fixed number of hidden neurons, so usually the solver chooses an arbitrary number of neurons and then varies it according to the results produced by the networks created. This procedure may be repeated a number of times until a network with a satisfying criterion is found.
Experiments can be made on the same training and test datasets, while varying other network parameters, such as learning rate, normalization, and the number of hidden units. The objective is to choose the neural network that presents the best performance from the experiments. The best performance is assigned to the network that presents a lower MSE error, but an analysis of generalization with test data is also useful.
The table below shows the experiments have that been run for all cities:
In order to facilitate the execution of experiments, we've designed a Java Swing Graphical User Interface (GUI), with which it is possible to select neural network parameters for training and the dataset.
Tip
This interface covers only neural networks with just one hidden layer; however, since the code is open, the implementation of a multilayer perceptron with multiple hidden layers is suggested as an exercise, as well as the choice of other algorithms for the training.
The charts show only the predicted maximum temperature; therefore, implementing an option for displaying the minimum temperature is also suggested.
After selecting the parameters, training begins when you click the Start Training button:
After running 12 experiments, we found the following MSE training errors for each dataset:
|
Experiment |
Cruzeiro do Sul |
Picos |
Campos do Jordão |
Porto Alegre |
|---|---|---|---|---|
|
#1 |
0.130156 |
0.147111 |
0.300437 |
0.323342 |
|
#2 |
0.512389 |
0.572588 |
0.428692 |
0.478379 |
|
#3 |
0.08659 |
0.094822 |
0.124752 |
0.114486 |
|
#4 |
0.360728 |
0.258596 |
0.168351 |
0.192012 |
|
#5 |
0.076476 |
0.074777 |
0.108991 |
0.085029 |
|
#6 |
0.328493 |
0.186793 |
0.152499 |
0.151248 |
|
#7 |
0.146801 |
0.130004 |
0.277765 |
0.19076 |
|
#8 |
0.431811 |
0.29629 |
0.364418 |
0.278864 |
|
#9 |
0.071135 |
0.081159 |
0.117634 |
0.091174 |
|
#10 |
0.332534 |
0.210107 |
0.170179 |
0.164179 |
|
#11 |
0.07247 |
0.089069 |
0.102137 |
0.076578 |
|
#12 |
0.33342 |
0.19835 |
0.155036 |
0.145843 |
The MSE error information only gives us an idea of how much the neural network output could match real data in the overall context. This performance can be verified by viewing the time series comparison and scatter plots:
These charts show that, although in many cases the temperature cannot be accurately predicted, a trend is being followed. This can be attested to by the correlation visible in the scatter plots. The last row of the table, showing the prediction for Porto Alegre, which has a subtropical climate and high temperature variations, shows a good prediction even for the extreme temperature variations. However, we remind the reader that forecasting weather needs to consider many additional variables, which could not be included in this example due to availability constraints. Anyway, the results show we've made a good start to search for a neural network configuration that can outperform these ones found.