Theano is a Python library created by a machine learning group in Montreal and is often associated with deep learning, although that is not necessarily its core purpose. Theano is tightly integrated with NumPy and can run code on CPU or GPU. If you are interested in the GPU option, refer to the documentation listed in the See also section. Theano also supports symbolic differentiation through symbolic variables.

According to the its documentation, Theano is a cross between NumPy and SymPy. It is possible to implement machine learning algorithms with Theano, but it's not as easy or convenient as using scikit-learn. However, you may get the potential advantages of higher parallelism and numerical stability.

In this recipe, we will perform linear regression of temperature data using gradient descent. Gradient descent is an optimization algorithm that we can use in a regression context to minimize fit residuals. The gradient measures how steep a function is. The algorithm takes many steps proportional to how steep the gradient is in order to find a local minimum. We are trying to go downhill, but we don't know in which direction we can find a local minimum. So, going for a large move down should on average get us down faster, but there is no guarantee. In some cases, it may help to smooth the function (smoother hill), so we don't spend a lot of time oscillating.

$ pip install --no-deps git+git://github.com/Theano/Theano.git 

How to do it...

The code is in the theano_tour.ipynb file in this book's code bundle:

1. The imports are as follows:

   import theano
   import numpy as np
   import theano.tensor as T
   import ch9util
   from sklearn.cross_validation import train_test_split
   from sklearn.metrics import r2_score
   import dautil as dl
   from IPython.display import HTML

2. Load the temperature data and define Theano symbolic variables:

   temp = dl.data.Weather.load()['TEMP'].dropna()
   X = temp.values[:-1]
   y = temp.values[1:]
   X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=16)
   w = theano.shared(0., name ='w')
   c = theano.shared(0., name ='c')
   
   x = T.vector('x')
   y = T.vector('y')

3. Define prediction and cost (loss) functions to minimize:

   prediction = T.dot(x, w) + c
   cost = T.sum(T.pow(prediction - y, 2))/(2 * X_train.shape[0])
   Define gradient functions as follows:
   gw = T.grad(cost, w)
   gc = T.grad(cost, c)
   
   learning_rate = 0.01
   training_steps = 10000

4. Define the training function as follows:

   train = theano.function([x, y], cost, updates =
                           [(w, w - learning_rate * gw),
                            (c, c - learning_rate * gc)])
   predict = theano.function([x], prediction)

5. Train the estimator as follows:

   for i in range(training_steps):
       train(X_train.astype(np.float), y_train)

6. Predict and visualize the prediction as follows:

   preds = predict(X_test)
   r2 = r2_score(preds, y_test)
   HTML(ch9util.scatter_predictions(preds, y_test, '', r2))

Refer to the following screenshot for the end result: