Physically Random and Cryptographically Strong Random Numbers

Pseudorandom numbers provided by the random module, while very good, are not of cryptographic quality. If you want higher-quality random numbers, you can call os.urandom (from the module os, not random), or instantiate the class SystemRandom from random (which calls os.urandom for you).

An alternative source of physically random numbers: http://www.fourmilab.ch/hotbits.

The random Module

All functions of the random module are methods of one hidden global instance of the class random.Random. You can instantiate Random explicitly to get multiple generators that do not share state. Explicit instantiation is advisable if you require random numbers in multiple threads (threads are covered in Chapter 14). Alternatively, instantiate SystemRandom if you require higher-quality random numbers. (See “Physically Random and Cryptographically Strong Random Numbers”.) This section documents the most frequently used functions exposed by module random:

The random module also supplies several other functions that generate pseudorandom floating-point numbers from other probability distributions (Beta, Gamma, exponential, Gauss, Pareto, etc.) by internally calling random.random as their source of randomness.

NumPy

If you need a lightweight one-dimensional array of numbers, the standard library’s array module may often suffice. If you are doing scientific computing, advanced image handling, multidimensional arrays, linear algebra, or other applications involving large amounts of data, the popular third-party NumPy package meets your needs. Extensive documentation is available online; a free PDF of Travis Oliphant’s Guide to NumPy book is also available.

NumPy provides class ndarray, which you can subclass to add functionality for your particular needs. An ndarray object has n dimensions of homogenous items (items can include containers of heterogenous types). An ndarray object a has a number of dimensions (AKA axes) known as its rank. A scalar (i.e., a single number) has rank 0, a vector has rank 1, a matrix has rank 2, and so forth. An ndarray object also has a shape, which can be accessed as property shape. For example, for a matrix m with 2 columns and 3 rows, m.shape is (3,2).

NumPy supports a wider range of numeric types (instances of dtype) than Python; however, the default numerical types are: bool_, one byte; int_, either int64 or int32 (depending on your platform); float_, short for float64; and complex_, short for complex128.

Creating a NumPy Array

There are several ways to create an array in NumPy; among the most common are:

• with the factory function np.array, from a sequence (often a nested one), with type inference or by explicitly specifying dtype

• with factory functions zeros, ones, empty, which default to dtype float64, and indices, which defaults to int64

• with factory function arange (with the usual start, stop, stride), or with factory function linspace (start, stop, quantity) for better floating-point behavior

• reading data from files with other np functions (e.g., CSV with genfromtxt)

Here are examples of creating an array, as just listed:

import numpy as np

np.array([1, 2, 3, 4])  # from a Python list
array([1, 2, 3, 4])

np.array(5, 6, 7)  # a common error: passing items separately (they
                   # must be passed as a sequence, e.g. a list)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: only 2 non-keyword arguments accepted

s = 'alph', 'abet'  # a tuple of two strings
np.array(s)
array(['alph', 'abet'], dtype='<U4')

t = [(1,2), (3,4), (0,1)]  # a list of tuples
np.array(t, dtype='float64')  # explicit type designation
array([[ 1.,  2.],
       [ 3.,  4.],
       [ 0.,  1.]]

x = np.array(1.2, dtype=np.float16)  # a scalar
x.shape
()
x.max()
1.2002

np.zeros(3)  # shape defaults to a vector
array([ 0.,  0.,  0.])

np.ones((2,2))  # with shape specified
array([[ 1.,  1.],
       [ 1.,  1.]])

np.empty(9)  # arbitrary float64s
array([  4.94065646e-324,   9.88131292e-324,   1.48219694e-323,
         1.97626258e-323,   2.47032823e-323,   2.96439388e-323,
         3.45845952e-323,   3.95252517e-323,   4.44659081e-323])

np.indices((3,3))
array([[[0, 0, 0],
        [1, 1, 1],
        [2, 2, 2]],

       [[0, 1, 2],
        [0, 1, 2],
        [0, 1, 2]]])

np.arange(0, 10, 2)  # upper bound excluded
array([0, 2, 4, 6, 8])

np.linspace(0, 1, 5)  # default: endpoint included
array([ 0.  ,  0.25,  0.5 ,  0.75,  1.  ]) 
 
np.linspace(0, 1, 5, endpoint=False)  # endpoint not included
array([ 0. ,  0.2,  0.4,  0.6,  0.8])

import io
np.genfromtxt(io.BytesIO(b'1 2 3
4 5 6'))  # using a pseudo-file
array([[ 1.,  2.,  3.],
       [ 4.,  5.,  6.]])

with io.open('x.csv', 'wb') as f:
    f.write(b'2,4,6
1,3,5')
np.genfromtxt('x.csv', delimiter=',')  # using an actual CSV file
array([[ 2.,  4.,  6.],
       [ 1.,  3.,  5.]])

Shape, Indexing, and Slicing

Each ndarray object a has an attribute a.shape, which is a tuple of ints. len(a.shape) is a’s rank; for example, a one-dimensional array of numbers (also known as a vector) has rank 1, and a.shape has just one item. More generally, each item of a.shape is the length of the corresponding dimension of a. a’s number of elements, known as its size, is the product of all items of a.shape (also available as property a.size). Each dimension of a is also known as an axis. Axis indices are from 0 and up, as usual in Python. Negative axis indices are allowed and count from the right, so -1 is the last (rightmost) axis.

Each array a (except a scalar, meaning an array of rank-0) is a Python sequence. Each item a[i] of a is a subarray of a, meaning it is an array with a rank one less than a’s: a[i].shape==a.shape[1:]. For example, if a is a two-dimensional matrix (a is of rank 2), a[i], for any valid index i, is a one-dimensional subarray of a that corresponds to a row of the matrix. When a’s rank is 1 or 0, a’s items are a’s elements (just one element, for rank-0 arrays). Since a is a sequence, you can index a with normal indexing syntax to access or change a’s items. Note that a’s items are a’s subarrays; only for an array of rank 1 or 0 are the array’s items the same thing as the array’s elements.

As for any other sequence, you can also slice a: after b=a[i:j], b has the same rank as a, and b.shape equals a.shape except that b.shape[0] is the length of the slice i:j (j-i when a.shape[0]>j>=i>=0, and so on).

Once you have an array a, you can call a.reshape (or, equivalently, np.reshape with a as the first argument). The resulting shape must match a.size: when a.size is 12, you can call a.reshape(3,4) or a.reshape(2,6), but a.reshape(2,5) raises ValueError. Note that reshape does not work in place: you must explicitly bind or rebind the array—that is, a = a.reshape(i,j) or b = a.reshape(i,j).

You can also loop on (nonscalar) a in a for, just as you can with any other sequence. For example:

for x in a:
    process(x)

means the same thing as:

for _ in range(len(a)):
    x = a[_]
    process(x)

In these examples, each item x of a in the for loop is a subarray of a. For example, if a is a two-dimensional matrix, each x in either of these loops is a one-dimensional subarray of a that corresponds to a row of the matrix.

You can also index or slice a by a tuple. For example, when a’s rank is >=2, you can write a[i][j] as a[i,j], for any valid i and j, for rebinding as well as for access; tuple indexing is faster and more convenient. Do not put parentheses inside the brackets to indicate that you are indexing a by a tuple: just write the indices one after the other, separated by commas. a[i,j] means the same thing as a[(i,j)], but the form without parentheses is more readable.

An indexing is a slicing when one or more of the tuple’s items are slices, or (at most once per slicing) the special form ... (also available, in v3 only, as Python built-in Ellipsis). ... expands into as many all-axis slices (:) as needed to “fill” the rank of the array you’re slicing. For example, a[1,...,2] is like a[1,:,:,2] when a’s rank is 4, but like a[1,:,:,:,:,2] when a’s rank is 6.

The following snippets show looping, indexing, and slicing:

a = np.arange(8)
a
array([0, 1, 2, 3, 4, 5, 6, 7])

a = a.reshape(2,4)
a
array([[0, 1, 2, 3],
       [4, 5, 6, 7]])
          
print(a[1,2])
6

a[:,:2]
array([[0, 1],
       [4, 5]])

for row in a:
    print(row)
[0 1 2 3]
[4 5 6 7]

for row in a:
    for col in row[:2]:  # first two items in each row
        print(col)
0
1
4
5

Matrix Operations in NumPy

As mentioned in “The operator Module”, NumPy implements the new³ operator @ for matrix multiplication of arrays. a1 @ a2 is like np.matmul(a1,a2). When both matrices are two-dimensional, they’re treated as conventional matrices. When one argument is a vector, you promote it to a two-dimensional array, by temporarily appending or prepending a 1, as needed, to its shape. Do not use @ with a scalar; use * instead (see the following example). Matrices also allow addition (using +) with a scalar (see example), as well as with vectors and other matrices (shapes must be compatible). Dot product is also available for matrices, using np.dot(a1, a2). A few simple examples of these operators follow:

a = np.arange(6).reshape(2,3)  # a 2-d matrix
b = np.arange(3)               # a vector

a
array([[0, 1, 2],
       [3, 4, 5]])

a + 1    # adding a scalar
array([[1, 2, 3],
       [4, 5, 6]])

a + b    # adding a vector
array([[0, 2, 4],
       [3, 5, 7]])

a * 2    # multiplying by a scalar
array([[ 0,  2,  4],
       [ 6,  8, 10]])

a * b    # multiplying by a vector
array([[ 0,  1,  4],
       [ 0,  4, 10]])

a @ b    # matrix-multiplying by vector
array([ 5, 14])

c = (a*2).reshape(3,2)  # using scalar multiplication to create
c                       # another matrix
array([[ 0,  2],
       [ 4,  6],
       [ 8, 10]])

a @ c    # matrix multiplying two 2-d matrices
array([[20, 26],
       [56, 80]])

NumPy is rich enough to warrant books of its own; we have only touched on a few details. See the NumPy documentation for extensive coverage of its many features.

SciPy

NumPy contains classes and methods for handling arrays; the SciPy library supports more advanced numeric computation. For example, while NumPy provides a few linear algebra methods, SciPy provides many more functions, including advanced decomposition methods, and also more advanced functions, such as allowing a second matrix argument for solving generalized eigenvalue problems. In general, when you are doing advanced numerical computation, it’s a good idea to install both SciPy and NumPy.

SciPy.org also hosts the documentation for a number of other packages, which are integrated with SciPy and NumPy: Matplotlib, which provides 2D plotting support; Sympy, which supports symbolic mathematics; IPython, a powerful interactive console shell and web-application kernel (the latter now blossoming as the Jupyter project); and pandas, which supports data analysis and modeling (you can find the pandas tutorials here, and many books and other materials here). Finally, if you’re interested in Deep Learning, consider using the open source TensorFlow, which has a Python API.