There are three ways to approach the computation of derivatives:

• Numerical differentiation refers to the process of approximation of the derivative of a given function at a point. In SciPy, we have the following procedures, which will be covered in detail:

  • For generic univariate functions, there is the central difference formula with fixed spacing.

  • It is always possible to perform numerical differentiation via Cauchy's theorem, which transforms the derivative into a definite integral. This integral is then treated with the techniques of numerical integration explained before.

• Symbolic differentiation refers to computation of functional expressions of derivatives of functions, pretty much in the same way as we would do manually. It is termed symbolic because unlike its numerical counterpart, symbols take the roles of variables rather than numbers or vectors of numbers. To perform symbolic differentiation, we require a computer algebra system (CAS), and in the SciPy stack, this is achieved mainly through the SymPy library (see http://docs.sympy.org/latest/index.html). Symbolic differentiation and posterior evaluation are good substitutes for numerical differentiation for very basic functions. However, in general, this method leads to overcomplicated and inefficient code. The speed of purely numerical differentiation is preferred, in spite of the possible occurrence of errors.

• Automatic differentiation is another set of techniques to numerically evaluate the derivative of a function. It is not based on any approximation schema. This is without a doubt the most powerful option in the context of high derivatives of multivariate functions.