Chapter 5

Nonlinear dynamics and chaos

We have thus far dealt with systems whose time evolution is described by a set of ODEs. Given initial conditions, we can integrate the ODEs and obtain numerical solutions at any point in the future. When the outcome is uniquely determined by the initial condition, we say the system is deterministic. Newtonian systems such as projectile and planetary motion are deterministic. For the most part (except unstable few-body systems alluded to in Section 4.5), we did not have to worry about any unpredictable results, in the sense that if we propagate the system from two nearby initial conditions, we should expect the final results to be close, at least not catastrophically different.

Starting with Poincaré's work on celestial mechanics, that notion of predictability had been challenged. But the field has rapidly accelerated in the last half century with what Edward Lorenz thought was a computation error when the computer generated results showed a totally unexpected outcome. Since then, numerical computation has led to the discovery of an entirely new field of science, deterministic chaos, often considered a glory of computational science. The field of chaos covers everything from physical systems to biological ones, from surprisingly simple models to complex ones [35]. When chaos occurs, the system becomes unpredictable and loses long term predictability, in the sense that if we start the evolution of the same system (same ODEs) with slightly different initial values, the outcomes are totally different. The so-called butterfly effect, stemming from the seminal work of Lorenz on hydrodynamic flows, refers to extreme sensitivity to initial conditions in weather in which a small disturbance may cause a storm half a world away. It is qualitatively different than a coin toss where random outcomes are a result of not catastrophic difference but the lack of precise initial conditions, assuming negligible air resistance (e.g., in vacuum).

Because these systems require knowledge of ODEs only, we can apply the well-tested techniques covered so far to study them before turning attention to systems described by partial differential equations starting next chapter.

A necessary ingredient for chaos to occur is nonlinearity in the system dynamics. Broadly speaking, if a system responds to input in a nonlinear (or disproportionate) fashion, we say it exhibits nonlinear dynamics. In this chapter, we will study nonlinear dynamics and chaos in systems from discrete logistic maps to continuous systems including driven nonlinear oscillators and the Lorenz model. We will examine how chaos develops, how to characterize it, and how to predict the unpredictable.

5.1 A first model: the logistic map

We begin with the simplest system exhibiting chaotic behavior, the so-called logistic map. We will explore this system in greater detail, for it helps us introduce several important concepts with clarity.

The logistic map is defined as

This is a discrete, nonlinear system where given the value x_n at stage n, its next value x_n+1 can be calculated (mapped). Here r is called the control parameter.¹ If we introduce the map function as

then Eq. (5.1) can be written as

We speak of this equation as a 1D map, which maps an existing point into a new one, deterministically. The map function f(x) is shown in Figure 5.1. Since the maximum of f(x) is at x = 1/2 with f_max = r, the series will remain bounded according to Eq. (5.1).

What could the logistic map represent, you may wonder? One way is to view it as the cycles of an insect population. For example, we could think of x_n as representing the scaled population (in some unit) of a colony of bugs at time n. The population in the next cycle, x_n+1, would crudely depend on two things: the number of existing bugs (x_n) for reproduction, and the amount of resources available (e.g., food), 1−x_n, where we assume the resource decreases at a rate proportional to the bug population. Then r would be a combination of birth rate and consumption rate. The detail of the model does not matter, but the nonlinear term in Eq. (5.1) does.

5.1.1 Map iteration

For a given control parameter r and initial value x₀, a series of map iterates may be determined as

We will use the short hand f⁽ⁿ⁾ to denote the nth iterate of the map as

We can explore the map iterates using Program 5.1. The program first reads the input for x₀ and r from the user, then calculates the map iterates up to a specified number of steps. At each step, the value x_n is also format-printed to the console, four values per line, controlled by the end variable (line 8). Like in Program 4.3, the format string “’%8f’ %x” prints 8 digits of the float x. The inline if-else statement produces either a space, or a new line (’ ’) every four steps. The results are plotted at the end.

from _future_ import print_function              # use print() as function
  2import matplotlib.pyplot as plt       # get matplotlib plot functions

  4x, r = input(’enter x0, r: ’)
n, xn = 40, []
  6for i in range(n):
       xn.append(x)                              # new line every 4 steps
  8      print(’%8f’ %x, end=’
’ if len(xn)%4==0 else ’ ’) # inline if
       x = 4*r*x*(1.0−x)                     # next iteration,   logistic  map
10
plt.figure()
12plt.plot(range(n), xn, ’--o’)           # plot dashed line with 'o’ symbol
plt.xlabel(’$n$’), plt.ylabel(’$x_n$’)
14plt .ylim (0,1), plt .text(33, .1, ’r=’+ repr(r))     # add text
plt.show()

We show in Figure 5.2 representative plots for several values of the control parameter r. For all values of r except the last, x_n (bugs population) undergoes an initial transient phase, then settles into a periodic pattern. For the smallest r = 0.2, the population quickly drops to zero and stays there. Too low a birth rate or too little resource cause the colony of bugs to go extinct. Increasing r to 0.7 yields a stable population. In both cases the system is in a period one cycle.

For the next three larger r values, the population repeats after two, four, and eight steps, respectively. We speak of period two, four, and eight cycles. For example, the first 24 points for r = 0.88 and x₀ = 0.8 as printed from the program are

0.800000 0.563200 0.865940 0.408629
0.850613 0.447289 0.870220 0.397539
0.843046 0.465764 0.875874 0.382689
0.831558 0.493043 0.879830 0.372168
0.822479 0.513945 0.879315 0.373542
0.823709 0.511148 0.879563 0.372882

We can see that after initial transients, the series quickly converges to a cycle of period four, i.e., the same value appearing after every four steps.

Here interesting dynamics is starting to emerge for increasing r, as cyclic populations start to appear. These periodic values are known as fixed points. Without the nonlinear term in the logistic map (5.1), the only fixed point would be zero. Increasing r further to 0.95, no apparent periodic structure is recognizable. As we will soon discover, the logistic map has entered the chaotic region.

5.1.2 Fixed points and attractors

As we have just seen, only fixed points appear to remain after transients. Let x* represent the fixed points. For period-one cycles, the value repeats after one step, so we have

We can rewrite the above relation using the map function (5.2) and (5.3) to get

Solving the above equation gives us two fixed points for period-one cycles as

Similarly, period two cycles repeat every two steps,

Correspondingly, the fixed points are given by two successive applications of the map function like in Eq. (5.4), resulting in a fourth-order (quartic) equation. The four roots are the fixed points of the period-two cycle,²

The first two fixed points are also those of the period-one cycle, because if a value repeats after one step, it automatically repeats after two steps.

For higher period cycles, it gets progressively messy to find the fixed points algebraically, and it would be easier to do so numerically. We can also locate the fixed points graphically, as illustrated in Figure 5.3 for period one and two (see Exercise E5.2).

The map functions are drawn as curves for f(x) and f⁽²⁾(x), respectively, for period one and two. Note the double hump in the period-two curve. The intersections of the diagonal lines and these curves are the fixed points: 2 for period one, 4 for period two, both including the point 0.

Figure 5.3 also demonstrates how the fixed points are attracted to, starting from x₀ = 0.1 in both cases. For period one, the map iterates converge to the larger fixed point ~ 0.64 instead of 0, though the latter is closer to the starting value. For period two, something more interesting happens: the iteration marches toward a fixed point (~ 0.7) quickly at first but does not stay there. Rather, it moves away and ends up at another fixed point (~ 0.47). This explains the appearance of these two points in Figure 5.2 for the corresponding r values.

The fixed points that the iterates eventually converge to are called attractors. All the periodic points observed in Figure 5.2 after the initial transients are attractors. However, not all fixed points are attractors. It begs the question: What is it that leads the iterates toward some fixed points but away from others? What makes a fixed point an attractor? The answer lies in the stability of fixed points.

5.1.3 Stability of fixed points

For convenience, let us formally introduce the (compound) map function of period n as

where f⁽ⁿ⁾ represents n successive operations defined in Eq. (5.5). It follows that for the fixed points of period n, x*, we have

For stability analysis, let us pick a point x from the small neighborhood surrounding a fixed point x*. As illustrated in Figure 5.4, if x starts at a distance ∈ away from x*, it will be mapped in the next step to x_n = g_n(x) which is at a distance ∈_n from the fixed point. When projected on the x-axis via the projection line y = x (similar to Figure 5.3), x_n will be at a horizontal distance ∈_n from x*.

The distances are related to the local properties of the map function g_n(x). Graphically from Figure 5.4, we see that ∈_n = h = ∈| tan θ|. For small ∈, , and we have

We can get the same result mathematically (Exercise E5.3).

If a small deviation leads back to the fixed point, i.e., if ∈_n < ∈, the fixed point will be stable. Conversely, if ∈_n > ∈, the fixed point is unstable, because points in that neighborhood will move away from it. Therefore, the condition for the stability of a fixed point depends on as,

If the derivative is zero, , the fixed point is said to be super-stable because the convergence is much faster than the exponential rate in normal cases.

The calculation of can be carried out numerically, or analytically by chain rule with the aid of Eq. (5.12),

For example, the derivatives for period one at the fixed points (5.8) are

According to Eq. (5.14), the first fixed point (x* = 0) is stable, i.e., 1, for r < 1/4, and the second one is stable for 1/4 < r < 3/4. It is for this reason that in Figure 5.2 the population drops to zero for r = 0.2 and reaches a finite value for r = 0.7 (Figures 5.2 and 5.3).

By the same token, only the last two fixed points of period two in Eq. (5.10) are stable (Exercise E5.4), as the first two are from period one (0 and ~ 0.7 in Figure 5.3, right panel), which as we have seen are unstable at r > 3/4.

The regions of stability of the fixed points are summarized in Figure 5.5 over the full range of the control parameter. At a given r, one of the fixed points of period one is stable until r = r₁ = 3/4 when both become unstable. Filling the void are two, new fixed points from period two, which double the number of stable fixed points.

However, the new stable fixed points themselves become unstable at r = r₂ ~ 0.862 (see Exercise E5.4). What then? Will period doubling continue? Figure 5.2 suggests yes. We will discuss how next.

5.1.4 Period doubling and route to complexity

To explore the region beyond r₁ in Figure 5.5, we map the stable fixed points (attractors) as a function of r in small increments with Program 5.2.

import matplotlib.pyplot as plt     # get matplotlib plot functions
  2ntrans, nsteps = 1000, 200            # num. of transients and to keep
r, rend, dr, x, xa = 0.7, 1.0, 0.001, 0.5, [0.0]* nsteps
  4plt.figure()
while r <= rend:
  6     for i in range(ntrans): x = 4*r*x*(1−x)     # discard transients
     for i in range(nsteps): xa[i ], x = x, 4*r*x*(1−x) # keep rest
  8      plt.plot ([r]*nsteps, xa, 'b,’)                     # blue pixel markers
      r = r + dr
10plt.xlabel('r’), plt.ylabel('x’), plt.show()

For each r, we iterate the map for ntrans steps and discard these points to get rid of transients. The number of steps it takes for the transients to pass would depend on r. For simplicity we use a fixed, large value, as it is adequate for our purpose. The next nsteps iterates are presumably the attractors. They are retained and plotted.

The result is a period doubling diagram in Figure 5.6. It shows one fixed point belonging to period one up to r₁ = 0.75, where it becomes unstable, and is taken over by two stable fixed points of period two, in accordance with earlier observations (Figure 5.5). They remain stable to r₂ ~ 0.862, which is the onset of period four. Period doubling, or bifurcation, continues at closer and closer values of r, e.g., r₃ ~ 0.886 for period eight, and r₄ ~ 0.891 for period sixteen (barely visible in Figure 5.6).

Further period doubling (16, 32, etc., not noticeable at the scale shown in Figure 5.6) occurs so close to each other and so fast that, within a narrow window around r ~ 0.9, the period becomes infinite. Subsequently, no apparent structure remains other than whole bands being filled. It shows that period doubling is the route to complexity. Although our example is the logistic map, it turns out to be universal of nonlinear systems. As we will see shortly, the logistic map becomes chaotic for r > 0.9.

Figure 5.6 shows the static structure after initial transients have been discarded. The dynamics of period doubling may be more fully appreciated in a 3D representation which also includes temporal changes. Figure 5.7 shows a snapshot from a VPython animation (Exercise E5.5). With time (n) increasing out of the page, the initial transients give way to steady “finger” sheets of bifurcation. At a given r, the series of points bounce alternately between the sheets but never successively between adjacent fingers. For instance, if we label the four branches of the period-4 cycle as 1 to 4 from top to bottom, the sequence is 1 → 4 → 2 → 3…, without direct jumps between adjacent pairs (1,2) and (3,4). As we will see later (Figure 5.8), branch 3 is closest to . If we start from there, we will arrive at the adjacent finger (branch 4) two steps later, i.e., half a cycle. This is true for higher cycles. This observation is important for our discussion of universal features including the Feigenbaum constants next.

5.1.5 Universal features

It turns out that several features in the logistic map are universal to a large family of nonlinear maps. One universal constant known as the Feigenbaum δ-number [28] can be estimated from Figure 5.6. Slightly beyond r₄, the logistic map has reached the limit of period doubling series, i.e., the order of periodic doubling becomes infinite. Let us denote that limit as r_∞ 0.8924864 (see Project P5.2 and Ch. 18 of Ref. [4]). The r_n value for the onset of period 2ⁿ is related to r_∞ and δ as

where c is some constant. Eliminating c gives

This says that the Feigenbaum number is the ratio of adjacent distances in r between successive bifurcations. Using the four approximate r values stated earlier, we find the ratio to be roughly δ ~ 4.7. More accurate evaluation of δ requires more precise determination of the r value. The exact Feigenbaum number is δ = 4.6692016… (see Ref. [12], Project P5.2).

Self-similar diagrams at different scales can be seen in Figure 5.6 in the magnification of selected regions. The boxed region in the original, upper figure is magnified and shown in the lower-left figure. The magnified region is very similar to the original diagram. We can enlarged a subregion in the already-magnified diagram, which again shows strong resemblance to its predecessor. This phenomenon is known as self similarity, i.e., the overall structure, including period doubling and onset of chaos, is replicated on the finer and finer scale.

It seems clear from Figure 5.6 that the successive bifurcations are copies spawned from prior ones, only shifted in location and shrunk in scale. Intuitively, we expect to be able to rescale, or renormalize, one bifurcation and reproduce it universally. Indeed, self-similarity is a universal phenomenon in nonlinear dynamics, and can be better understood with renormalization theory discussed in Section S:5.A. For example, the approximate r_n values and the δ number can be estimated from renormalization (Project S:P5.1). The beauty of the logistic map is that it exhibits these universal features while keeping things simple and accessible [19].

Feigenbaum's α number

Another result of renormalization is that we can estimate the second Feigenbaum number, the α-number which measures the scaling in the vertical direction in the period doubling diagram (Figure 5.6).

The α-number is a constant defined as the limiting ratio

where d_n is the vertical displacement from the fixed point x* = 1/2 to the nearest branch of the period-2ⁿ cycle in the bifurcation diagram (see Figure 5.8). The negative sign in Eq. (5.19) refers to the displacement d_n flipping directions between the cycles.

Let be the value of the control parameter of the period-2ⁿ cycle where one of the fixed points is x* = 1/2. The map function is an extremum at this point (see Figure 5.1 or 5.3), and its derivative is zero. Therefore, this fixed point is superstable, hence the r* designation. For example, the first two r values for the superstables are and (Project P5.2). In general, for the period-2ⁿ cycle satisfies

It is understood that f⁽²ⁿ⁾ depends implicitly on .

The vertical displacement d_n starts from x* and ends at the nearest branch point of the bifurcation tree, which occurs exactly at the halfway point through the cycle per our discussion of Figure 5.7. In other words, the displacement is given by

Knowing d_n, we can calculate α from Eq. (5.19). There is an easier, approximate way, from renormalization where we obtain α = −2.24, compared to the exact value ~ −2.50 in Eq. (5.19) (see Project S:P5.2).

5.2 Chaos

As we have seen above, orderly period doubling ceases to exist for r ≥ r_∞ and the bifurcation diagram looks rather “chaotic”. How does the abrupt change happen? Can we quantify it? Earlier we gave a general definition of chaos as the sensitive dependence on initial conditions and loss of long term predictability. With the logistic map detailed so far, we are in a position to demonstrate this, and in the process to give an operational definition of chaos.

5.2.1 Sensitivity to initial conditions

Suppose we start the logistic map with two initial values differing by a small amount, and iterate the two series in parallel. How do the series compare? Program 5.3 (Appendix 5.A) lets us explore this question.

The heart of the program iterates the two series separately and simultaneously, and records their differences as follows.

x2 = x1 + 0.01                      # initial difference
for i in range(n):
      xn1.append(x1), xn2.append(x2), diff.append(abs(x1−x2))
      x1, x2 = 4*r*x1*(1.0−x1), 4*r*x2*(1.0−x2)       # parallel iterates

Results generated from the program with different control parameters are shown in Figure 5.9 with a small difference in the initial values. For the control parameter r = 0.8, the initial difference is rapidly reduced. The two series overlap each other, and are practically identical after a couple of steps, yielding the period-2 cycle as first seen in Figure 5.2.

The situation is quite different for the larger r = 0.9 case, where the difference between them grows quickly. By around n = 5, the two series are so far apart that they are no longer recognizable as coming from two nearby initial values. In other words, they become unpredictable. We note that the system is still deterministic, but something happens at this r that makes it very sensitive to the initial conditions.

We can get a better idea of what is happening here by studying the difference between the series. It is shown in Figure 5.10 for a small but arbitrary initial difference, produced with the same program.

While there are some scatter in the data, the overall trend is that the initial difference shrinks exponentially (note the semilog scale) for r = 0.8, but grows exponentially for r = 0.9. In the former case, the difference becomes insignificant, and we can predict where the system will be even if the initial conditions are not known precisely. However, in the latter case, any difference in the initial conditions, no matter how small, will lead to such exponentially diverging outcomes that any long term prediction is meaningless. It is for this reason that we saw the different behaviors in Figure 5.9. We stress that it is not due to inaccuracies in numerics, it is a characteristic of nonlinear systems in the chaotic regime.

5.2.2 A measure of chaos

We are now ready to discuss a more precise definition and a quantitative measure of deterministic chaos. Chaos is characterized by extreme sensitivity to initial conditions. Figure 5.10 suggests that the separation between nearby initial points will shrink or grow exponentially. Let us denote two series differing slightly in the initial value by ∈,

To first order, we expect that the separation will behave as

where c is some proportional constant. The factor λ is called the Lyapunov exponent. Accordingly, systems can be classified as

The meaning of λ refers to the average rate of exponential convergence or divergence of nearby trajectories. We can use λ as a quantitative measure of chaoticity, and give an operational definition of chaos: a system is chaotic if the Lyapunov exponent is positive. The more positive the λ, the more chaotic, or disorder. Conversely, the more negative the λ, the more predictable the system.

Note that the Lyapunov exponent should not depend on the initial separation in the limit ∈ → 0. In fact, given the values of Δx₁ and Δx₂ at t₂ and t₂, we can eliminate c∈ and obtain λ from Eq. (5.23),

Accurate determination of λ requires an infinitesimal initial separation and a very long evolution time.

In most cases, λ can be determined only numerically. For example, using the data in Figure 5.10 and Eq. (5.25), we can estimate that λ ~ (ln 10⁻¹³ − ln 10⁻³)/28 = −0.8 for r = 0.8, and λ ~ +0.2 for r = 0.9. These estimates are accurate only to their signs, and their magnitudes are crude because, numerically, we start with a finite initial separation and end in finite evolution time. In practice, the separation is limited by the range of the system, |Δx| < 1 for the logistic map, and cannot grow indefinitely in Eq. (5.23). As a result, it is important that we measure the slope (such as in Figure 5.10) before the separation levels off, and preferably average it over some initial conditions.

5.2.3 Lyapunov exponent of the logistic map

It turns out that the Lyapunov exponent of the logistic map can also be determined analytically. From Eqs. (5.22), (5.23), and (5.11), we have the separation of two nearby points after n iterations

Solving for λ yields

In the limit of small ∈ → 0 and large n 1, the argument of the logarithm function becomes a derivative at x₀, and the second term is negligible,

Recalling the chain rule (5.15), we have

To use Eq. (5.29), we choose an initial x₀, wait some steps for transients to pass, generate g_i recursively from Eq. (5.12) and calculate f′(g_i) with Eq. (5.15), for some suitably large n. You may be worried that λ obtained this way may depend on the initial value x₀. The dependence is very weak, if we take care to let the transients pass, and if n is sufficiently large. Essentially, the summation in Eq. (5.29) will wash out any trace of x₀, in both regular and the chaotic regions.

Program 5.4 (Appendix 5.A) calculates and plots λ. Experiment with ntrans (as well as x₀) a little, as sometimes it takes a while for the transients to pass. When λ is independent of all the parameters for a given r (say to within 3 or 4 digits), we know it has converged.

The results as a function of r are shown in Figure 5.11. For the control parameter up to r ~ 0.892, the Lyapunov exponent is negative, so the dynamics is regular. Bifurcations occur at the cusps when λ comes up to zero (but not crossing zero) and then abruptly falls back down. The locations of the cusps are the same as the vertical lines in Figure 5.6. Afterwards, λ is largely positive and increasing, signaling increasing disorder or chaoticity. But embedded in the land of chaos are narrow windows of stable islands where λ dips down to negative values. One of these contains the stable fixed points of period three marked by the arrow. There are other regions embedded as well, including period five, six, seven, etc. (See Project P5.1.)

5.3 A nonlinear driven oscillator

The logistic map lets us explore the concept of chaos in a simple model system. We now discuss a real, continuous physical system, the simple pendulum, a classic apparatus (Figure 5.12) made up of a weight connected to a pivot by a thin rod. It oscillates periodically about the equilibrium, the vertical line. This system exhibits chaos that can be experimentally measured.

The equation of motion for a damped pendulum is

where ω₀ is the natural angular frequency dependent on the rod length and gravitational acceleration, b the damping coefficient due to viscous friction or linear drag. The sin θ term comes from the tangential gravitational force. Without damping and in the small angle approximation, Eq. (5.30) reduces to the simple harmonic oscillator (2.47).³ However, the nonlinearity present in this term is crucial for nonlinear dynamics.

With damping, the pendulum would stop moving unless an external driving force is applied. We then speak of a nonlinear, driven oscillator. Including the driving force, and converting to a set of first-order ODEs, we have the equations of motion

where F_d is the driving force amplitude, and ω_d the driving angular frequency.

We will assume scaled units for F_d and b according to the units of time or frequency. If time is in seconds, then F_d will be in 1/s², and b in 1/s. Their units will be omitted unless otherwise noted.

In comparison to the logistic map, F_d plays the role of the control parameter. We expect that for some F_d values, the system should exhibit chaotic behavior, as does the logistic map for certain values of r. To investigate the behavior of the system, we solve Eq. (5.31) numerically with an ODE solver next.

5.3.1 Regular and chaotic motion

Program 5.5, named “Nonlindro”, integrates the equations of motion of the nonlinear driven oscillator. The system dynamics (5.31) is calculated in the function pendulum(),

def pendulum(y, t): # y = [theta, omega], omega_0 = 1
      return [y[1], −ma.sin(y[0]) − b*y[1] + fd*ma.cos(omega_d*t)]

The two-element input array y[] contains the angular displacement and angular velocity as .

The results, θ(t) and ω(t), are shown in Figure 5.13 for two driving amplitudes. For F_d = 0.7, the angular displacement and angular velocity both undergo an initial transient period. During this period, the oscillator loses track of where it came from due to damping, and adjusts the motion to the driving force. After the transients, it oscillates regularly and periodically with the same frequency as the driving frequency. This is akin to pushing a child on a swing at regular intervals. No matter what the initial conditions were, the final oscillations are the same. The system has no memory effect (we will see this again in a linear driven oscillator, Section 6.1).

At a larger F_d = 1.1, the motion looks rather irregular. There is no clearly defined transient period, and the oscillator never settles into a regular motion. The angle θ(t) shows wild swings over the top, exceeding the regular range |θ| < π. The same is true for the angular velocity, which shows no apparent structure. We suspect that, whilst the motion is regular for F_d = 0.7, it is chaotic for the larger driving force.

To confirm this, we need to calculate the Lyapunov exponent in each case. We can do this by modifying program Nonlindro slightly to start two oscillators with a small difference in the initial conditions, and see how the difference develops in time. We leave this to Project P5.4 and only focus on discussing the results.

Figure 5.14 shows how the difference in angular velocity, Δω = |ω₁(t) − ω₂(t)|, changes over time from two nearby initial conditions. For the smaller perturbation of F_d = 0.7, the difference becomes smaller with increasing time. There are oscillations in the data due to the oscillating nature of the system, just as there were scatter in the logistic map. But the overall trend is clear: the rate of convergence is exponential. The slope of the trend line, the Lyapunov exponent, is negative. Thus the motion is regular, nonchaotic, for the smaller F_d.

However, for the larger driving force F_d = 1.1, the difference grows exponentially with time. Here, the data fluctuates more wildly than before because of the stronger perturbation. The difference also seems to saturate a little, reaching a plateau for t ~ 70 to 100 s. This usually happens in any system that is bound physically, like in the logistic map where |Δx| < 1. In this case, the amount of energy in the system must be finite. Therefore, the difference cannot grow indefinitely. No matter. Here, it is the initial trend of the separation before saturation occurs that is important in measuring the Lyapunov exponent, or the chaoticity. The trend line drawn between 0 to 60 s certainly gives a positive Lyapunov exponent.

After all, if the trajectories diverge exponentially, the system is extremely sensitive to the initial condition, and after a short time they will “forget” where they came from. This, however, is different than the no-memory effect mentioned in the regular motion above. There, every point converged to the same motion. Here, no two points lead to the same motion. Put another way, the current path could have come from any point in the chaotic region. We cannot trace backward to reconstruct the original path, even though the system is deterministic.

Quantitatively, estimating from both trend lines in Figure 5.14, we find the Lyapunov exponents to be λ ~ −0.25 s⁻¹ for F_d = 0.7 s⁻², and λ ~ +0.1 s⁻¹ for F_d = 1.1 s⁻². This means that, over one period of the driving force (2π/ω_d), the separation between two nearby trajectories is reduced by a factor ~ 0.07 in regular motion, and increased by a factor ~ 3 in chaotic motion. Both these numbers confirm our expectations from Figure 5.13.

In the chaotic regime, we can think of τ = 1/λ as the time scale of trajectory divergence. Predictions of motion beyond τ would be meaningless.

5.3.2 The phase space and the Poincaré map

As we will see shortly, another useful way to study nonlinear dynamics is to plot the phase space trajectories showing θ and ω together. As mentioned earlier (Section 2.4), phase space is a concept taken from Hamiltonian mechanics, a multidimensional space encompassing all coordinates and momenta of a dynamical system. In the case of the driven oscillator, it refers to the θ and ω space. Sample trajectories calculated with a slightly modified Nonlindro are shown in Figure 5.15.

At F_d = 0.7 the path starts from the initial point and quickly converges to an ellipse in which the motion repeats indefinitely. This is not surprising since we know from earlier discussions that the motion is regular at this driving amplitude. This ellipse is an attractor, though it is not a single point for a continuous system. On the other hand, at F_d = 1.1, the path looks very complex. Here the numerical values of θ have no limits (see Figure 5.13), although as an observable it should be within [−π, π]. We thus remap θ whenever it exceeds this range as

The path wanders all over the diagram like spaghetti and there is no stable attractor for it to evolve around. If we increase F_d to 1.2, the system returns to a simple pattern again, though it is not obvious from the plot because of the continuous nature of the variables.

To make more sense out of continuous changes, a better idea is not to plot every point. Rather, we look at them at discrete times. The result is a distribution of points taken at specific time intervals, also known as a Poincaré map. To produce a Poincaré map, we plot only the points in synchronization with the driving force, e.g., when it is maximum or minimum, after the initial transients pass. The technique is used in Program 5.6. The results are shown in Figure 5.16.

For regular motion (F_d = 0.7), the plot is simply remarkable: only two points, corresponding to two samplings in each period of the driving force, one at the beginning of the period, and one at the halfway point. Incidentally, at F_d = 1.2 we again see two points, even though it looks more complex in Figure 5.15. We can guess that the motion must be regular too. In these Poincaré maps, regular motion repeats periodically as discrete points. It is similar to stroboscopic imaging. For example, a spinning disc may rotate too fast to see the marking on it. But, if we were to shine light on it at periodic intervals, then instead of seeing a blur, some patterns could be recognized. Poincaré maps are indispensable for understanding complex dynamics, especially in higher dimensions.

For F_d = 1.1, there are many more points in the Poincaré map, but they trace out relatively simple structures. The points do not fill up the graph, rather they follow a disjointed line, like a shoreline. When a point is recorded, we know it will be on the shoreline, though not exactly where. This shows that even chaotic motion obeys certain rules. We can predict the unpredictable, so to speak.

The appearance of shoreline structures is a result of the trajectory meandering through available phase space. It is a signature of chaos. To understand how it comes about, it is helpful to think of the Poincaré map as a cut of a topological surface, like a twisted donut, in a multidimensional space. We can imagine the phase space trajectory moving along and wrapping around the surface. When a cut is made, the cross section of the surface, the shoreline structure, is exposed, and that is what we see. It is called the Poincaré surface of section.

Figure 5.17 shows the Poincaré surface of section of the nonlinear driven oscillator for F_d = 1.1. It represents several cuts, or layers of the Poincaré map. The first layer (bottom) is recorded at the start of the period, the second layer one step after the start, the third one two steps after, and so on. The bottom layer is the same as the Poincaré map shown Figure 5.16 (half of it to be precise, since only one point per period is taken).

The shape of this surface of section is relatively simple. In other chaotic systems it can be quite complex. Generally, the surface over which the phase space trajectory travels is referred to as a torus. The Poincaré maps represent cross sections of these tori. In more than three dimensions of phase space, the Poincaré surface of section is an abstract multidimensional surface. For regular motion, the surface of section reduces to a curve.

5.3.3 Birfurcation diagram

Figure 5.16 tells us that the motion is regular at F_d = 0.7, chaotic at 1.1, and regular again at 1.2. The fact that there are regions of regular motion below and above this particular value of F_d = 1.1 should not surprise us. We have seen similar islands of regular motion sandwiched between chaotic regions in the logistic map (see Figure 5.6).

As in the logistic map, a bifurcation diagram can give us an overview of regions of regular and chaotic motion. We use the same discrete sampling technique to calculate the bifurcation diagram for the driven oscillator. For each F_d, we wait for the transients to pass, and record data points in synchronization with the driving force. Program 5.6 can be used for this purpose, only the graphing part needs to be modified.

The results are shown in Figure 5.18. We see period doubling structures similar to Figure 5.6. There are period-2 cycles in the beginning, which bifurcate into period-4, period-8, etc., and quickly becomes period infinite, i.e., chaotic. The chaotic region is followed by windows of regular motion again (F_d ~1.05 and 1.2). This all looks very much like what we have in the logistic map. We again see that the route to complexity is through period doubling.

The enlarged figure is so strikingly similar to the logistic map (see Figure 5.6) that we could have mistaken for it without looking carefully at the graph labels. Two totally different systems, yet quite similar structures. It shows that deterministic chaos is not randomness, has universal properties, and plays by a set of predictable rules.

5.4 The Lorenz flow

Simple as they appear, the two examples we have discussed so far would be far less understood or illuminating as to their chaotic nature without digital computation. Since Lorenz's work, computational studies have firmly established chaos as a new field of science.

In the seminal paper published in 1963 on hydrodynamic flow [59], Lorenz wrote: “… it is found that nonperiodic solutions are ordinarily unstable with respect to small modifications, so that slightly differing initial states can evolve into considerably different states”. The term chaos was not a common scientific vocabulary then and the science of chaos was yet to be advanced. Today, we recognize, unmistakenly, that Lorenz was speaking about chaos. He is widely credited with the discovery of chaos.

Lorenz was studying convection problems of hydrodynamic flow in climatic and atmospheric physics. Given the very limited computing power at that time by today's standard,⁴ Lorenz modeled the process using a much simplified version of the Navier-Stokes equations, which are essentially Newton's laws expressed in continuous variables like density, temperature, pressure, etc., to describe fluid dynamics. The result is the Lorenz model, which consists of three ODEs as

Equations (5.33a) to (5.33c) approximately describe the convective flow of fluid between two plates kept at a temperature difference. The variable x represents the flow intensity, y the temperature difference between the rising and falling currents, and z the temperature gradient. The parameters σ, r and b are positive constants.⁵ Given the idealization of the Lorenz model, we do not expect it to accurately describe realistic flow, which surely requires a great deal more variables and parameters than present in the model. However, the model does capture one of the most important features in the actual system: the instability and sensitivity to initial conditions. For this reason, we will treat the model as a nonlinear physical system, and not expect it to predict the exact behavior of actual systems.

Though simplified, the Lorenz model exhibits complex behaviors typical of nonlinear systems. Note the nonlinear terms −xz in Eq. (5.33b) and xy in Eq. (5.33c). In σ, r and b, we have three control parameters. Relative to the previous examples, this is a rather large parameter space to explore. We will follow Lorenz and use fixed values σ = 10, b = 8/3, and vary r as our sole control parameter. We will see that as r is varied, the motion will transition between regular and chaotic behaviors.

5.4.1 Solutions of the Lorenz model

To follow the dynamics, we can use the RK4 method to integrate the Lorenz equations as we did for the nonlinear oscillator, because the model is dissipative (Eq. (5.33c), for example). We can follow any of the variables, as they behave similarly for a given control parameter. We choose to show the results of y for three values of r in Figure 5.19 (see Project P5.5).

For the two smaller r values, y increases rapidly in the beginning, reaches a peak value, then drops sharply. Because it represents the temperature difference of rising and falling currents, convective flow is too far out of equilibrium when the difference becomes large, causing subsequent instabilities in the form of rapid rises and falls. After the initial phase, y oscillates around an equilibrium point with reduced amplitude (fixed point), and eventually settles to that point like a damped oscillator.

For r = 28, however, the behavior of y after the initial phase is very different. It oscillates with increasing amplitude, reaching a peak at t ~ 17. Thereafter, it falls and rises sharply, sometimes having two or three peaks of the same sign before flipping to the other side. It never reaches a steady state. The motion becomes irregular and chaotic.

At this stage, the system becomes very sensitive to initial conditions or small perturbations. In fact it is so sensitive that one must take care to make sure the results are converged. For the r = 28 results shown in Figure 5.19, calculations with the RK4 method converge only for step size of ~ 0.005 or smaller.

Incidentally, in his original calculation, Lorenz used a quasi second-order method with a step size of 0.01. Compared to our converged results, Lorenz's calculation was accurate only up to about t ~ 17. There is considerable disagreement between the two results after that point. However, this does not alter the conclusions Lorenz drew: long term predictions in this system are not feasible. The key factor is that when the system is chaotic, a small numerical error will be amplified exponentially just like a small separation between two trajectories. In this sense, numerical accuracy is less critical in assessing long term trends, as long as the results are statistically representative of the chaotic nature of the problem (numerical accuracy still needs to be treated with care, of course). Weather systems, though much more complicated than the idealized Lorenz model, behave similarly in the essential point, that is: the fundamentally chaotic nature of weather prevents reliable long term forecasts.⁶

5.4.2 The anatomy of a strange attractor

The wild oscillations observed above seem to be random by the variable y alone. But, when viewed in the three-dimensional phase space, the trajectories are in fact meandering between two regions known as a strange attractor, tracing out a butterfly-like figure, illustrated in Figure 5.20.

Two “eyes” of the attractor (or steady state solutions, Exercise E5.6) are located roughly at the centers of the butterfly's wings, on the opposite side of the x-z plane. From the starting point, the trajectory spirals outward from one eye for a few rounds, then crosses the x-z plane to the neighborhood of the other eye, where it swirls around about two times before crossing the x-z plane again toward the first eye. Thereafter, the cycle repeats. Because the attractor is in the chaotic regime, they show sensitive dependence on initial conditions. This behavior seems abnormal, and is one reason the attractor has been called a strange attractor.

After many crossings, a beautiful figure emerges like a butterfly. We can better visualize the structure of the butterfly with VPython. Figure 5.21 shows one 3D view where we are peering down in the negative z direction, with the x-axis to the right and y-axis up. We can clearly see the spirals about the two eyes and the crossings through the x-z and y-z planes. The structure formed by the points is approximately the shape of the Poincaré surface of section.

Note, however, the trajectory never crosses itself. This is a general property of phase space for any system. Also note from Figure 5.21 that there is a “thickness” to the spirals, i.e., adjacent spirals are out of plane with each other. So the space traversed by the trajectory has a “volume” structure. However, as we will discuss shortly, it has a fractal dimension greater than 2 but less than 3, meaning it is a volume-like object with zero volume. This is the other technical reason the attractor is labeled strange: it has a fractal dimension (Section 5.6).

Because the weather system is nonlinear, and chaotic under certain conditions, a small difference in its initial conditions may lead to totally different outcomes. This is sometimes referred to as the butterfly effect. The flapping of a butterfly's wings somewhere in the world may cause either nothing, or a storm somewhere else. The unpredictability is not just due to the limitation of computer modeling. More powerful computers and more refined initial conditions can improve weather forecasts, but will not alter the inherit unpredictability and limited long-range forecasts of weather systems.

As we learned from the nonlinear driven oscillator, even chaotic motion is predictable in the Poincaré map (see Figure 5.16). How do we make such a map for the Lorenz model which does not have a periodic driving force? We do this by making a cut along a given plane in phase space, say y-z plane, such that every time the trajectory crosses this plane, we plot a point at (y, z). When plane-crossing happens, x = 0. This means that when we integrate the equations of motion (5.33a) to (5.33c), we keep track of the value of x, and if it changes signs, we plot the pair of values (y, z).

A Poincaré map for the strange attractor is shown in Figure 5.22. As the trajectory moves between the wings of the strange attractor, it crosses the y-z plane numerous times (Figure 5.22, top). We mark the position on the plane at each crossing. In the end, the marked points form a “V” shaped distribution (Figure 5.22, bottom).

Suppose the trajectory is spiraling around the eye on the negative side of the x-axis at a given moment. By following the trajectory, we observe that it will move to the positive side of the x-axis by crossing on the right branch of the “V”, and returning to the other side on the left branch. The order is never violated. Over a long time, there are equal number of crossings on each branch. However, there is no telling how many spirals around an eye it will execute before crossing, or where exactly on the “V” it will cross (other than the branch).

A Poincaré map is just one slice of the Poincaré surface of section (Figure 5.17). The one shown in Figure 5.22 is a slice of the Poincaré surface of section for the Lorenz model at x = 0. We can build up a complete surface of section by making a series of Poincaré maps on planes parallel to the y-z plane at regular intervals of x. The finished surface of section should resemble Figure 5.21 (see Project P5.5).

5.5 Power spectrum and Fourier transform

We have discussed ways to characterize chaotic dynamics in several nonlinear systems. They include the Lyapunov exponent, the Poincaré map, and the nonperiodicity of the trajectories. There is another powerful method, the power spectrum.

The power spectrum method can reveal frequencies in the system. For regular, periodic motion, we expect that the spectrum will be dominated by discrete frequencies. If the system becomes chaotic, what do we expect to see in the spectrum?

Power spectra can be obtained via the Fourier transform defined by

The inverse Fourier transform can be written as

The functions g(ω) and f(t) are uniquely related to each other by Eqs. (5.34) and (5.35). Furthermore, they satisfy the following integral relation,⁷

If the function f(t) represents the amplitude of a wave, we expect the energy carried by the wave to be proportional to the square of the amplitude, |f(t)|². We can therefore interpret |g(ω)|² (or simply |g(ω)|) as the energy density in frequency domain, namely, the power spectrum.

As an example, we show the power spectra of the logistic map at several control parameters in Figure 5.23. They are obtained by Fourier transform explained below. At the two smaller values of r, we see discrete peaks. The number of peaks corresponds to period-4 and period-16 cycles, respectively. This agrees with the period of stable fixed points in regular motion. By far the largest contribution comes from ω = 0 component. It is a result of non-sinusoidal data.

Increasing r by just a small amount r = 0.892 → 0.893, we see the spectra change qualitatively. There is a “noisy” background beneath the peaks. The logistic map is weakly chaotic since it is at the boundary between regular and chaotic regions (Figure 5.6). Some quasi-periodic trajectories still persist. In the case of fully chaotic motion (r = 0.99), the spectrum is nearly continuous, except the ω = 0 peak which is always present. There are no periodic trajectories, so the power spectrum resembles the spectrum of white noise. In general, when a system becomes chaotic, the power spectrum will be dominated by white noise whose frequencies extend the full range.

The FFT method

The spectra are generated by fast Fourier transform (FFT) of the map iterates. The data is the same as that used in making Figure 5.6. For numerical work, a discrete version of the continuous Fourier transform (5.34) is required. The discrete Fourier transform (DFT) takes the form

where f(t_k) are the N sampled data points at equidistant intervals t_k over a period of T. The discrete angular frequencies are

The direct approach to DFT as given by Eq. (5.37) is straightforward to implement. For each component g_m, the summation over k requires N operations (multiplications plus exponential function evaluations). To compute all N components, N² operations are needed. For large N, it is increasingly inefficient and slow.

The FFT algorithm can compute the Fourier transform in N ln N operations [18]. With the FFT method, one can achieve large gains in speed. It is one of the few truly significant algorithms discovered in digital computing that has had a tremendous impact on computational physics and nearly every field of scientific computing. Though FFT is widely available in nearly every programming language, it is well worth the effort to understand how it works to not only appreciate its power and elegance, but also to have a portable and flexible FFT function, and to be able to modify it if necessary.

The FFT method is described separately in Section S:5.B, and the Python codes shown in Program S:5.1. From now on we will include these functions in our FFT library fft.py, and import fft before use.

Figure 5.23 is produced with the FFT library. Because the Fourier transforms are generally complex, the power spectrum plots the absolute magnitude of the transformed data. The use of FFT is not limited to calculating the power spectrum. It can be used to find the onset of period doubling, for example. Exploration of power spectra of other systems are left to projects.

5.6 Fractals

Earlier we mentioned that the strange attractor of the Lorenz model apparently has a fractional dimension, i.e., non-integer dimensions like 1- or 2-dimensional space that we are used to. For example, the attractor has a dimension ~ 2.05, so it is strange. How does this happen? A rough explanation is that the region around the attractor is chaotic in some directions but regular in others, i.e., some Lyapunov exponents (λ) are positive and some negative. So the area is stretched (positive λ) in some directions but compressed in others (negative λ), making the area irregular and non-measurable in normal integer dimensions.

To understand fractional dimensions, suppose we wish to measure the length of a line, L. Typically, we would choose a measuring unit (scale), say l, and count how many units are needed to cover the line, say N(l), then we would have the answer, L = N(l) × l. So normally, we expect N(l) ∝ 1/l¹. We say the dimension of a line is 1. For an area, we could use a unit square of area σ = l² as the basic measuring unit, and would expect the number of unit squares to cover an area to vary as N(σ) ∝ 1/σ = 1/l². This is then a 2-dimensional space (surface).

In general we define the dimension of an object as

where l is the scale, and N(l) the number of basic units required to cover the object. The dimension is d_f, also known as the Hausdorff dimension. For regular lines, d_f = 1 and for surfaces, d_f = 2 as discussed above.

Note that Eq. (5.39) is another case of scaling laws, which we have seen to be very useful in prior cases (see Eq. (3.33) and Figure 4.8).

To obtain d_f, we just need to measure an object with two different scales l and l′, count the numbers of units needed in each case, N(l) and N(l′), and determine d_f as

Let us see what happens if we apply Eq. (5.40) to several self-similar objects. First, let us start with a line of unit length, cut out the middle third, and form a triangle, as shown in Figure 5.24. As we repeat the process indefinitely, a self-similar curve is formed. It is called a Koch curve. To determine the dimension of the curve, we count the number of units needed to cover the curve at various unit sizes, and we get Table 5.1.

According to Eq. (5.40), the dimension of this Koch curve is

So it is not an integer dimension, but a fractional one. It is between 1 and 2, between a regular line and a regular area. We can think of it as an object of infinite length but zero area. Indeed, the length of the Koch curve does go to infinity as we add more and more triangles. We see that the process produces a self-similar object which has a fractional dimension, or a fractal. We could construct other types of fractals such as the Sierpinski carpet, which consists of taking out the middle third of a square repeatedly, or the Cantor set, etc., all having fractional dimensions (Exercise E5.7 and Exercise E5.8).

Fractal dimensions of attractors

How do we define the fractal dimension of a strange attractor like that shown in Figure 5.21, which does not have a well defined shape?

It turns out a rather elegant yet simple measure is more suitable for chaotic trajectories. Consider the trajectory spiraling around the strange attractor in the Lorenz model (Figure 5.21). Let us pick a point P on the trajectory and place a sphere of radius R at that point, like one of those depicted in the figure. Because the trajectory is chaotic, it will travel over all the allowed volume in phase space. If we wait long enough, the trajectory is bound to return to the neighborhood of the sphere.

We can count the number of times the trajectory enters the sphere over a given amount of time. Let us denote it by N_P (R). We expect that N_P (R) will increase with increasing R. Let us assume a power-law relation,

where C is a proportional constant, and d_c is called the correlation dimension [86].

Roughly, correlation dimension measures the frequency (or density) of the same trajectory visiting the neighborhood around point P. To find d_c, vary the radius and record N_P (R) for different values of R. We can then plot N_P (R) as a function of R on a log-log scale. Experiments show that over an intermediate range of R, the slope is a straight line. This slope is d_c, the correlation dimension.

The value of d_c thus obtained will have a weak dependence on the location of the point P. To get a more accurate value, the above procedure is repeated at different points, and we take the average over them. By this method, we can obtain the correlation dimension of the strange attractor, ~ 2.05, see Project P5.6. It is thus a fractal with infinite area but zero volume, like an infinitely crumbled piece of (elastic) paper.

Mandelbrot fractals

There are many kinds of fractals. A common feature among them is self-similarity. We have seen self-similarity in chaotic systems as well. Perhaps the most well-known ones are the Mandelbrot fractals, shown in Figure 5.25. The Mandelbrot fractals are generated by the equation which defines the Mandelbrot set,

Here c is a complex number. The number c is in the Mandelbrot set if for large iterations z_n remains bounded.

To produce the Mandelbrot fractals, we pick a pair of numbers (x, y) and set c = x+iy. We then iterate Eq. (5.42) and see if |z_n| remains finite. It can be shown that if |z_n| > 2, then it will grow with each iteration and tend to infinity, so this c is not in the set. We record the number of steps it takes until |z_n| reaches the value 2, and color-code that point accordingly (Program 5.7).

Figure 5.25 shows successive magnifications of the Mandelbrot fractals starting with the top figure. The smaller figures are displayed in column-major order. The white box in each picture is blown up in the next level. We see at each scale, self-similar structures repeat themselves. By the last level shown, we have reached the limit of double precision. Renormalization would be required to go deeper. The level of complexity is infinitely deep.

Modify Program 5.7 to explore different parts of the domain. It is not possible to see the complete structure of the set. No one has, or ever will. Have fun making fractals, your own, beautiful, unique fractals. Who knew doing science would be so much fun?

Chapter summary

Through the logistic map, this chapter opened our discussion on nonlinear dynamics and a window to the rich and bewildering world of chaotic dynamics in deterministic systems. We carefully examined the logistic map with a mixture of numerical and analytical results, which enabled us to introduce several important concepts in the description of chaotic dynamics. We described periodic orbits revolving around stable fixed points. The fixed points become unstable as the control parameter increases, destroying the old periodic orbits but creating new ones of higher order in a series of period doubling. This process continues until at a critical point when all periodic orbits have been destroyed. The system becomes chaotic.

We discussed ways to characterize and quantify chaos, including the Lyapunov exponent, phase space trajectories, the Poincaré map and surface of section, and the power spectrum with the introduction of FFT. All these concepts are universal and demonstrated in nonlinear systems we presented, such as the driven pendulum and the Lorenz model of flow. Through the study of these exemplar systems, we see that chaotic motion is unpredictable in the long term, but it is not randomness. The evolution toward chaotic motion mostly follows the same pattern, the destruction of periodic orbits, the birth of self-similar but more complex periodic orbits, and eventually, sometimes abruptly, the onset of chaos. Besides chaos characterization, there are ways for limited chaos control (e.g., see [91]) since even chaotic systems follow deterministic laws.

Self-similarity at infinitely sub-divisible levels leads to fractals, including the strange attractor of the Lorenz model. We used a variety of visualization and numerical techniques to aid our discovery of the interesting world of nonlinear dynamics and chaos, including unique perspectives in visualizing the Poincaré maps in 3D and the structure of the butterfly attractor.

5.7 Exercises and Projects

Exercises

E5.1	(a) Expand Eq. (5.9) for the fixed points of period 2 and show explicitly that the quartic equation is (b) Solve the above equation by using explicit formulas for the roots of quartic equations, or, via a more insightful approach, by successively reducing it to a cubic and then a quadratic equation by re-using the period-one fixed points (5.8). Do it by hand, or better yet, let SymPy work for us (for free), In [1] : from sympy import * In [2] : x, r = symbols('x r’) In [3] : f = 4*r*x*(1−x) In [4] : f2= 4*r*f*(1−f) In [5] : factor (f2−x) # factor expression After factoring out these terms, the resulting equation should look like where x1 = 0 and x2 = 1 − 1/4r are the fixed points of period one (5.8), b = −1 − 1/4r, and c = (1+1/4r)/4r. Solving the remaining quadratic equation yields the results, Eq. (5.10).
E5.2	(a) Write a program that generates the cobweb diagram showing the convergence toward attractors as illustrated in Figure 5.3. The procedure is as follows: plot the map function for a given cycle, e.g., y = f(x) for period one, and f(2)(x) for period two; place the initial point on the projection (diagonal) line, y = x; draw a vertical line to find its next value on the y(x) curve; draw a horizontal line to the projection line y = x for the seed value for the next cycle; and repeat the last two steps. (b) Apply your program to other cases such as f(3)(x) and f(4)(x). Optionally, find the fixed points with SymPy, and compare with the graphical results.
E5.3	Prove the relation (5.13) mathematically. Start with Assume x is in the small neighborhood of x*, i.e., x = x* + ∈, and expand gn(x) in powers of ∈.
E5.4	Show that the last two fixed points of period-2 (5.10) are stable for .
E5.5	Create a program to visualize the dynamics of period doubling using VPython. It should produce output like the sample Figure 5.7. Generate the time series for a given r, and vary it from ~ 0.7 to 1. Describe how the points move between the branches. Annotate the path for a representative value of r if possible.
E5.6	Find the steady state solutions of the Lorenz model (5.33a) to (5.33c), i.e., solutions when dx/dt = dy/dt = dz/dt = 0. Analyze their stabilities.
E5.7	A Cantor set can be formed this way (Figure 5.26): start with a line, cut out the middle third, remove the middle thirds of the remaining two segments, and repeat this process indefinitely. Calculate the fractal dimension of the set. Discuss its meaning. Figure 5.26: The first three levels of the Cantor set.
E5.8	Figure 5.27 shows the Sierpinski carpet. Describe how it is made, and calculate its dimension. Discuss your result. Figure 5.27: The first three iterations of the Sierpinski carpet.
E5.9	Generate a Julia set by solving f(z) = z3 + 1 with Newton's method as follows. Pick a point (x, y) in the complex plane as our initial guess and set z1 = x + iy. Iterate Newton's method (3.18) explicitly as The iterations will converge to one of three roots (attractors), namely [−1, exp(±iπ/3)]. Check which one after each step within some tolerance. Record the number of iterations, and put a point at (x, y) using a color scheme. For instance, color the point in hues of red, green, or blue, depending on which root it converged to. Explore the basins of attraction and their boundaries. Discuss your observations regarding self-similarity and connectedness of the points.

Projects

P5.1	We have seen from Figure 5.6 that there is a period-3 stable window embedded in the chaotic region of the logistic map around ~ 0.96. Why does it not appear at smaller values of r? Explore this and related questions of the period-3 orbits. (a) Plot the map function f3(x) at r = 0.959, and the straight line y = x. Find the intersection of x = f3(x). How many fixed points do you see? Which ones are stable? (b) Repeat the above for r = 0.95. Are there any stable fixed points? (c) Decrease r from 0.959. At each r generate a power spectrum of the map iterates after discarding transients. What is the lower limit of r when no period-3 cycles exist? Compare your results with . (d)* Investigate the other visible stable windows around r ~ 0.935, 0.926, and 0.907, corresponding to period-5, 7, and 6, respectively.
P5.2*	We take up the task of calculating the r values for the superstable fixed points in the logistic map. Direct calculations are useful in understanding the universal properties, including the Feigenbaum constants and the rate of convergence. (a) Show analytically that the first two values for period-1 and period-2 cycles are and , respectively. (b) Write a program to calculate the higher series by solving Eq. (5.20) with Newton's root finder. The following suggested strategy should be useful. For a given n ≥ 2: Define , with iterating according to Eq. (5.5). Define h(r) = dg(r)/dr, and evaluate the derivative in a way analogous to the chain rule (5.15), but with respect to r (not x). Again, iterate (recursion-like) the intermediate gi ≡ xi as above. Set the initial guess as where δn−1 is the latest δ-number computed according to (5.18). Call Newton's root finder with g, h, and the initial r. Cross fingers and record the returned value. That is . Repeat above steps for the next n, etc. See how far you can push to higher n, or how close to r∞. (c) Compute the α numbers at each n. Make a table listing both the δ and the α numbers. Plot the results. Is the convergence uniform? Where does it break down? Why? Discuss these and other observations you have made.
P5.3	Investigate the properties of the sine map, (a) Plot the map function and compare it to the logistic map. (b) Find the fixed points of period-1 and period-2 cycles. Do this with a root finder. Discuss their stability. (c) Generate a bifurcation diagram. Note where the bifurcations occur. Locate the r values for the onset of period-2n cycles for n = 1 to 5. Do the first two values agree with your analysis above? Discuss and compare your results with the logistic map. (d) Use the FFT technique to find the higher rn values. What is the onset of chaos? Compute the Feigenbaum δ number with these values. (e) Calculate the Lyapunov exponent of the map. Discuss similarities and differences with the logistic map. (f) Pick two representative r values corresponding to a positive and a negative Lyapunov exponent, respectively. Iterate the sine map with a small difference in the initial conditions, and obtain the Lyapunov exponent graphically. Compare your direct, graphical results with the numerical results above.
P5.4	We will carry out further investigation of the nonlinear driven pendulum in this project. It is assumed that you will program each part as necessary. (a) The Lyapunov exponents shown in Figure 5.14 are from only one initial condition. Compute these values more accurately by averaging results from several different initial conditions. Space them out over the ranges of θ and ω. How do the results change? How sensitive are they? (b) For chaotic motion at Fd = 1.1, generate Poincaré maps from a number of different initial conditions, say 5 or more. Graph the data on the same plot. Are the shorelines different from Figure 5.16? Do they fill up the phase space? Explain. (c) Produce the bifurcation diagram in Figure 5.18. (d) Calculate the power spectra at Fd = 0.7 and Fd = 1.1. Use at least 1024 points. What is the difference between the two? Pick a few interesting regions from the bifurcation diagram, calculate and compare the spectra. (e)* Visualize chaotic motion of two pendula through animation. Write a program to interactively display their positions. Introduce a small difference in their initial conditions, and see them either “attract” each other in regular motion or diverge from each other in chaotic motion. Add key detection to change Fd interactively (e.g., “up” to increase and “down” to decrease).
P5.5	Write a program to explore the Lorenz model and the strange attractor. (a) Compare your results with Figure 5.19 for y. What is the characteristic time scale of the problem? What is your step size? How do you check convergence? (b) Once your results are converged, calculate and plot the results for x and z for the same r values. Also compute the power spectra and analyze them. (c)* Make a butterfly diagram and a Poincaré map (Figure 5.22). Construct the Poincaré surface of section by producing a series of Poincaré maps at several surfaces of constant x. Stack them next to each other using ‘3d’ projection in Matplotlib as ax = plt.subplot(1, 1, 1, projection=’3d’) ...... ax.plot ([x]*len(y), y, z, 'o’) where y and z are the points for the Poincaré map at x. (d)* Repeat parts (a) and (b) using RK45n (or if you used it already, switch to RK4n). Compare their performance, and the values of step size when the results are first converged.
P5.6	We quoted 2.05 as the fractal dimension of the strange attractor in the Lorenz model. Let us confirm it by computing the correlation dimension from Eq. (5.41) in this project. We will first try a simple approach to make things work. Once it is working, we can try a more complex method. (a) Start a trajectory in the vicinity of the strange attractor, say at (−7, −2, 30). You can use the program developed in Project P5.5 if applicable. Integrate the Lorenz equations with r = 28. Wait for transients to pass, 1,000 steps would suffice if the step size is 0.01. (b) Take the first point after the transients as our center point P, with position (X, Y, Z). Let us imagine we make three spheres of different radii Ri (i = 1, 2, and 3) centered at P. For example, radii of 1, 2, and 3 work well. Initialize three counters that will keep track of the numbers the trajectory has entered the spheres. (c) After every step, check if the current position of the trajectory is inside any of the three spheres. A point is inside if If so, increment the corresponding counter. The total number of integration steps should be such that the counter for the smallest sphere should be 100 or so. Roughly 20,000 to 30,000 steps are enough. (d) When the integration stops, plot the three counter values versus the radii on a log-log scale. Calculate the three slopes between the three points (1 to 2, 2 to 3, and 1 to 3). Take the average, and this is an approximate correlation dimension at P. It should be 5% to 10% within the value 2. (e) Once the above steps are working, try the same procedure for different center points. Average all the values, and it should be approaching 2.05. (f)* Now let us try a dozen or so different-sized spheres. The radii should differ by some constant factor so that when plotted on the log-log scale, they spread out evenly. The factor should be such that the largest radius should be no more than half the size of the attractor. Note that if the trajectory is inside a sphere of given size, then it is also inside the spheres of larger sizes (outer spheres). Discuss the shape of your curve. What happens at the two extremes of the radii? Compare your results with the three-sphere results. (g)* Repeat part (f), but instead of spheres, use boxes of different sizes. Are the results comparable? Comment on your findings.
P5.7	The double-well oscillator is a particle moving in a potential with two minima on either side of a local maxima, . The equation of motion is where b, Fd, and ωd have the same meaning as those in the driven nonlinear pendulum (5.31). It is a useful model for two-state systems. (a) Sketch the potential, then graph it. Find the natural frequency for small oscillations without the driving force. (b) Predict how the particle will move if Fd is zero, small, and large. Write down your predictions. (c) Simulate the double-well oscillator, assuming fixed values b = 0.25 and ωd = 1 (in some scaled units). Plot the position and velocity curves (x-t and v-t), for Fd = 0.1, 0.2, and 0.4. Describe the results, and how they compare with your predictions. (d) Compute the power spectrum for each Fd above. Relate your results above to the power spectra. (e)* Construct Poincaré maps, one for each Fd. What do they represent? Are any of them strange attractors? Quantify your answers.

5.A Program listings and descriptions

This program calculates and plots the difference of the logistic map from two initial values as shown in Figures 5.9 and 5.10. Note the latter is on a semilog scale.

  1import matplotlib.pyplot as plt        # get matplotlib plot functions
import math as ma                           # get math functions
  3
def lyapunov(r):                               # compute Lyapunov exponent
  5      sum, x, ntrans, nsteps = 0.0, 0.5, 2000, 2000           # try diff num.
      for i in range(ntrans):                 # let transients pass
  7            x = 4*r*x*(1−x)
      for i in range(nsteps):                # sum up next n steps
  9            x = 4*r*x*(1−x)
            dfdx = 4.0*r*(1.0−x−x)       # Eq. (5.15)
11            sum += ma.log(abs(dfdx))
       return sum/float(nsteps)           # lambda
13
ra, lyap, r, dr = [],  [],  0.7, 0.0001
15while r < 1.0:
     r = r + dr
17      ra.append(r), lyap.append(lyapunov(r))

19plt.figure()
plt.plot(ra, lyap, ’,’)                     # ’,’=pixels
21plt.axhline(0.), plt.ylim(−2, 1)           # draw horiz.line, set y limits
plt.xlabel(’r’), plt.ylabel(’λ’)
23plt.show()

The Lyapunov exponent is computed from the analytic expression (5.29) in Program 5.4.

import matplotlib.pyplot as plt     # get matplotlib plot functions
  2import ode, math as ma                # get ODE, math functions

  4def pendulum(y, t): # y = [theta, omega], omega_0 = 1
      return [y[1], −ma.sin(y[0]) − b*y[1] + fd*ma.cos(omega_d*t)]
  6
def solution(n_periods):     # find solutions for n_periods
  8       bins = 40                   # number of points per period
       t, y, h = 0.0, [1.0, 0.0], 2*pi/(omega_d*bins) # init values
10       ta, theta, omega = [],  [],  []
       for i in range(n_periods*bins):
12             ta.append(t), theta.append(y[0]), omega.append(y[1])
             t, y = t+h, ode.RK4n(pendulum, y, t, h)
14       return ta, theta, omega

16b, omega_d = 0.5, 0.6         # damping coeff., driving frequency
subnum, pi = 1, ma.pi          # subplot number, pi
18plt.figure()
for fd in [0.7, 1.1]:
20      ax1 = plt.subplot(2, 2, subnum)          # 2x2 subplots
      ax2 = plt.subplot(2, 2, subnum+2)
22     ta, theta, omega = solution(n_periods = 5)
     ax1.plot(ta, theta), ax2.plot(ta, omega)
24     if (subnum == 1):                                          # subplot specific label
          ax1.set_ylabel(’$\theta$ (rad)’)
26          ax2.set_ylabel(’$\omega$ (rad/s)’)
     subnum, fdtxt = subnum + 1, ’$F_d=$’+repr(fd)
28     ax1.text(17, max(theta), fdtxt),  ax2.text(17, max(omega), fdtxt)
     plt.xlabel('t (s)’)
30plt.show()

Program “Nonlindro” integrates the equations of motion of the nonlinear driven oscillator (5.31), which are calculated in pendulum(). The two-element array y contains the angular displacement and angular velocity as . It returns the derivatives in a list to be used with non-vectorized ODE solvers, because the two ODEs are simple and speed is important in such calculations like the bifurcation diagram (see Project P5.4). The integration uses the non-vectorized Runge-Kutta RK4n which is faster than vectorized RK4 for a handful of ODEs. Since energy is not conserved, there is no need to use symplectic integrators such as the leapfrog method. The parameters b, fd, and omega_d are global variables defined in the main program.

To maintain accuracy, the step size must be small compared to the characteristic time. In this case, it should be the smaller of the two time scales: the natural period 2π/ω₀, and the period of the driving force 2π/ω_d. In practice, we aim to achieve consistent accuracy by specifying a fixed number of points per period (line 8), and calculate the step size accordingly (line 9).

The results are plotted in four subplots (Figure 5.13) made up of 2-rows by 2-columns (line 20) arranged as

Each subplot is positioned by a subplot number, subnum, that ranges from 1 to 4, starting from the top-left, increasing across and then down.

  1import matplotlib.pyplot as plt        # get matplotlib plot functions
import ode, math as ma                   # get ODE, math functions
  3
def remap(x):        # remap theta to [−pi,pi]
  5      if (abs(x) > pi): x = (x − 2*pi if x>0 else x + 2*pi)
      return x
  7
def pendulum(y, t): # y = [theta, omega], omega_0 = 1
  9      return [y[1], −ma.sin(y[0]) − b*y[1] + fd*ma.cos(omega_d*t)]

11def poincare(transient, n_periods):   # transient periods, n_periods
       bins = 40                       # number of points per period
13        t, y, h = 0.0, [0.6, 0.0], 2*pi/(omega_d*bins) # init values
       theta, omega = [], []
15      for i in range(bins*transient):     # discard transients
            t, y = t+h, ode.RK4n(pendulum, y, t, h)
17            y[0] = remap(y[0])
       for i in range(bins*n_periods):
19             if (i%(bins//2) == 0):                  # record every half a period
                 theta.append(y[0]), omega.append(y[1])
21             t, y = t+h, ode.RK4n(pendulum, y, t, h)
             y[0] = remap(y[0])
23
       return theta, omega
25
b, omega_d = 0.5, 0.6         # damping coeff., driving frequency
27subnum, pi = 1, ma.pi          # subplot number, pi
plt.figure()
29for fd in [0.7, 1.1, 1.2]:
      theta, omega = poincare(transient = 20, n_periods = 400)
31      ax = plt.subplot(3, 1, subnum)       # 3x1 subplots
      ax.plot(theta, omega, ’.’), ax.set_xlim(−pi, pi)
33      if (subnum == 2): ax.set_ylabel(’$\omega$ (rad/s)’)
      subnum, fdtxt = subnum + 1, ’$F_d=$’+repr(fd)
35      ax.text(−3, min(omega), fdtxt)

37plt.xlabel(’$\theta$ (rad)’)
plt.show()

The poincare() function takes as input two parameters: the first specifies the number of periods to wait for transients, and the second the number of periods to record data. Data is recorded only at discrete points in a given period, at the beginning and in the middle in this case (line 19). Results are displayed in multiple plots with manual control of x-axis limits.

  1import numpy as np, matplotlib.pyplot as plt
from numba import jit                      # comment out if jit not available
  3@jit                                                   # just−in−time compiling
def mandelbrot(c, maxi):
  5       z = c
       for m in range(maxi):                 # maxi=max iterations
  7             z = z*z + c
             if (z.real*z.real + z.imag*z.imag >=4.0): break
  9       return m

11xl, xr, yb, yt = −2.2, 0.8, −1.2, 1.2                     # box size
nx, ny, maxi = 600, 481, 100
13x, y = np.linspace(xl,xr,nx), np.linspace(yb,yt,ny)
fractal = np.zeros((ny, nx, 3))                             # fractal RGB image
15for i in range(nx):
     for k in range(ny):
17            m = mandelbrot(x[i] + 1j*y[k], maxi)   # point in complex plane
            fractal [k, i] = [m, 2*m, 3*m]                # RGB color mix
19plt.imshow(fractal/maxi, extent=[xl,xr,yb,yt])   # plot as image
plt.show()

The program computes the Mandelbrot fractal using complex arithmetic. In Python, complex numbers are formed by combining real values with the pure imaginary number represented by 1j. If z is a complex number, z.real and z.imag return the real and imaginary parts, respectively.

The box size can be changed to look at different parts of the domain. At finer scales, the maximum iteration needs to be increased. The fractal is stored as an RGB image with a color mixing scheme (line 18, experiment a bit to your liking). The image is displayed by plt.imshow() on line 19. It is similar to a contour plot but may be visually more effective in cases like this.

Mapping the Mandelbrot set is computationally intensive. The bottleneck is in determining if a point is in the set. It is an ideal candidate for a speed boost. Here we use Numba for simplicity. The @jit statement instructs Numba to compile mandelbrot at runtime. The speedup factor is about six in this case, quite good considering how effortless it is on our part. If Numba is not installed, comment out the two lines. There are other remedies for speed boost (see Section 1.3.2). Another example is given in Program S:11.4.