In this chapter we study multipoint methods with memory, a task which is very seldom considered in the literature in spite of high computational efficiency of this kind of root-solvers. Most of these methods are modifications of multipoint methods without memory with optimal order of convergence, studied in the previous chapters. Using two new approaches for calculating a self-correcting parameter, a “sliding” secant-technique and Newton’s interpolation with divided differences, extremely fast convergence of new methods with memory is attained without additional function evaluations. As a consequence, these multipoint methods possess a very high computational efficiency.
Other type of multipoint methods with memory is based on inverse interpolation and a special choice of initial approximations, which significantly increases the accuracy of produced approximations to the roots. Most of the results presented in this chapter are original contributions of the authors of this book.
Recall that Traub classified iterative methods with memory in the following way (see Section 1.1):
(I) Let be determined by new information at and reused information at by the iterative process
(6.1)
Then is called a one-point iteration function with memory, which defines an iterative method with memory.
(II) Let represent quantities . If is calculated iteratively by
(6.2)
then is called a multipoint iteration function with memory.
The corresponding iterative methods defined by (6.1) and (6.2) are called one-point and multipoint methods with memory, respectively. In this chapter we deal only with multipoint methods with memory (II). Since one complete iteration of this type of methods consists of steps, sometimes we will also use the term p-step methods.
To estimate the convergence rate of the family of multipoint iterative methods (6.2) with memory, in this chapter we will use the concept of R-order of convergence introduced by Ortega and Rheinboldt (1970) and Theorem 1.4.
First we give Traub’s study of Steffensen-like method (Traub, 1964, p.179)
(6.3)
where is an arbitrary parameter. Let as in the previous chapters, and let be a simple real zero of a real function . Using a power series there follows
(6.4)
Let us recall that the error of Newton’s iteration is . Therefore, if is determined in such a way that and if is sufficiently small, then the error given by (6.4) is smaller than the error of Newton’s iteration. In particular, choosing in (6.3), we derive the third-order method
considered in Section 2.1. This method can be regarded as a two-point method since the Newton approximation is calculated in the first step, and then the improved approximation is obtained as .
Let us return to the iterative method (6.3). By calculating recursively as the iteration proceeds, we shall have a self-accelerating method. Let be a given initial parameter and let
(6.5)
Traub derived the following estimation for ,
Since implies , from the last relation it follows that
This means that there is a constant such that
According to Theorem 1.4 and (1.12) we obtain the order of the iterative method (6.3) as the unique positive root of the quadratic equation
that is, the order is . Therefore, the order of the method (6.5) with memory is higher than that of Newton’s method, which also requires two F.E. per iteration.
The second example studied by Traub (1964, Sections 8.4, 8.6) is the self-accelerating version of the well-known secant method. Given , let us define the iteration function
(6.6)
If , then (6.6) reduces to the secant method
(6.7)
of order .
By varying in (6.6) from step to step, with the new value of calculated by using information from the previous and current step, it is possible to increase the order of the secant method (6.7). The following error relation was derived by Traub (1964, Section 8.4)
Set and choose as the best estimate of Newton’s approximation available from the previous computation. Given initial approximations , let us define the iterative secant-type method with memory as follows:
(6.8)
In a similar way as for the method (6.5), it may be shown that there is a constant such that
(6.9)
According to Theorem 1.4, the order of the iterative method (6.6) is , the unique positive root of the equation which is obtained from the relation (6.9).
In this section we present two methods with memory constructed by inverse interpolation. The basic idea comes from Neta (1983) who derived a very fast three-point method of R-order 10.815. Following Traub’s terminology (Traub, 1964, p. 62), these methods will be referred to as interpolating iterative methods.
Let be two starting initial approximations to the sought root . We will now construct a two-point method calculating first by the values of at and the value of at . Then a new approximation is calculated using the values of at and the value of at .
We use inverse interpolation to compute . Let
(6.10)
be a polynomial of second degree satisfying
(6.11)
(6.12)
(6.13)
(6.14)
(6.15)
and let
denote Newton’s I.F. In view of (6.10) and (6.13) we obtain so that, together with (6.14), it follows from (6.10)
(6.16)
In the next step, we find by carrying out the same calculation but using instead of . The constant appearing in (6.10) is now given by and we find from (6.10)
(6.17)
where is calculated by (6.16).
We need two initial approximations and to start the iterative process (6.16)(6.17). However, let us observe that may take the value at the first iteration without any additional computational cost. Indeed, appears anyway in (6.16) and (6.17) for . To avoid unnecessary evaluation at the last step of iterative process, is calculated only if the stopping criterion is not fulfilled. In that case we calculate , increase to and apply the next iteration. This fine manipulation does not increase the theoretical value of the order of convergence (which is obtained as a result of a limit process), but considerably improves the accuracy of approximations at the beginning of iterative process. Practical examples show that such a choice of in (6.18) and (6.27)–(6.29) (presented later) significantly increases the accuracy of obtained approximations, see Tables 6.4–6.10.
The relations (6.16) and (6.17) define the two-point method with memory (Petković et al., 2011b):
(6.18)
is defined by (6.15).
The order of convergence of the method (6.18) is given in the following theorem.
Theorem 6.1
The two-point method (6.18) has R-order of convergence at least , where is the spectral radius of the matrix
Proof
We shall use Herzberger’s matrix method (Herzberger, 1974) on the order of a single step s-point method . A matrix , associated with this method, has the elements
The order of an s-step method is the spectral radius of the product of matrices
According to the relations (6.16) and (6.17) we form the respective matrices,
characteristic polynomial of the matrix is
Its roots are 4.5612˙, 0.43845˙; therefore the spectral radius of the matrix is , which gives the lower bound of the R-order of the method (6.18).
Now we will consider the three-point method with memory derived by Neta (1983). This method requires three function and one derivative evaluation per iteration, except in the first iteration. Neta (1979) constructed the sixth-order family of three-point methods in the form
(6.19)
already presented by (5.16). Note that if , then the correction factor in the last two steps is exactly the same.
The last step of (6.19) uses values of at 3 points and the value of the derivative at . Now let be computed from the values of at and the value of at . Let be computed from the values of at and the value of at . Clearly, the information used is the same as that of the sixth-order method (6.19) except that we need three starting values .
We use inverse interpolation to compute . Let
(6.20)
be a polynomial of degree three satisfying
(6.21)
(6.22)
(6.23)
(6.24)
(6.21) and (6.22) we immediately obtain
(6.25)
Introduce the notations
for . Then from (6.23) and (6.24) we obtain the system of two linear equations
with the solution
(6.26)
Let be defined by (6.15). Having in mind (6.25) and (6.26), after some manipulation we find
(6.27)
In a similar fashion we obtain using the already calculated :
(6.28)
Similarly, for (using available values and ) we have
(6.29)
The three-point method with memory is defined by (6.27), (6.28), and (6.29), as presented in Neta (1983).
At first glance, it seems that the need for three initial approximations for starting the method (6.27)–(6.29) is a great disadvantage. This would have been true if we had to calculate additional initial approximations and by some iterative method, spending extra function evaluations. However, the same as for the method (6.18), assuming that we have an initial approximation (necessary for any iterative method), the next initial approximation can be calculated as not requiring extra cost since is anyway needed in the first iteration. A lot of practical experiments showed that another approximation can be taken sufficiently close to the already calculated , for example
Note that the methods (6.18) and (6.27)–(6.29) may converge slowly at the beginning of the iterative process if the initial value (and, consequently, and ) is not sufficiently close to the sought root . However, this is the case with all methods that start with Newton’s iteration.
The convergence rate of the method (6.27)–(6.29) is considered in the following theorem.
Theorem 6.2
The three-point method (6.27)–(6.29) has R-order of convergence at least , where is the spectral radius of the matrix
Proof
To prove this theorem we use the same matrix technique as in the proof of Theorem 6.1. According to the relations (6.27)–(6.29) we form the respective matrices,
The characteristic polynomial of the matrix is . Its roots are 1, 0.18493˙, 10.8151˙; therefore the spectral radius of the matrix is , which gives the lower bound of the R-order of the method (6.27)–(6.29).
In a similar way we can construct a four-point method using inverse interpolating polynomial of degree four
The corresponding matrices and the resulting matrix are presented below:
The spectral radius of the final matrix is and it determines R-order of the four-point method with memory, constructed by the inverse interpolating polynomial of degree four. However, this method attains extremely high order of convergence but with relatively high computational cost and four initial approximations. On the other hand, approximations of considerably great accuracy are not necessary for real-life problems. For these reasons, we shall not discuss this method here.
In this section we study a family of two-point derivative free methods for solving nonlinear equations of high computational efficiency presented in Petković et al. (2011c). The construction of this family is based on the use of suitable functions of two variables and the variation of a free parameter in each iterative step. This parameter is calculated employing information from the current and previous iteration so that the presented methods may be regarded as methods with memory following Traub’s classification (Traub, 1964, p. 8).
An additional motivation for studying this type of methods arises from a surprising fact that such classes of methods have been very seldom considered in the literature in spite of their high computational efficiency. Another convenient advantage is the fact that the proposed methods do not use derivatives. We cite pioneering results in this topic presented by Traub (1964, pp. 185–187), the three-point method (Neta, 1983) and the recently developed method with memory (Petković et al., 2010c) with the order .
In many practical situations it is preferable to avoid calculations of derivatives of . First multipoint derivative free methods were developed by Kung and Traub (1974). We will consider a modified Kung-Traub’s family (5.3) with memory in a general form in Section 6.4.
For we obtain from (5.3) the derivative free method
(6.30)
of Steffensen’s type with quadratic convergence, where is a nonzero real constant (see Traub, 1964, p. 185). Taking , from (5.3) there follows the derivative free two-point family of fourth-order methods
(6.31)
family (6.31) requires three F.E. and has fourth order of convergence, which means that it supports the Kung-Traub conjecture.
Our goal is to generalize the Kung-Traub method (6.31) and to increase the convergence rate of generalized methods without additional function evaluations. In this manner we can obtain new methods for finding simple roots of nonlinear equations of high computational efficiency. The acceleration of convergence is attained using the variation of a free parameter appearing in (6.31) in each iteration step.
Let
be the function that appears in the Steffensen-like method (6.30). Obviously, is an approximation to the first derivative assuming that is small enough. To construct a family of derivative free two-point methods with memory, we start from the family of two-point iterative methods
(6.32)
where is at least twice differentiable function that depends on two real variables
(6.33)
The family (6.32) for the fixed is considered in Section 2.4 under the conditions (2.95).
Now we impose an additional condition on related to (2.95) and state the following theorem.
Theorem 6.3
If an initial approximation is sufficiently close to a zero of and the weight function appearing in (6.32) satisfies the conditions
(6.34)
the error relation related to the family of two-point methods (6.32) is given by
(6.35)
Proof
Introduce the abbreviations
In what follows we will derive the error relation (6.35), which is essential to our study.
Using Taylor’s series about the root , we obtain
(6.36)
and
(6.37)
view of (6.36) and (6.37) we find
(6.38)
(6.38) we get
(6.39)
Using (6.36) and (6.37) we find , and by (6.36), (6.37), and (6.39) we can express and given by (6.33). Assume that is sufficiently close to the root , then and are close to 0. Hence, we can represent the function of two variables occurring in (6.32) by Taylor’s series about the point (0,0) in the form
(6.40)
the subscript indices and indicate the appropriate partial derivatives. The error relation of the two-step iterative scheme (6.32) is
(6.41)
Using the conditions (6.34) and the expansions (6.36)–(6.40), with the help of symbolic computation in Mathematica we start from (6.41) and obtain the error relation (6.35).
Remark 6.1
It was proved by Petković et al. (2010c) that the fourth order of the method (6.32) can be attained under the relaxed conditions
(6.42)
The additional condition in Theorem 6.3 enables the term in (6.35) to be squared; this fact is of essential importance, which will be shown later. Otherwise, the relaxed conditions (6.42) (assuming ) give only linear factor and, consequently, slower convergence of the methods with memory, see Petković et al. (2010c).
We observe from (6.35) that the order of convergence of the family (6.32) is four when . If we could provide , the order of the family (6.32) would exceed four. However, the value is not available in practice and we use an approximation , calculated by available information. Then, setting , we can achieve order of convergence of modified methods exceeding 4 without using any additional function evaluations. Moreover, we will show that a special approximation of can produce two-point methods with memory of order six. We will see later that is calculated using information from the current and previous iteration, in other words, depends on the iteration index . However, we omit the iteration index for simplicity.
Henceforth, we will often write , for brevity. We consider four methods for approximating :
• (Newton’s interpolating approach), where
is Newton’s interpolating polynomial of second degree, set through three best available approximations (nodes) , and .
• (improved Newton’s interpolating approach), where
is Newton’s interpolating polynomial of third degree, set through four best available approximations (nodes) , and .
Then the self-accelerating parameter can be calculated as the iteration proceeds as
(6.43)
(6.44)
(6.45)
(6.46)
To calculate by (6.45) and (6.46), we need the expressions of and . Since
and
find
(6.47)
and
(6.48)
Remark 6.2
The secant methods (6.43) and (6.44) are, in fact, the derivatives of Newton’s interpolating polynomials of first order at the nodes , and , respectively.
Remark 6.3
The accelerating method with calculated by (6.43), actually Traub’s method (Traub, 1964) from 1964, was used in Petković et al. (2010c) to increase the order from 4 to under the conditions (6.42). The accelerating methods (6.44), (6.45), and (6.46), together with the additional condition , are simple and very useful, providing considerable improvement of convergence rate without any additional function evaluations, see Petković et al. (2011c).
By defining as the iteration proceeds using (6.43)–(6.45), or (6.46), Petković et al. (2011c) proposed a family of derivative free two-point methods with memory corresponding to (6.32),
(6.49)
use the term method with memory following Traub’s classification (Traub, 1964, p. 8) and the fact that the evaluation of depends on data available from the current and previous iterative step. Accelerating methods obtained by calculated free parameter may also be called self-accelerating methods. The initial value should be chosen before starting the iterative process, for example, using one of the ways proposed in Traub (1964, p. 186).
Note that the iterative scheme (6.49) defines a family of two-step methods. We can apply different weight functions that satisfy the conditions (6.34). For convenience, we give the list of functions of simple form:
Using simple rearrangement, it is easy to show that the choice
gives the Kung-Traub method (6.31) with memory as a special case,
(6.50)
be the order of an iterative method , then we may write
(6.51)
where tends to the asymptotic error constant of when .
To avoid higher order terms in some relations, which make only “parasite” parts of Taylor’s expansions and do not influence the convergence order, we employ the notation used in Traub’s book (Traub, 1964): If and are null sequences and
where is a nonzero constant, we shall write
In our convergence analysis we also use the Bachman-Landau o-notation: For the sequences and which tend to 0 when we write
in other words, is dominated by asymptotically.
An auxiliary estimation necessary in this analysis is given in the following lemma.
Lemma 6.1
Let be Newton’s interpolating polynomial of degree that interpolates a function at distinct interpolation nodes contained in an interval and the derivative is continuous in . Assume that
(1) all differences are sufficiently small, that is, all nodes are sufficiently close to the zero of ;
Then
(6.52)
Proof
The error of the Newton interpolation is given by the well-known formula
(6.53)
see (4.114). Differentiating (6.53) yields at the point
(6.54)
In the neighborhood of the zero , the function and its derivatives may be expanded in Taylor’s series (),
(6.55)
(6.56)
. Substituting (6.55) and (6.56) into (6.54) and taking into account the conditions of Lemma 6.1, after short arrangement we arrive at the relation (6.52).
Remark 6.4
The condition (2) of Lemma 6.1 is typical for multipoint methods with memory. If define iterative null sequences with orders , this condition means that .
The convergence analysis of the two-point family (6.49) will be conducted for each approximation of separately. First we state the convergence theorem of the family that uses the calculation of by (6.43).
Theorem 6.4
Let the self-accelerating parameter in (6.49) be calculated by the expressions . If an initial approximation is sufficiently close to a zero of , then the R-order of convergence of the two-point methods (6.49)(6.43), (6.49)(6.44), (6.49)(6.45) , and (6.49)(6.46) with memory is at least and 6 , respectively.
Proof
From some convenient applications, we rewrite the error relation (6.35) in the form
(6.57)
where
is now a varying quantity due to variable .
Method (6.43).
From Lemma 6.1 for (see Remark 6.2) we have so that
Hence
so that
(6.58)
Having in mind (6.51), substituting (6.58) in (6.57) yields
From (6.51) we have
(6.59)
Equating exponents of in the last two relations we obtain the quadratic equation
The positive root of this equation determines the lower bound of the R-order of convergence of the method (6.49)(6.43).
Method (6.44).
Calculating by (6.44) and using Lemma 6.1 for , we arrive at the following relation
(6.60)
Assume that the order of convergence of the sequence of errors is , then we may write
(6.61)
Hence, by (6.51) and (6.61), we obtain
(6.62)
On the other hand, by combining (6.38), (6.51), (6.60), and (6.61) we find
whence
(6.63)
From (6.51), (6.57), (6.60), and (6.61) we obtain
(6.64)
By comparing exponents of on the right-hand side of (6.62) and (6.63), and then on the right-hand side of (6.64) and (6.59), we form the following system of equations
non-trivial solution and . Therefore, the order of (6.49)(6.44) is at least five.
Method (6.45).
In regard to Lemma 6.1, taking in (6.52) we obtain
From the last relation and (6.45) we find
(6.65)
Using (6.38), (6.51), (6.61), and (6.65), we obtain the following error relation
(6.66)
a similar manner, in regard to (6.51), (6.57), (6.61), and (6.65), we find
(6.67)
Comparing the error exponents of in pairs of relations (6.62)(6.66) and (6.59)(6.67), we form the system of equations in and
solution of this system is , and we conclude that the lower bound of the order of the methods with memory (6.49)(6.45) is at least .
Method (6.46).
Let the errors appearing in the kth iteration be denoted by . Take in (6.52), according to Lemma 6.1 we have
Hence
(6.68)
Proceeding as before, in view of (6.62) and (6.59) we may write
(6.69)
(6.70)
(6.71)
, from (6.36), (6.38), and (6.57) we have
(6.72)
(6.73)
(6.74)
By combining the above expressions (6.68)–(6.74) we derive the following error relations
(6.75)
(6.76)
(6.77)
appropriate exponents of in pairs of relations (6.69)(6.75), then (6.70)(6.76), and (6.71)(6.77), we arrive at the following system of equations in , and ,
the positive solution , and . Hence we conclude that the lower bound of the order of the methods with memory (6.49)(6.46) is at least six.
Theorem 6.4 gives the lower bound of the order of convergence of the family (6.49) in the case of the accelerating approaches (6.43)–(6.46). We observe that the methods (6.49) with memory are considerably accelerated (even up to 50%) relative to the corresponding methods (6.32) without memory. The main advantage of the presented methods with memory is their very high computational efficiency, significantly higher than the efficiency of the existing two-point methods and even higher than the efficiency of three-point methods without memory of optimal order eight, see Section 6.6.
From (4.124) we see that the order of convergence of the family (4.113) is eight when . It can be proved that the order of (4.113) would be 12 taking . Since the value is not known, we use an approximation , calculated by available information and set in (4.113). In this way we find that the order of convergence of the modified methods exceeds eight without the use of any new function evaluations.
In this section we consider the following four methods for approximating :
• (Newton’s interpolating approach), where
Newton’s interpolating polynomial of second degree, set through three best available approximations (nodes) and .
As in Section 6.3, the main idea in constructing methods with memory consists of the calculation of the parameter as the iteration proceeds by the formula
see Džunić et al. (2012). Regarding the above methods for approximating , we present the following four formulae for the calculation of the self-accelerating parameter :
(6.78)
(6.79)
(6.80)
(6.81)
,
(6.82)
Since is calculated using (6.78)–(6.81), the function given by (4.105) is replaced by
(6.83)
The symbol is added to indicate the use of variable . Substituting instead of in (4.105) and starting from (4.113), we state the following derivative free family of three-point methods with memory proposed by Džunić et al. (2012),
(6.84)
is defined by (6.83), , and is a weight function of two variables that satisfies (6.34). Recall that the denominator is given by (4.112).
Now we state the convergence theorem for the family (6.84) of three-point methods with memory.
Theorem 6.5
Let the self-accelerating parameter in the iterative scheme (6.84) be calculated by the expressions (6.78)–(6.81). If an initial approximation is sufficiently close to a zero of , then the order of convergence of the three-point methods (6.84)(6.78) , (6.84)(6.79), (6.84), and (6.80)(6.81) with memory is at least , , and , respectively.
Proof
From (6.51) we have
(6.85)
According to the error relations (4.116), (6.35), and (4.124) with the self-accelerating parameter , we can write the corresponding error relations for the methods (6.84) with memory
(6.86)
(6.87)
(6.88)
The expressions for and are evident from (6.35) and (4.124) and depend on the iteration index since is recalculated in each iteration. Higher order terms are omitted since they do not influence the convergence order.
Let . Using Taylor’s series about the root , we obtain
(6.89)
This relation will be used for different values of . Now we determine the order of convergence of the family (6.84) for all approaches (6.78)–(6.81).
Method (6.78).
Using the expansion (6.89) for and , we obtain
to this, we calculate by (6.78) and find
(6.90)
Substituting (6.90) into (6.88) yields
(6.91)
Hence we can find a positive constant so that the inequality
(6.92)
holds. Starting from (6.92) and having in mind Theorem 1.4 and (1.12), we form the quadratic equation .The positive root of this equation determines the lower bound of the order of convergence of the method (6.84)(6.78).
Method (6.79)
Similar to the derivation of (6.90), we calculate by the more accurate secant method (6.79) and obtain
(6.93)
Assume that the iterative sequence has the order , then,
(6.94)
Combining (6.51), (6.86), (6.93), and (6.94), we get
(6.95)
According to (6.51), (6.91), and (6.94), we obtain
(6.96)
By comparing exponents of on the right-hand side of (6.94) and (6.95), and then on the right-hand side of (6.85) and (6.96), we form the following system of equations
non-trivial solution . Therefore, the order of the methods with memory (6.84)(6.79) is at least nine.
Method (6.80).
Considering the most accurate secant method (6.80), assume that the iterative sequence has the order , that is,
(6.97)
Proceeding in a similar way as for the methods (6.78) and (6.79), we obtain from (6.80)
which leads to the error relations
(6.98)
and
(6.99)
By comparing exponents of in two pairs of relations (6.97)(6.98) and (6.85)(6.99), we form the system of equations
non-trivial solution of this system is given by , we conclude that the order of the method with memory (6.84)(6.80) is at least ten.
Method (6.81)
By virtue of Lemma 6.1 for , we obtain
According to this and (6.81) we find
(6.100)
Using (6.100) and the previously derived relations, we obtain the error relations for the intermediate approximations
(6.101)
(6.102)
In a similar fashion we find the error relation for the final approximation within the considered iteration
(6.103)
As before, comparing the error exponents of in three pairs of relations (6.94)(6.101), (6.97)(6.102), and (6.85)(6.103), we form the system of three equations in and
trivial solution of this system is and we conclude that the lower bound of the order of the method with memory (6.84)(6.81) is eleven.
In this way we have completed the analysis of all accelerating methods (6.78)–(6.81) so that the proof of Theorem 6.5 is finished.
In this section we study multipoint methods with memory of arbitrary order of convergence (Džunić and Petković, 2012a). We restrict our attention to the Kung-Traub family (5.3) (Kung and Traub, 1974) and the Zheng-Li-Huang family (Zheng et al., 2011) for the following reasons:
(1) both families of n-point methods have similar structure, the order and require F.E. per iteration, which means that they generate optimal methods in the sense of the Kung-Traub conjecture;
(2) the order of convergence can be arbitrary high (in the form );
(3) these families are derivative free, which is convenient in all situations when the calculation of derivatives of is complicated.
Although the Kung-Traub family (5.3) is given in Section 5.2 in a general form, in order to represent both mentioned families in a unique form, we again display the family (5.3) in this section. This unique representation enables us to carry out the convergence analysis of both families simultaneously. These families will be modified by a specific approach to very efficient generalized methods with memory.
Kung-Traub’s family
Kung and Traub (1974) stated the following derivative free family (K-T for short) of iterative methods without memory.
K-T family:For an initial approximation , arbitrary and , define iteration function as follows:
(6.104)
is the inverse interpolating polynomial of degree at most such that
The Kung-Traub iterative method is defined by starting from , where is the iteration index. It was proved by Kung and Traub (1974) that the order of convergence of the family (6.104) is , see Section 5.2.
The approximation in the jth step within the kth iteration will be called intermediate approximation with the associated intermediate error . Following this terminology, is the penultimate approximation and is the ultimate approximation of the kth iteration.
The following error relation for the family (6.104), also called the ultimate error relation, was proved in Kung and Traub (1974)
(6.105)
where
(6.106)
and is the inverse function of . It is obvious from (6.104) and (6.105) that the intermediate error relation, equivalent to (6.105),
(6.107)
holds for each .
Zheng-Li-Huang’s family
Zheng et al. (2011) proposed other derivative free family (Z-L-H for short) of n-point methods of arbitrary order of convergence . This family is constructed using Newton’s interpolation with forward divided differences, see Section 5.5. Unlike (5.59), in this section we deal with the error factor .
Z-L-H family: For an initial approximation , arbitrary and , the n-point methods are defined by
(6.108)
The entries are intermediate approximations of Z-L-H family, while is the ultimate approximation.
The following theorem was proved by Zheng et al. (2011).
Theorem 6.6
Let be a sufficiently differentiable function having a simple zero in an open interval . If is close enough to , then the n-point family (6.108) converges with at least order and satisfies the error relation
(6.109)
where
(6.110)
(6.111)
As in the case of Kung-Traub’s family (6.104), the intermediate error relations are given by
(6.112)
where constants are calculated recursively by (6.110) and (6.111). The ultimate error relation is also given by (6.112) for .
We wish to show that constants in error relations (6.109) are of the form
(6.113)
where
(6.114)
(6.115)
For the assertion (6.113)is obvious. Let us assume that (6.113) and (6.115) are true for all . Set for short. According to (6.111) we find
, by induction, we conclude that the intermediate error relations can be written in the following form
(6.116)
where is defined by (6.114) and (6.115). Note that (6.116) includes ultimate error relation for , that is,
(6.117)
Now we will show that the Kung-Traub family (6.104) and the Zheng-Li-Huang family (6.108) can be extremely accelerated without any additional function evaluations. The construction of new families of n-point derivative free methods is based on the variation of a free parameter in each iterative step. This parameter is calculated using information from the current and previous iteration so that the presented methods may be regarded as the methods with memory.
As mentioned by Petković et al. (2010c) and Petković et al. (2011c) (see, also, Sections 6.3 and 6.4), the factor in the error relations (6.105) and (6.107) (for the K-T family), and (6.116) and (6.117) (for the Z-L-H family) plays the key role in constructing families with memory. The error relations (6.107) and (6.116) can be presented in the unique form
(6.118)
where
being the iteration index. Constants depend on the considered family; obviously, from (6.107), while they can be determined recursively from (6.115) for Z-L-H family. However, in this section we concentrate on the lower bound of the convergence order of the methods with memory so that specific expressions of and asymptotic error constants are not of interest. The use of the unique relation (6.118) enables us to consider simultaneously both families with memory based on (6.104) and (6.108).
We observe from (6.105) and (6.117) that the order of convergence of the families (6.104) and (6.108) is when . It is not difficult to show that the order of these families would be if we could provide . However, the value is not known in practice and we could use only an approximation , calculated by available information. Then, setting , we achieve order of convergence of the modified methods exceeding without using any new function evaluations.
The beneficial approach in approximating
is to use only available information, in other words, we can increase the convergence rate without additional function evaluations. We present three models for approximating :
(II) (Newton’s interpolation approach), where
is Newton’s interpolating polynomial of second degree, set through the three best available approximations (nodes) .
(III) (improved Newton’s interpolation approach), where
is Newton’s interpolating polynomial of third degree, set through the four best available approximations (nodes) .
Using divided differences, we find
(6.119)
(6.120)
that the Zheng-Li-Huang family (6.108) is very suitable for the application of Newton’s interpolating approaches (II) and (III) since divided differences are already calculated in the implementation of the iterative scheme (6.108).
According to the above methods (I), (II), and (III), we present the following three formulae for calculating the self-accelerating parameter :
(6.121)
(6.122)
(6.123)
use of Newton’s interpolation of higher order is also feasible but the increase of convergence rate is modest, see Remark 6.9.
Substituting the fixed parameter in the iterative formulae (6.104) and (6.108) by the varying parameter calculated by (6.121), (6.122), or (6.123), we state the following families of multipoint methods with memory:
K-T family with memory: For an initial approximation , arbitrary and calculated by (6.121), (6.122), or (6.123) and , define the iteration function as follows:
(6.124)
Z-L-H family with memory: For an initial approximation , arbitrary calculated by (6.121), (6.122), or (6.123) and , the n-point method is defined by
(6.125)
is the same as in (6.108).
We use the term method with memory following Traub’s classification (Traub, 1964, p. 8) and the fact that the evaluation of depends on the data available from the current and previous iterative step.
To determine the convergence rate of the families (6.124) and (6.125), we use previously presented technique. We distinguish three approaches for the calculation of the varying parameter given by the formulae (6.121) (method (I)), (6.122) (method (II)), and (6.123) (method (III)).
Method (I) – Secant approach
According to Lemma 6.1 for and we find
Hence
(6.126)
Suppose that the order of convergence of the improved families with the error relation (6.118) is , than we may write
(6.127)
where tends to the asymptotic error constant when . Hence
(6.128)
In a similar fashion, if we suppose that is an iterative sequence of the order for fixed , then
(6.129)
Combining (6.118), (6.126), (6.128), and (6.129), we arrive at
(6.130)
, in a similar way,
(6.131)
exponents of in (6.129) and (6.130), and then in (6.128) and (6.131), for we form the system of equations with unknown orders and ,
(6.132)
As a special case, let us consider successive approximations and in the secant method (6.121), that is,
(6.133)
Such accelerating method was first considered in Traub’s book (Traub, 1964) and recently in Petković et al. (2010c). Obviously, this approach gives the worst approximation to and corresponds to the values and . In this case it is sufficient to consider only the second equation of (6.132), which reduces to the quadratic equation
The positive solution
yields the lower bound of the order of the families.
The above consideration leads to the following theorem.
Theorem 6.7
Let the self-accelerating parameter in the iterative methods (6.124) and (6.125) be calculated by (6.121) for and by (6.133) for . If an initial approximation is sufficiently close to a zero of , then the order of convergence of the families (6.124) and (6.125) of n-point methods with memory is at least for , and for .
Obviously, the use of a closer intermediate approximation in the secant method (I) will give a better approximation to and, consequently, higher order of the modified methods (6.124) and (6.125), based on (6.104) and (6.108) and dealing with varying . Some particular values of the order are given in Table 6.1.
Now we estimate the order of convergence of the families (6.124) and (6.125) with memory when Newton’s interpolating approaches (II) and (III) are applied. Lemma 6.1 and the estimation (6.52) will be used.
Method (II) – Newton’s interpolation of second degree
First, let us consider the case and assume that the orders of the iterative sequences , and are at least , and , respectively, that is,
Hence
(6.134)
(6.135)
(6.136)
By virtue of Lemma 6.1 for we estimate
According to this and (6.122) we find
(6.137)
Using (6.137) and the previously derived relations, we obtain the error relations for the intermediate approximations
(6.138)
(6.139)
,
(6.140)
Proceeding as before, we equate error exponents of in three pairs of error relations (6.134)(6.138), (6.135)(6.139), and (6.136)(6.140) to form the following system of equations in unknown orders , and ,
(6.141)
solutions are
Therefore, the order of convergence of the families (6.124)–(6.122) and (6.125)–(6.122) is at least for . For example, the order of the three-point families (6.124) and (6.125) is at least 11, the four-point families have the order at least 22, etc. (see Table 6.1), assuming that is calculated by (6.122).
The case slightly differs from the previous analysis; Newton’s interpolating polynomial is constructed at the nodes , and and we may formally take in (6.141) and remove the first equation. Then the system of equations (6.141) reduces to
the solutions and . Therefore, the families (6.124)–(6.122) and (6.125)–(6.122) of two-point methods with memory have the order at least . Note that this result appears in Theorem 6.4 for the family (6.49)–(6.45) of two-point methods with memory.
We summarize results of the previous study and state the following convergence theorem.
Theorem 6.8
Let the self-accelerating parameter in the iterative methods (6.124) and (6.125) be calculated by (6.122). If an initial approximation is sufficiently close to a zero of , then the order of convergence of the families (6.124) and (6.125) of n-point methods with memory is at least for and at least for .
Method (III) – Newton’s interpolation of third degree
The computation of by (6.123)employs more information compared to (6.122) and we expect to achieve faster convergence. The presented convergence analysis confirms our expectation.
Let and assume that the order of the iterative sequences , and is at least p, q, s, and r, respectively, that is,
According to this, we find
(6.142)
(6.143)
(6.144)
(6.145)
Using Lemma 6.1 for , we obtain
From the last relation and (6.122) we find
(6.146)
Combining (6.146) and the previously derived relations, we derive the following error relations
(6.147)
(6.148)
(6.149)
(6.150)
In a similar way as before, equating exponents of in four pairs of error relations (6.142)(6.147), (6.143)(6.148), (6.144)(6.149), and (6.145)(6.150), we form the following system of equations,
(6.151)
solutions are . Therefore, the order of convergence of the families (6.124)–(6.123) and (6.125)–(6.123) is at least for . For example, the order of the four-point families (6.124) and (6.125) is at least 23.
To estimate the order of the three-point families (6.124)–(6.123) and (6.125)–(6.123), we put and in the system (6.151), remove the first equation and solve the system of three equations
solutions are
Therefore, in this particular case the order is at least .
The particular case must be examined separately. The corresponding Newton’s interpolating polynomial is constructed through the points , and . Analysis of the error sequences , and (of orders , and ) and the same arguments as above lead to the system
, we find and conclude that the lower bound of the order of the two-point methods with memory (6.124)–(6.123) and (6.125)–(6.123) is at least six.
We summarized our results in the following theorem.
Theorem 6.9
Let the self-accelerating parameter in the iterative methods (6.124) and (6.125) be calculated by (6.123). If an initial approximation is sufficiently close to a zero of , then the order of convergence of the families (6.124) and (6.125) of n-point methods with memory is at least for , at least for and 6 for .
Remark 6.7
Applying Steffensen-like method (6.3) with the self-correcting parameter calculated by (6.81) at the points , it can be proved that the order of the accelerated method reaches 3. Some characteristics of this method are given in Tables 6.1 and 6.2 in the row for . We shall not go into details here.
Remark 6.8
From Theorems 6.8 and 6.9 we conclude that the multipoint methods for are of limited value as far as practical problems are concerned. Indeed, multipoint methods with extraordinary fast convergence produce root approximations of considerable accuracy, not required for solving most practical problems. However, in this paper we have studied general families and general results on the convergence rate as a contribution to a general theory of iterative processes, emphasizing practical importance of particular cases and .
The lower bounds of the order of the families (6.124) and (6.125) for calculated by (6.121), (6.122), and (6.123) are given in Table 6.1 for several entries of and .
From Table 6.1 we observe that the order of convergence of the families (6.124) and (6.125) with memory is considerably increased relative to the corresponding basic families (6.104) and (6.108) without memory (entries in the last row). The increment in percentage is also displayed. It is evident that the self-corrections (6.122) and (6.123), obtained by Newton’s interpolation with divided differences, give the best results. It is worth noting that the improvement of convergence order in all cases is attained without any additional function evaluations, which points to a very high computational efficiency of the proposed methods with memory. Several values of the efficiency index
where is the order of the considered iterative method and is the number of F.E. per iteration, are given in Table 6.2. Numerical examples entirely confirm the theoretical results given in Theorems 6.7, 6.8, and 6.9.
Remark 6.9
It is obvious that the use of Newton’s interpolating polynomials of degree higher than 3 can provide further increase of convergence order of n-point methods for . For example, using the described convergence analysis it is not difficult to prove that the order of the fourth-order families (6.124) and (6.125) with memory is 12 if the self-correcting parameter is calculated as
(6.152)
(see numerical results in Tables 6.11 and 6.12). However, for the reasons given in Remark 6.8, we are not interested in multipoint methods of extremely high order.
In this section we present results of numerical examples, obtained by the methods considered in this chapter. We employed the computational software package Mathematica with multiple-precision arithmetic relying on the GNU multiple-precision package GMP (Granlund, 2010). The errors are given in Tables 6.4– 6.12, recalling that the notation means .
aThe method (2.104) is divergent for , but it converges for .
The tested methods were applied to the functions – listed in Table 6.3. These functions have nontrivial behavior; for illustration, we show in Figure 6.1 the graph of the function which has . All tested methods as well as other existing multipoint and one-point methods run with difficulties if the initial approximation lies in the interval .
Figure 6.1 The graph of the function
We have compared two-point methods (6.18) and (6.49) with memory to several optimal two-point iterative methods of fourth order which also require three F.E. We have also compared the three-point methods (6.27),(6.28),(6.29) and (6.124) with some of the existing three-point methods of optimal order eight.
We first consider two-point methods. Beside the two-point methods (6.49) with memory (including Kung-Traub’s method (6.50) as a special case when ), we have tested the following two-point methods without memory considered in Chapter 2:
– King’s family (2.57) (King, 1973a),
Recall that King’s family gives the following special cases: Ostrowski’s method (Ostrowski, 1960) for , Kou-Li-Wang’s method (Kou et al., 2007b) for , and Chun’s method (Chun, 2008) for .
– Jarratt’s method (2.122) (Jarratt, 1966)
– Maheshwari’s method (2.85) (Maheshwari, 2009)
– Ren-Wu-Bi’s method (2.104) (Ren et al., 2009)
– Kung-Traub’s two-point method with derivative (2.113) (Kung and Traub, 1974)
We have applied these methods to Examples 6.1–6.4. The errors for the first four iterations are given in Tables 6.4–6.7. K-T is the abbreviation for Kung-Traub’s methods (6.50) and (2.113). It is evident that approximations to the roots given in Tables 6.4–6.7 possess great accuracy. Results of the fourth iteration are given only for demonstration of convergence speed of the tested methods and they are not required for practical problems at present.
According to the results presented in Tables 6.4–6.7 and a number of numerical examples, we can conclude that the convergence behavior of the two-point methods (6.49) with memory (including the modified Kung-Traub method (6.50)), based on the self-correcting parameter calculated by (6.43)–(6.46), is considerably better than the existing methods (6.31), (2.57), (2.85), (2.104), (2.113), and (2.122) for most examples. It is obvious from these tables that self-correcting calculation by the Newton interpolation (6.46) gives the best results. Having in mind that all of the tested methods have the same computational cost, we can conclude that the family of two-point methods (6.49) with memory is the most efficient. More precisely, calculating the computational efficiency of an iterative method of order , requiring function evaluations, by Ostrowski-Traub’s formula we find
Note that the last four entries do not refute the Kung-Traub conjecture because they are related to the methods (6.49)with memory, which are not the subject of the Kung-Traub conjecture. Also, observe that the efficiency indices and are even higher than the efficiency index of optimal three-point methods (without memory) of order eight.
In this part we present numerical results obtained by applying three-point methods to the functions , and . Apart from Neta’s method (6.27)–(6.29) and already mentioned Kung-Traub’s families (6.104) and with and without memory, we have also tested the following three-point methods:
– Bi-Wu-Ren’s method (4.2), choosing two variants denoted by method 1 and method 2 in the same manner as in Bi et al. (2009b).
– Petković-King’s method (4.12), (Petković, 2010). We have chosen King’s method in the first two steps of (4.12), which is stressed by the specific name of the tested method.
– Neta-Petković’s method (4.44), (Neta and Petković, 2010).
Note that several three-point methods with optimal order eight have been derived in the period 2009–2011, e.g., Bi et al. (2009a), Geum and Kim (2010), Liu and Wang (2010), Džunić et al. (2011), Petković et al. (2010a), Thukral and Petković (2010), and Wang and Liu (2010a). However, these methods have a similar convergence behavior and produce results of approximately the same quality; therefore we have omitted them. The errors of approximations for the first three iterations are given in Tables 6.8–6.10.
To demonstrate the convergence behavior of the generalized methods with memory, considered in Section 6.5, we present two more examples where we have dealt with the functions and , given in Table 6.3. The errors of produced approximations are given in Tables 6.11 and 6.12. These tables include the values of the computational order of convergence calculated by (1.10) taking into consideration the last three approximations.
For better readability, in this section we display explicitly five accelerating formulae for the calculation of the self-correcting parameter , previously given by (6.121) (for various values), (6.122), and (6.123):
(6.153)
(6.154)
(6.155)
(6.156)
(6.157)
In all numerical examples, the initial value was used.
From Tables 6.11 and 6.12 and many tested examples we can conclude that all implemented methods produce approximations of great accuracy. We observe that the methods with memory considerably increase the accuracy. The quality of the calculation of by (6.153)–(6.157) can also be noticed from Tables 6.11 and 6.12: Newton’s interpolation (6.157) evidently gives the best results, which is expected bearing in mind that this approach provides the highest order of convergence. From the last column of Tables 6.11 and 6.12 we observe that the computational order of convergence , calculated by (1.10), matches very well the theoretical order given in Theorems 6.7, 6.8, and 6.9.
Throughout this chapter we have considered n-point methods with memory. Before estimating their computational efficiency we give a review of their orders and the number of required F.E., restricting our analysis to two-point and three-point methods.
Remark 6.10
We have chosen the methods (2.57) and (6.104) as the representatives from their classes, other methods give results of approximately the same quality so that they were not tested in these numerical experiments. Second, the number of F.E. of the methods (6.18) and (6.27)–(6.29) is denoted with and to point that the number of F.E. is respectively 4 and 6 in the first iteration.
From Remark 6.10 and the corresponding iterative formulae, we see that the number of F.E. of the methods (6.18) and (6.27)–(6.29) depends on the number of iterative steps performed to fulfill a given termination criteria (e.g., the required accuracy of approximations to the roots). For this reason it is not possible to compare the methods listed in Table 6.13 without counting total number of iterations as a parameter. It is convenient to compute the efficiency index of an iterative method by the formula
where is the total number of iterations, is the order, and is the number of F.E. at the jth iteration. Obviously, if , then the above formula reduces to the well-known formula .
From Tables 6.4–6.7 we observe that the interpolating iterative method (6.18) belongs to the group of two-point methods that produce the most accurate approximations in all presented examples. The method (6.27)–(6.29), derived by inverse interpolation of the third degree, is dominant over the tested three-point methods regarding the accuracy of approximations (see Tables 6.8–6.10). However, one should say that the method (6.18) uses one function evaluation more and the method (6.27)–(6.29) even two more F.E. at the first iteration. These additional calculations decrease their computational efficiency, which is evident from Table 6.14 which contains results of a comparison of computational efficiency of the tested methods. This table gives a more precise answer to a certain extent. It is clear that the efficiency indices of the methods (6.18) and (6.27)–(6.29) approach the efficiency indices of the methods with memory presented in Sections 6.3, 6.4, 6.5 as the number of total iterations increases since the negative effect of the expensive first iteration fades away. This closeness is noticeable when the number of iterations is four or five. On the other hand, applying iterative methods with very fast convergence, the number of iterations greater than three is very seldom needed in practice so that the mentioned limit of the efficiency indices is pushed out.
According to the entries of the efficiency indices displayed in Table 6.5 and the fact that multipoint methods of very fast convergence are of limited value for solving real-life problems (see Remark 6.8), it could be concluded that the two-point method (6.49)–(6.48) is most preferable from a practical point of view.
1. Bi W, Ren H, Wu Q. Three-step iterative methods with eight-order convergence for solving nonlinear equations. J Comput Appl Math. 2009a;225:105–112.
2. Bi W, Wu Q, Ren H. A new family of eight-order iterative methods for solving nonlinear equations. Appl Math Comput. 2009b;214:236–245.
3. Chun C. Some fourth-order iterative methods for solving nonlinear equations. Appl Math Comput. 2008;195:454–459.
4. Džunić J, Petković M. On generalized multipoint root-solvers with memory. J Comput Appl Math. 2012a;236:2909–2920.
5. Džunić J, Petković MS, Petković LD. Three-point methods with and without memory for solving nonlinear equations. Appl Math Comput. 2012;218:4917–4927.
6. Džunić J, Petković MS, Petković LD. A family of optimal three-point methods for solving nonlinear equations using two parametric functions. Appl Math Comput. 2011;217:7612–7619.
7. Geum YH, Kim YI. A multi-parameter family of three-step eighth-order iterative methods locating a simple root. Appl Math Comput. 2010;215:3375–3382.
8. Granlund, T., 2010. GNU MP: The GNU multiple precision arithmetic library, edition 5.0.1.
9. Herzberger J. Über Matrixdarstellungen für Iterationverfahren bei nichtlinearen Gleichungen. Computing. 1974;12:215–222.
10. Jarratt P. Some fourth order multipoint methods for solving equations. Math Comp. 1966;20:434–437.
11. King RF. A family of fourth order methods for nonlinear equations. SIAM J Numer Anal. 1973a;10:876–879.
12. Kou J, Li Y, Wang X. Fourth-order iterative methods free from second derivative. Appl Math Comput. 2007b;184:880–885.
13. Kung HT, Traub JF. Optimal order of one-point and multipoint iteration. J ACM. 1974;21:643–651.
14. Liu L, Wang X. Eighth-order methods with high efficiency index for solving nonlinear equations. Appl Math Comput. 2010;215:3449–3454.
15. Maheshwari AK. A fourth-order iterative method for solving nonlinear equations. Appl Math Comput. 2009;211:383–391.
16. Neta B. A sixth order family of methods for nonlinear equations. Int J Comput Math. 1979;7:157–161.
17. Neta B. A new family of higher order methods for solving equations. Int J Comput Math. 1983;14:191–195.
18. Neta B, Petković MS. Construction of optimal order nonlinear solvers using inverse interpolation. Appl Math Comput. 2010;217:2448–2455.
19. Ortega JM, Rheinboldt WC. Iterative Solution of Nonlinear Equations in Several Variables. New York: Academic Press; 1970.
20. Ostrowski AM. Solution of Equations and Systems of Equations. New York: Academic Press; 1960.
21. Petković MS. On a general class of multipoint root-finding methods of high computational efficiency. SIAM J Numer Anal. 2010;47:4402–4414.
22. Petković MS, Ilić S, Džunić J. Derivative free two-point methods with and without memory for solving nonlinear equations. Appl Math Comput. 2010a;217:1887–1895.
23. Petković MS, Džunić J, Neta B. Interpolatory multipoint methods with memory for solving nonlinear equations. Appl Math Comput. 2011b;218:2533–2541.
24. Petković MS, Džunić J, Petković LD. A family of two-point methods with memory for solving nonlinear equations. Appl Anal Discrete Math. 2011c;5:298–317.
25. Petković LD, Petković MS, Džunić J. A class of three-point root-solvers of optimal order of convergence. Appl Math Comput. 2010a;216:671–676.
26. Ren H, Wu Q, Bi W. A class of two-step Steffensen type methods with fourth-order convergence. Appl Math Comput. 2009;209:206–210.
27. Thukral R, Petković MS. Family of three-point methods of optimal order for solving nonlinear equations. J Comput Appl Math. 2010;233:2278–2284.
28. Traub JF. Iterative Methods for the Solution of Equations. Englewood Cliffs, New Jersey: Prentice-Hall; 1964.
29. Wang X, Liu L. New eighth-order iterative methods for solving nonlinear equations. J Comput Appl Math. 2010b;234:1611–1620.
30. Zheng Q, Li J, Huang F. Optimal Steffensen-type families for solving nonlinear equations. Appl Math Comput. 2011;217:9592–9597.