User:Emil.catinas/sandbox

Convergence order

The speed at which a convergent sequence $$(x_k)_{k\geq0}$$ approaches its finite limit $$x^\ast$$ is measured by several mathematical definitions - the convergence orders, often referred to as rates of convergence.

It is usual to say - but this is only half of the story - that the sequence $$(x_k)_{k\geq0}$$ converges quadratically if for some constant c>0 one has


 * $$|x^\ast-x_{k+1}|\leq c|x^\ast-x_k|^2, \qquad \forall k\geq k_0. \,$$

Such inequalities only imply that the lower Q-order of the sequence is at least 2.

Although strictly speaking, the definition of a limit does not give information about any finite first part of the sequence, the concept of convergence order is of practical importance when working with a sequence generated by an iterative method, as then typically fewer iterations are needed to yield a useful approximation if the convergence order is high (especially for reasonably good initial guesses). This may even make the difference between needing ten or a million iterates.

Similar concepts are used for discretization methods. The solution of the discretized problem converges to the solution of the continuous problem as the grid size goes to zero, and the speed of convergence is one of the factors of the efficiency of the method. However, the terminology in this case is different from the terminology for iterative methods.

Series acceleration is a collection of techniques for improving the (linear) rate of convergence of a series discretization??. Such acceleration is commonly accomplished with sequence transformations.

The convergence orders, are important notions in Numerical Analysis and in Mathematical Analysis.

High convergence orders (p>1)
Convergence orders of sequences from R

Orders defined by errors

We first discuss orders defined by the errors $$|x^\ast-x_{k}|$$.

Convergence order (classical order or C-order)

Consider a sequence $$(x_k)_{k\geq0}$$ from R, which converges to a finite limit $$x^\ast$$.

We analyze first the quadratic convergence (which is most encountered in some iterative methods such as Newton method), the cubic convergence, and then we give the definition of the arbitrary order p₀>1.

Quadratic convergence (classical quadratic or C-quadratic).

The sequence $$(x_k)_{k\geq0}$$ converges C-quadratically (with C-order 2) to x^{∗} if there exists the finite and nonzero value C₂ (called asymptotic error constant) such that the following limit exists

lim_{k→∞}((|x^{∗}-x_{k+1}|)/(|x^{∗}-x_{k}|²))=C₂. def C2 quadr order

Example

x_{k}=((1/2))^{2^{k}},k≥0

for which C₂=1.

Example The Newton method for solving the nonlinear equation f(x)=0, under some standard assumptions, converges locally with C-order 2 and the asymptotic constant is given by C₂=|((f′′(x^{∗}))/(2f′(x^{∗})))|.

Remark Relation implies that ∀ε>0,∃k₀≥0 such that

(C₂-ε)|x^{∗}-x_{k}|²≤|x^{∗}-x_{k+1}|≤(C₂+ε)|x^{∗}-x_{k}|²,∀k≥k₀.

When the asymptotic constant C₂ is not very large, relations above say that the error is approximately squared at each step. Another interpretation (first made by Sir I. Newton, in 1675) is that the number of significant digits is doubled at each iteration step [Ypma95]. The definition was first given, in a more intuitive fashion, by J. Fourier in 1818 [?], for an arbitrary order p₀?.

An intriguing observation was made by Igarashi and Ypma[Iga-Ypma], who noted that, in general, a smaller value for C₂ does not necessarily lead to a faster numerical/practical convergence of the iterates.

We notice that if $$(x_k)_{k\geq0}$$ converges C-quadratically, then

lim_{k→∞}((|x^{∗}-x_{k+1}|)/(|x^{∗}-x_{k}|^{p})) =0,∀pwith02.

The proof is immediate, as if p<2, we can write p=2-ε and then ((|x^{∗}-x_{k+1}|)/(|x^{∗}-x_{k}|^{2-ε}))=((|x^{∗}-x_{k+1}|)/(|x^{∗}-x_{k}|²))|x^{∗}-x_{k}|^{ε}→C₂=0. One proceeds similarly when p>2.

Relations (?) above (which are used in fact in defining the Q-quadratic convergence below) are explained by the fact that in case of C-quadratic convergence, the value 2 in the exponent of the denominator from (?) is "finely tuned": trying to lower the exponent leads to a fraction with magnitude tending to zero (denominator too large), trying to raise it leads to a fraction with magnitude tending to infinity (denominator too small).

Convergence profile of a sequence (I)

The expressions

Q_{p}(k):=((|x^{∗}-x_{k+1}|)/(|x^{∗}-x_{k}|^{p}))

are called quotient convergence factors.[OR70]

For the given sequence $$(x_k)_{k\geq0}$$, by considering these quotient convergence factors for all p≥1 and denoting their limit points by

Q_{p}:=lim_{k→∞}Q_{p}(k)=lim_{k→∞}((|x^{∗}-x_{k+1}|)/(|x^{∗}-x_{k}|^{p})),

we may represent Q_{p} as a "function" of p, obtaining a "graph", which, by an abuse of notation, contains the representation of the infinity. When the limit does not exist, the graph is extended to represent lim infQ_{p}(k) and lim supQ_{p}(k), leading to what was called in [BEQ90] as the convergence profile of the sequence).

The C-quadratic convergence requires a graph of the following form: zero value for Q_{p} for all 02. The Q-quadratic convergence defined below demands the same conditions, excepting at p=2, (where the limit Q₂ may be zero, infinite, or may not exist at all). Cubic convergence (C-cubic order)

The sequence converges C-cubically (with C-order 3), when

lim_{k→∞}((|x^{∗}-x_{k+1}|)/(|x^{∗}-x_{k}|³))=C₃∈(0,+∞)

Example x_{k}=2^{-3^{k}} ,C₃=1?. C-order p₀

Definition. The sequence $$(x_k)_{k\geq0}$$ converges to x^{∗} with C-order p₀>1 if there exists the finite and nonzero asymptotic error constant C_{p₀} such that

lim_{k→∞}((|x^{∗}-x_{k+1}|)/(|x^{∗}-x_{k}|^{p₀}))=C_{p₀}.

The C-order p₀ from the above formula, if exists, is uniquely defined.

Remark. The convergence order does not necessarily have an integer value:

Example. The secant method, under some standard assumptions, converges locally with C-order p₀=((1+√5)/2)=1.68… (the golden ratio) and the asymptotic constant is given by C_{p₀}=|((f′′(x^{∗}))/(2f′(x^{∗})))|^{p₀-1} [see Ostrowski][Brent-Winograd-Wolfe].

Example. Even convergence with infinite C-order exists (i.e., for which Q_{p}=0, ∀p>1). Indeed, let

x_{k}=((1/2))^{2^{2^{k}}},k

Is the C-order enough to measure the convergence order of all sequences? The answer is negative,

As we shall see, in order to define a useful definition of convergence order, the essential property of the convergence profile of the sequence is that from figure ??, except the value at the jump point p₀.