1.6 Global Convergence and the Armijo Rule

The requirement in the local convergence theory that the initial iterate be near the solution is more than mathematical pedantry. To see this, we apply Newton's method to find the root x* = 0 of the function F( x) = arctan( x) with initial iterate x ₀ = 10. This initial iterate is too far from the root for the local convergence theory to hold. In fact, the step

while in the correct direction, is far too large in magnitude.

The initial iterate and the four subsequent iterates are

As you can see, the Newton step points in the correct direction, i.e., toward x* = 0, but overshoots by larger and larger amounts. The simple artifice of reducing the step by half until F ( x) has been reduced will usually solve this problem.

In order to clearly describe this, we will now make a distinction between the Newton direction d = ? F ?( x) ^?1 F( x) and the Newton step when we discuss global convergence. For the methods in this book, the Newton step will be a positive scalar multiple of the Newton direction. When we talk about local convergence and are taking full steps ( ? = 1 and s = d), we will not make this distinction and only refer to the step, as we have been doing up...