4.) Numerical Differentiation 167 that, given any continuous function defined on a closed interval, there exists a polynomial that is arbitrarily close to the function at every point in the interval. Also, the derivatives and integrals of polynomials are eas- fly obtained and evaluated. It should not be surprising, then, that most Procedures for approximating integrals and derivatives use the polyno- mials that approximate the function. ee 4.1 Numerical Differentiation ‘The derivative of the function f at x9 is Sexy = tum £044) = fo) GCE pe ccrecree nh receecee This formula gives an obvious way to generate an approximation to f"(x): simply compute f(t +A) — FG) h for small values of k. Although this may be obvious, it is not very successful. due to our old nemesis, roundoff error. But it is certainly the place to start. Yo approximate f’(xo), suppose first that to € (a, 6), where f € C7[a. 6]. and that x1 = 49 +h for some A # 0 that is sufficiently small to ensure that x) © [ah], We construct the first Lagrange polynomial Py,.(x) for f determined by xy and «;. with its error term: G = 0) = 41) f= Pos) + Go (E(x) Fete so BY) Fv t RX ae Fr i , for some &(x) in {a, b, Differentiating gives fl%e +h) — Fld | py For ~ ede — we) ey] OG) = coerce pees 0 | eerie pereereeeee oe) | FE) by = Di(fER)) ym L004 N= feo £GFigure 4.1 EXAMPLE 1 Table 4.1 CHAPTER 4 «+ Numerical Differentiation and integration One difficulty with this formula is that we have no information about D, f"(&(x)), so the truncation error cannot be estimated. When x is xo, however, the coefficient of D. f"(E(#)) is 0, and the formula simplifies to (xo +h) — f (ro! mings f'(ay) = LE AN= LOO _F gy 4) For small values of h, the difference quotient [f(x +h) ~ f(0)1/h can be used to approximate f"(xo) with an error bounded by M|/il/2, where M is a bound on | f"¢x)| for x € [a,b]. This formula is known as the forward-difference formula if h > 0 (see Figure 4.1) and the backward-difference formula if h < 0. ' Slope fx) flea + A) flr) Stope “2 Let f(x) = Inx and xo = 1.8. The forward-difference formula SO.8+1)~ f.8) h is used iv appruaimaie /"(1.8) with enor Wag") _ Uhl It ao a 0 and right-endpoint approximations with k < 0.172 EXAMPLE 2 Table 4.2 CHAPTER 4 = Numerical Differentiation and Integration Values for f(x) = xe* are given in Table 4.2. x fey 18 10,889365 19 12.703199 20 © 14.78112 21 17.148957 22 19,855030 Since f'(x) = (x + 1)e%, we have f"(2.0) = 22.167168. Approximating /’(2.0) using the various three- and five-point formulas produces the following results. Three-Point Formulas Using (4.4) with h = 0.1: |—3f (2.0) + 4f (2.1) — F(2.2)] = 22.032310, 3f(2.0) +4 f (1.9) — f(1.8)] = 22.054525, Using (4.4) with h = -0.1 : by Using (4.5) with A = 0.1: gs1 f (2.1) — f (1.9)] = 2.228790, Using (4.5) with h = 0.2: AGLf(2.2) - f(1.8)] = 2.414163. The errors in the formulas are approximately 135 x 1071, 1.13 x 107', 6.16 x 107, and — 2.47 x 107", respectively. Five-Point Formula Using (4.6) with h = 0.1 (the only five-point formula applicable): Gia® — 8F(1.9) + 8f(2.1) — f(2.2)] = 22.166996. ‘The error in this formuia is approxisatciy 1.69 x 10°. ‘The five-point formula is clearly superior. Note also that the error from Eq, (4,5) with J}, — 0.1 is approwimately half of the magnitde of tha error produced using Fa (4 4) with either h = 0.1 or h = —0.1 . Methods can also be derived to find approximations to higher derivatives of a function using only tabulated values of the function at various points. The derivation is algebraically tedious, however, $0 only a representative procedure will be presented. Expand a function f in a third Taylor polynomial about a point xo and evaluate at xo +h and xo —h. Then 1 1 Flo +h) = Fl20) + FOR AF cok + LCN? + EOEEXAMPLE 3 4.1 Numerical Differentiation 173 and 1 1 $0) = fsa) — f*eodh-+ EF aay? = 2 FMM? + FEI, where x —h <6 <39 <6 0 and that the third derivative of f is bounded by a number M > 0, then Feo+h)— f(to—h)| _& f'0) = To reduce the truncation error, h?M /6, we must reduce h. But as h is reduced, the roundoff. error ¢/h grows, In practice, then, it is seldom advantageous to let h be too small since the roundoff error will dominate the calculations. Consider using the values in Table 4.3 to approximate /"(0.900), where f(x) = sinx. The true value is cos 0,900 = 0.62161 0.800 0.71736 0.901-—0.78395 0.850 0.75128 = 0.902 0.78457 0.880 0.77074 = 0.9080.78643 0.890 0.77707 0.910—_0.78950 0.895 0.78021 0920 ——0,79560. 0.898 — 0.78208. 0.950 0.81342. 9.899 0.78270 1.000 gata? Using the formula £0. 900-4 hy — F(0.900 — hy F'@.900) = 2h with di values of h, gives the approximation The optimal choice for h appears to lie between 0.005 and 0.05. If we perform some analysis on the error term, eh) =o + w eM zieTable 4.4 4.1. Numerical Differentiation 175 ‘Approximation h to f'(0.900) Error 0.001 0.62500 0,00339 0.002 0.62250 0.00089 0.005 0.62200 0.00039 0.010 0.62150, 0.00011 0.020 0.62150 0.00011 0.050 0.62140, 0.00021 0.100 0.62055, 0.00106 we can use calculus to verify (see Exercise 23) that a minimum for e occurs at h = /3e7M, where M= max |f"(x)| = max [cos x| = cos 0.8 © 0.69671 10 8009.00 xef0.8001.00) Since values of f are given to five decimal places, it is reasonable to assume that the round- off error is bounded by e = 0.000005. Therefore, the optimal choice of h is approximately (0.000005) _ 06871 = 0.028, a which is consistent with the results in Table 4.4. In practice, we cannot compute an optimal / (0 use in approximating he derivative, since we have no knowledge of the third derivative of the function. But we must remain aware that reducing the step size will not always improve the approximation. . ‘We have considered only the roundoff-error problems that are presented by the three- point formula Eq. (4.5), but similar difficulties occur with all the differentiation formulas. ‘The reason can be traced to the need to divide by a power of h. As we found in Section 1.2 (see, in particular, Example 3), division by small numbers tends to exaggerate roundoff error and this operation should be avoided if possible. In the case of numerical differen- tiation. it is impossible to avoid the problem entirely. althongh the higher-omlor methods reduce the difficulty. Keep in mind that as an approximation method, numerical differentiation is unstable, since the small values of h needed to reduce truncation ertor also cause the roundoff error to grow. This is the first class of unstable methods we have encountered, and these techniques, purposes, the formulas are needed for approximating the solutions of ordinary and partial- differential ecuations, sr EXERCISE SET 4.1 1. Use the forward-difference formulas and backward-difference formulas to determine each missing entry in the following tables.