Numerical Analysis/Differentiation/Examples

< Numerical Analysis < Differentiation

When deriving a finite difference approximation of the $j$ th derivative of a function $f : \mathbb{R} \rightarrow \mathbb{R}$ , we wish to find $a_1, a_2, ..., a_n \in \mathbb{R}$ and $b_1, b_2, ..., b_n \in \mathbb{R}$ such that

f^{(j)}(x_0) = h^{-j}\sum_{i=1}^{n}a_i f(x_0 + b_i h) + O(h^k) \text{ as } h \to 0

or, equivalently,

h^{-j}\sum_{i=1}^{n}a_i f(x_0 + b_i h) = f^{(j)}(x_0) + O(h^k) \text{ as } h \to 0

where $O(h^k)$ is the error, the difference between the correct answer and the approximation, expressed using Big-O notation. Because $h$ may be presumed to be small, a larger value for $k$ is better than a smaller value.

A general method for finding the coefficients is to generate the Taylor expansion of $h^{-j}\sum_{i=1}^{n}a_i f(x_0 + b_i h)$ and choose $a_1, a_2, ..., a_n$ and $b_1, b_2, ..., b_n$ such that $f^{(j)}(x_0)$ and the remainder term are the only non-zero terms. If there are no such coefficients, a smaller value for $k$ must be chosen.

For a function of $m$ variables $g:\mathbb{R}^m \rightarrow \mathbb{R}$ , the procedure is similar, except $x_0, b_1, b_2, ..., b_n$ are replaced by points in $\mathbb{R}^m$ and the multivariate extension of Taylor's theorem is used.

Single-Variable

In all single-variable examples, $x_0 \in \mathbb{R}$ and $f : \mathbb{R} \rightarrow \mathbb{R}$ are unknown, and $h \in \mathbb{R}$ is small. Additionally, let $f$ be 5 times continuously differentiable on $\mathbb{R}$ .

First Derivative

Find $a, b, c \in \mathbb{R}$ such that $\frac{a f(x_0 + h) + b f(x_0 + c h)}{h}$ best approximates $f'(x_0)$ .

Solution:

First, we find the Taylor series expansion of $\frac{a f(x_0 + h) + b f(x_0 + c h)}{h}$ with remainder term to be

\frac{a f(x_0 + h) + b f(x_0 + c h)}{h} = \frac{a + b}{h}f(x_0) + (a + b c) f'(x_0) + \frac{h}{2} (a + b c^2) f''(x_0) + O(h^2) \text{ as } h \to 0\,.

If we can find a solution to the system

\begin{align}0 &=& a + &b&\\ 1 &=& a + &b c&\\ 0 &=& a + &b c^2&\end{align}

then we can substitute that solution into the Taylor expansion to obtain

\frac{a f(x_0 + h) + b f(x_0 + c h)}{h} = f'(x_0) + O(h^2) \text{ as } h \to 0\,.

The system of equations has exactly one solution: $a = \frac{1}{2}$ , $b = \frac{1}{-2}$ , $c = -1$ , so

\frac{f(x_0 + h) - f(x_0 - h)}{2 h} = f'(x_0) + O(h^2) \text{ as } h \to 0\,.

Let $f : \mathbb{R} \rightarrow \mathbb{R}$ be 42 times continuously differentiable on $\mathbb{R}$ . Find the largest $n \in \mathbb{N}$ such that

\frac{df}{dx}(x_0) = \frac{-f(x_0 + 2 h) + 8 f(x_0 + h, y_0) - 8 f(x_0 - h) + f(x_0 - 2 h)}{12 h} + O(h^n) \text{ as } h \to 0

In other words, find the order of the error of the method.

Solution:

The Taylor expansion of the method is

\begin{align}\frac{-f(x_0 + 2 h) + 8 f(x_0 + h, y_0) - 8 f(x_0 - h) + f(x_0 - 2 h)}{12 h} &= \frac{-1 + 8 - 8 + 1}{12 h}f(x_0) + \frac{-2 + 8 + 8 - 2}{12} f'(x_0) \\ &+ h\frac{(-4 + 8 - 8 + 4)}{24} f''(x_0) + h^2\frac{(-8 + 8 + 8 - 8}{72} f'''(x_0) \\ &+ h^3\frac{(-16 + 8 - 8 + 16}{288} f^{(4)}(x_0) + h^4\frac{(-32 + 8 + 8 - 32)}{1440} f^{(4)}(x_0) + O(h^5) \text{ as } h \to 0\,.\end{align}

Simplifying this algebraically gives

\frac{-f(x_0 + 2 h) + 8 f(x_0 + h, y_0) - 8 f(x_0 - h) + f(x_0 - 2 h)}{12 h} = f'(x_0) + \frac{h^4}{-30} f^{(5)}(x_0) + O(h^5) \text{ as } h \to 0\,.

The multiple of $h^4$ cannot be removed, so $n = 4$ and by properties of Big-O notation,

\frac{-f(x_0 + 2 h) + 8 f(x_0 + h, y_0) - 8 f(x_0 - h) + f(x_0 - 2 h)}{12 h} = f'(x_0) + O(h^4) \text{ as } h \to 0\,.

Second Derivative

Find $a, b, c \in \mathbb{R}$ such that $\frac{a f(x_0 - h) + b f(x_0) + c f(x_0 + h)}{h^2}$ best approximates $f''(x_0)$ .

Solution:

First, we find the Taylor series expansion of $\frac{a f(x_0 - h) + b f(x_0) + c f(x_0 + h)}{h^2}$ , with remainder term to be

\frac{a f(x_0 - h) + b f(x_0) + c f(x_0 + h)}{h^2} = \frac{a + b + c}{h^2}f(x_0) + \frac{c - a}{h} f'(x_0) + \frac{a + c}{2} f''(x_0) + \frac{h}{6} (c - a) f'''(x_0) + O(h^2) \text{ as } h \to 0\,.

If we can find a solution to the system

\begin{align}0 &=& &a& + &b& + &c&\\ 0 &=& &-a& + &0& + &c&\\ 2 &=& &a& + &0& + &c&\\ 0 &=& &-a& + &0& + &c&\end{align}

then we can substitute that solution into the Taylor expansion and obtain

\frac{a f(x_0 - h) + b f(x_0) + c f(x_0 + h)}{h^2} = f''(x_0) + O(h^2) \text{ as } h \to 0\,.

The system of equations has exactly one solution: $a = 1$ , $b = -2$ , $c = 1$ so

\frac{f(x_0 - h) - 2 f(x_0) + f(x_0 + h)}{h^2} = f''(x_0) + O(h^2) \text{ as } h \to 0\,.

Multivariate

In all two-variable examples, $x_0, y_0 \in \mathbb{R}$ and $f : \mathbb{R}^2 \rightarrow \mathbb{R}$ are unknown, and $h \in \mathbb{R}$ is small.

Non-Mixed Derivatives

Because of the nature of partial derivatives, some of them may be calculated using single-variable methods. This is done by holding constant all but one variable to form a new function of one variable. For example if $g_y(y) = f(x_0, y)$ , then $\frac{df}{dy}(x_0, y_0) = \frac{dg}{dy}(y_0)$ .

Find an approximation of $\frac{d}{dy}f(x_0, y_0)$

Solution:

Because we are differentiating with respect to only one variable, we can hold x constant and use the result of one of the single-variable examples:

\frac{f(x_0, y_0 + h) - f(x_0, y_0 - h)}{2 h} = \frac{df}{dy}(x_0, y_0) + O(h^2) \text{ as } h \to 0

Mixed Derivatives

Mixed derivatives may require the multivariate extension of Taylor's theorem.

Let $f : \mathbb{R}^2 \rightarrow \mathbb{R}$ be 42 times continuously differentiable on $\mathbb{R}^2$ and let $g : \mathbb{R}^3 \rightarrow \mathbb{R}$ be defined as

g(x_0, y_0, h) = \frac{f(x_0 + h, y_0 + h) + f(x_0 - h, y_0 - h) - f(x_0 - h, y_0 + h) - f(x_0 + h, y_0 - h)}{4 h^2}\,.

Find the largest $n \in \mathbb{N}$ such that

\frac{d^2 f}{dx dy}(x_0, y_0) = g(x_0, y_0, h) + O(h^n) \text{ as } h \to 0\,.

In other words, find the order of the error of the approximation.

Solution:

The first few terms of the multivariate Taylor expansion of $f$ around $(x_0,y_0)$ are

\begin{align}f(x_0 + x, y_0 + y) &= f(x_0, y_0)\\ &+ x \frac{df}{dx}(x_0, y_0) + y \frac{df}{dy}(x_0, y_0)\\ &+ \frac{x^2}{2} \frac{d^2f}{dx^2}(x_0, y_0) + x y \frac{d}{dy}\frac{df}{dx}(x_0, y_0) + \frac{y^2}{2} \frac{d^2f}{dy^2}(x_0, y_0)\\ &+ \frac{x^3}{6} \frac{d^3f}{dx^3}(x_0, y_0) + \frac{x^2 y}{2} \frac{d}{dy}\frac{d^2f}{dx^2}(x_0, y_0) + \frac{x y^2}{2} \frac{d^2}{dy^2}\frac{df}{dx}(x_0, y_0) + \frac{y^3}{6} \frac{d^3f}{dy^3}(x_0, y_0)\\ &+ \frac{x^4}{24} \frac{d^4f}{dx^4}(x_0, y_0) + \frac{x^3 y}{6} \frac{d}{dy}\frac{d^3 f}{dx^3}(x_0, y_0) + \frac{x^2 y^2}{4} \frac{d^2}{dy^2}\frac{d^2 f}{dx^2}(x_0, y_0) + \frac{x y^3}{6} \frac{d^3}{dy^3}\frac{d f}{dx}(x_0, y_0) + \frac{y^4}{24} \frac{d^4f}{dy^4}(x_0, y_0)\\ &+ O(x^5) + O(y^5) \text{ as } x, y \to 0\,.\end{align}

We substitute the expansion for $f$ into the approximation $g$ to obtain

\begin{align}g(x_0, y_0, h) &= \frac{1 + 1 - 1 - 1}{4 h^2}f(x_0, y_0) \\ & + \frac{1 - 1 + 1 - 1}{4 h}\frac{df}{dx}(x_0, y_0) + \frac{1 - 1 - 1 + 1}{4 h} \frac{df}{dy}(x_0, y_0) \\ & + \frac{1 + 1 - 1 - 1}{8} \frac{d^2f}{dx^2}(x_0, y_0) + \frac{1 + 1 + 1 + 1}{4} \frac{d}{dy}\frac{df}{dx}(x_0, y_0) + \frac{1 + 1 - 1 - 1}{8} \frac{d^2f}{dy^2}(x_0, y_0) \\ & + h \left(\frac{1 - 1 + 1 - 1}{24} \frac{d^3f}{dx^3}(x_0, y_0) + \frac{1 - 1 - 1 + 1}{8} \frac{d}{dy}\frac{d^2f}{dx^2}(x_0, y_0) + \frac{1 - 1 + 1 - 1}{8} \frac{d^2}{dy^2}\frac{df}{dx}(x_0, y_0) + \frac{1 - 1 - 1 + 1}{24} \frac{d^3f}{dy^3}(x_0, y_0) \right)\\ & + h^2 \left(\frac{1 + 1 - 1 - 1}{96} \frac{d^4f}{dx^4}(x_0, y_0) + \frac{1 + 1 + 1 + 1}{24} \frac{d}{dy}\frac{d^3 f}{dx^3}(x_0, y_0) + \frac{1 + 1 - 1 - 1}{16} \frac{d^2}{dy^2}\frac{d^2 f}{dx^2}(x_0, y_0) + \frac{1 + 1 + 1 + 1}{24} \frac{d^3}{dy^3}\frac{d f}{dx}(x_0, y_0) + \frac{1 + 1 - 1 - 1}{96} \frac{d^4f}{dy^4}(x_0, y_0) \right) \\ & + O(h^3) \text{ as } h \to 0\,.\end{align}

Because of the careful choices of coefficients, we can simplify this to

g(x_0, y_0, h) = \frac{d}{dy}\frac{df}{dx}(x_0, y_0) + \frac{h^2}{6}\left(\frac{d}{dy}\frac{d^3 f}{dx^3}(x_0, y_0) + \frac{d^3}{dy^3}\frac{d f}{dx}(x_0, y_0)\right) + O(h^3) \text{ as } h \to 0\,.

We note that Big-O notation permits us to write the last 3 terms as $O(h^2) \text{ as } h \to 0$ . Thus,

g(x_0, y_0, h) = \frac{d}{dy}\frac{df}{dx}(x_0, y_0) + O(h^2) \text{ as } h \to 0\,.

Because the multiples of $h^2$ are unaffected by adding more terms to the Taylor expansion, $n = 2$ is the greatest natural number satisfying the conditions given in the problem.

Example Code

Implementing these methods is reasonably simple in programming languages that support higher-order functions. For example, the method from the first example may be implemented in C++ using function pointers, as follows:

//  Returns an approximation of the derivative of f at x.
double derivative (double (*f)(double), double x, double h =0.01) {
  return (f(x + h) - f(x - h)) / (2 * h);
}

This article is issued from Wikiversity - version of the Saturday, April 02, 2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.