Polynomial Approximation of a Function

Introduction 4.5.1  
All polynomials are functions, but not all functions are polynomials. It is however possible to approximate any sufficiently differentiable function by a polynomial. This procedure is called a Taylor expansion.

Theorem 4.5.2 (Taylor)  
Let $f : [a,b] \to \hbox {I \hskip -5.2pt {R}}: x \mapsto f(x)$ be an $n$ times differentiable function, and let $x\not=x_0$ in $]a,b[$, then there exists a point $c$ in $]a,b[$ such that

$$f(x) = f(x_0) + \sum_{k=1}^{n-1} \frac{f^{(k)}(x_0)}{k!} (x-x_0)^k + \frac{f^{(n)}(c)}{n!} (x-x_0)^n$$ (4.5.1)

Proof.
Choose a number $M$ such that

$$f(x) = f(x_0) + \sum_{k=1}^{n-1} \frac{f^{(k)}(x_0)}{k!} (x-x_0)^k + \frac{M}{n!} (x-x_0)^n$$ (4.5.2)

Now we have to prove that there is a $c$ such that $f(c)=M$. Define

$$g(t) = f(x_0) + \sum_{k=1}^{n-1} \frac{f^{(k)}(x_0)}{k!} (t-x_0)^k + \frac{M}{n!} (t-x_0)^n - f(x)$$ (4.5.3)

Then note that $g(x)=f(x)-f(x)=0$ and $g(x_0)=f(x_0)-f(x_0)=0$, so by Rolle's Theorem, there exists an $x_1$ for which $g'(x_1)=0$.
We can play this game over and over again: $g'(x_0)=0$ and $g'(x_1)=0$, so again by Rolle's Theorem, there exists an $x_2$ for which $g''(x_2)=0$.
After $n$ steps we get an $x_n$ such that $g^{(n)}(x_n)=0$. This $x_n$ is our $c$ for which $f^{(n)}(c)=M$. $\qedsymbol$

Remark 4.5.3  
Taylor's theorem essentially proves that all $n$ times differentiable functions can be approximated by polynomials of the $n$-th degree. Note that this approximation is always local by nature: the approximation can be very good around point $c$, but will become increasingly imprecise when going away from this point.

200pt

In this example the function $\sin(x)$ is approximated by the polynomial

$$p(x) = x - \frac{1}{6} x^3 + \frac{1}{120} x^5$$ (4.5.4)

Note that the approximation is quite good in the interval $\left[-\frac{\pi}{2},\frac{\pi}{2}\right]$ but then evolves from poor to downright irrelevant when going outside the interval. It is clear that the $\sin$ function can never be accurately approximated over the entire axis by a polynomial because a polynomial tails off to plus or minus infinity for large enough numbers and the $\sin$ function spends all its life between $-1$ and $1$. On a finite interval, however, the approximation can be arbitrarily close. Taylor's theorem assures that this is true for all sufficiently differentiable functions. The following corollary is merely a formal expression of this reasoning.

Corollary 4.5.4  
Let $f : [a,b] \to \hbox {I \hskip -5.2pt {R}}: x \mapsto f(x)$ be an $n$ times differentiable function, and let $x\not=x_0$ in $]a,b[$. And let $\vert f^{(n)} (x)\vert \leq M$ for all $x$ in $[a,b]$ for any $n$. Then

$$\forall x \in ]a,b[ : f(x) = f(x_0) + \lim_{n \to +\infty} \sum_{k=1}^{n-1} \frac{f^{(k)}(x_0)}{k!} (x-x_0)^k$$ (4.5.5)

Proof.
We essentially have to prove that the rest term in the Taylor expansion converges to 0. First notice

$$0 \leq \lim_{n \to \infty}\frac{\vert f^{(n)}(c)\vert}{n!}\vert x-x_0\vert^n \leq \lim_{n \to \infty}\frac{M}{n!}(b-a)^n$$ (4.5.6)

and as $\lim_{n \to \infty}\frac{a^n}{n!}=0$ for any $a$, as we have seen in (2.0.21), we have proven the corollary. $\qedsymbol$

Remark 4.5.5  
This fact allows us to rewrite a very long list of functions as Taylor series. The number $\vert x-x_0\vert$ where the Taylor expansion is valid is called the radius of convergence. Outside this domain, the expansions are possibly no longer valid.

Taylor Series Expansions

Function	Taylor Series Expansion	Radius of CVG
$(1 \pm x)^{\frac{1}{2}}$	$$1 \pm \frac{1}{2} x - \frac{1 \cdot 1}{2 \cdot 4} x^2 \pm \frac{... ...^3 - \frac{1 \cdot 1 \cdot 3 \cdot 5}{2 \cdot 4 \cdot 6 \cdot 8} x^4 \pm \ldots$$	$-1 \leq x \leq 1$
$(1 \pm x)^{-\frac{1}{2}}$	$$1 \mp \frac{1}{2} x + \frac{1 \cdot 3}{2 \cdot 4} x^2 \mp \frac{... ...^3 + \frac{1 \cdot 3 \cdot 5 \cdot 7}{2 \cdot 4 \cdot 6 \cdot 8} x^4 \mp \ldots$$	$-1 < x < 1$
$\sin(x)$	$$x - \frac{x^3}{3!} + \frac{x^5}{5!} - \frac{x^7}{7!} + \frac{x^9}{9!} + \ldots$$	$x \in \hbox {I \hskip -5.2pt {R}}$
$\cos(x)$	$$1 - \frac{x^2}{2!} + \frac{x^4}{4!} - \frac{x^6}{6!} + \frac{x^8}{8!} + \ldots$$	$x \in \hbox {I \hskip -5.2pt {R}}$
$\tan(x)$	$$x + \frac{x^3}{3} + \frac{2x^5}{15} - \frac{17x^7}{315} + \frac{62x^9}{2835} + \ldots$$	$-\frac{\pi}{2} < x < \frac{\pi}{2}$
$e^x$	$$1 + \frac{x}{1!} + \frac{x^2}{2!} + \frac{x^3}{3!} + \frac{x^4}{4!} + \ldots$$	$x \in \hbox {I \hskip -5.2pt {R}}$
$\ln(1+x)$	$$x - \frac{x^2}{2} + \frac{x^3}{3} - \frac{x^4}{4} + \frac{x^5}{5} + \ldots$$	$x \in \hbox {I \hskip -5.2pt {R}}$

This is just the tip of the iceberg. Every self respecting formula book has about $50$ Taylor expansions listed.