Linear Approximation

In mathematics, a linear approximation is an approximation of a general function using a linear function (more precisely, an affine function). Linear approximations are widely used to solve (or approximate solutions to) equations. Linear approximation is achieved by using Taylor's theorem to approximate the value of a function at a point.

Given a twice continuously differentiable function $f$ of one real variable, Taylor's theorem states that:

$f(x)=f(a)+f'(a)(x-a)+R_2$

where $R_2$ is the remainder term (the difference between the actual value of $f(x)$ and the approximation found by the addition of the first two terms).

The linear approximation is obtained by dropping the remainder:

$f(x)=f(a)+f'(a)(x-a)$

This is a good approximation for $x$ when it is close enough to $a$; since a curve, when observed on a smaller and smaller scale, will begin to resemble a straight line. Therefore, the expression on the right-hand side is just the equation for the tangent line to the graph of $f$ at $(a, f(a))$. For this reason, this process is also called the tangent line approximation.

If $f$ is concave-down in the interval between $x$ and $a$, the approximation will be an overestimate (since the derivative is decreasing in that interval). If $f$ is concave-up, the approximation will be an underestimate.

Since the line tangent to the graph is given by the derivative, differentiation is useful for finding the linear approximation. If one were to take an infinitesimally small step size for $a$, the linear approximation would exactly match the function.

Linear approximations for vector functions of a vector variable are obtained in the same way, with the derivative at a point replaced by the Jacobian matrix. For example, given a differentiable function with real values, one can approximate for close to by the following formula:

$\displaystyle{f(x,y)~f(a,b)+\frac{df}{dx}(a,b)(x-a)+\frac{df}{dy}(a,b)(y-b)}$