The derivative of $f(x)$ has the interpretation as the slope of the tangent line. The tangent line is the line that best approximates the function at the point.
Using the point-slope form of a line, we see that the tangent line to the graph of $f(x)$ at $(c,f(c))$ is given by:
\[ ~ y = f(c) + f'(c) \cdot (x - c). ~ \]
This is written as an equation, though we prefer to work with functions within Julia. Here we write such a function as an operator - it takes a function f and returns a function representing the tangent line.
tangent(f, c) = x -> f(c) + f'(c) * (x - c)
(Recall, the -> indicates that an anonymous function is being generated.)
This function along with the f' notation, as defined in the CalculusWithJulia package, through Base.adjoint(f::Function)=x->ForwardDiff.derivative(f, float(x)), is loaded with:
using CalculusWithJulia using Plots
To illustrate the tangent line, we can make some graphs:
f(x) = x^2 plot(f, -3, 3) plot!(tangent(f, -1), -3, 3) plot!(tangent(f, 2), -3, 3)
The graph shows that near the point, the line and function are close, but this need not be the case away from the point. We can express this informally as
\[ ~ f(x) \approx f(c) + f'(c) \cdot (x-c) ~ \]
with the understanding this applies for $x$ "close" to $c$.
This section gives some implications of this fact and quantifies what "close" can mean.
There are several approximations that are well known in physics, due to their widespread usage. Three are:
That $\sin(x) \approx x$ around $x=0$:
f(x) = sin(x) plot(f, -pi/2, pi/2) plot!(tangent(f, 0), -pi/2, pi/2)
That $\log(1 + x) \approx x$ around $x=0$:
f(x) = log(1 + x) plot(f, -1/2, 1/2) plot!(tangent(f, 0), -1/2, 1/2)
That $1/(1-x) \approx x$ around $x=0$:
f(x) = 1/(1-x) plot(f, -1/2, 1/2) plot!(tangent(f, 0), -1/2, 1/2)
In each of these cases, a more complicated non-linear function is well approximated in a region of interest by a simple linear function.
The plot shows the tangent line with slope $dy/dx$ and the actual change in $y$, $\Delta y$, for some specified $\Delta x$. The small gap above the sine curve is the error were the value of the sine approximated using the drawn tangent line. We can see that approximating the value of $\Delta y = \sin(c+\Delta x) - \sin(c)$ with the often easier to compute $f'(c)\Delta x$ - for small enough values of $\Delta x$ - is not going to be too far off provided $\Delta x$ is not too large.
This approximation is known as linearization. It can be used both in theoretical computations and in pratical applications. To see how effective it is, we look at some examples.
If $f(x) = \sin(x)$, $c=0$ and $\Delta x= 0.1$ then the values for the actual change in the function values and the value of $\Delta y$ are:
f(x) = sin(x) c, deltax = 0, 0.1 f(c + deltax) - f(c), f'(c) * deltax
(0.09983341664682815, 0.1)
The values are pretty close. But what is $0.1$ radians? Lets use degrees. Suppose we have $\Delta x = 10^\circ$:
c, deltax = 0, 10*pi/180 actual=f(c + deltax) - f(c) approx = f'(c) * deltax actual, approx
(0.17364817766693033, 0.17453292519943295)
They agree until the third decimal value. The percentage error is just $1/2$ a percent:
(approx - actual) / actual * 100
0.5095057975210231
We are traveling 60 miles. At 60 miles an hour, we will take 60 minutes (or one hour). How long will it take at 70 miles an hour? (Assume you can't divide, but, instead, can only multiply!)
Well the answer is $60/70$ hours or $60/70 \cdot 60$ minutes. But we can't divide, so we turn this into a multiplication problem via some algebra:
\[ ~ \frac{60}{70} = \frac{60}{60 + 10} = \frac{1}{1 + 10/60} = \frac{1}{1 + 1/6}. ~ \]
Okay, so far no calculator was needed. We wrote $70 = 60 + 10$, as we know that $60/60$ is just $1$. This almost gets us there. If we really don't want to divide, we can get an answer by using the tangent line approximation for $1/(1+x)$ around $x=0$. This is $1/(1+x) \approx 1 - x$. (You can check by finding that $f'(0) = -1$.) Thus, our answer is approximately $5/6$ of an hour or 50 minutes.
How much in error are we?
abs(50 - 60/70*60) / (60/70*60) * 100
2.7777777777777684
That's about $3$ percent. Not bad considering we could have done all the above in our head while driving without taking our eyes off the road to use the calculator on our phone for a division.
A 10cm by 10cm by 10cm cube will contain 1 liter (1000cm$^3$). In manufacturing such a cube, the side lengths are actually $10.1$ cm. What will be the volume in liters? Compute this with a linear approximation to $(10.1)^3$.
Here $f(x) = x^3$ and we are asked to approximate $f(10.1)$. Letting $c=10$, we have:
\[ ~ f(c + \Delta) \approx f(c) + f'(c) \cdot \Delta = 1000 + f'(c) \cdot (0.1) ~ \]
Computing the derivative can be done easily, we get for our answer:
fp(x) = 3*x^2 c, Delta = 10, 0.1 approx = 1000 + fp(c) * Delta
1030.0
This is a relative error as a percent of:
actual = 10.1^3 (actual - approx)/actual * 100
0.029214763452615394
The manufacturer may be interested instead in comparing the volume of the actual object to the $1$ liter target. They might use the approximate value for this comparison, which would yield:
(1000 - approx)/approx * 100
-2.912621359223301
This is off by about $3$ percent. Not so bad for some applications, devastating for others.
A simple pendulum is comprised of a massless "bob" on a rigid "rod" of length $l$. The rod swings back and forth making an angle $\theta$ with the perpendicular. At rest $\theta=0$, here we have $\theta$ swinging with $\lvert\theta\rvert \leq \theta_0$ for some $\theta_0$.
According to Wikipedia - and many introductory physics book - while swinging, the angle $\theta$ varies with time following this equation:
\[ ~ \theta''(t) + \frac{g}{l} \sin(\theta(t)) = 0. ~ \]
That is, the second derivative of $\theta$ is proportional to the sine of $\theta$ where the proportionality constant involves $g$ from gravity and the length of the "rod."
This would be much easier if the second derivative were proportional to the angle $\theta$ and not its sine.
Huygens used the approximation of $\sin(x) \approx x$, noted above, to say that when the angle is not too big, we have the pendulum's swing obeying $\theta''(t) = -g/l \cdot t$. Without getting too involved in why, we can verify by taking two derivatives that $\theta_0\sin(\sqrt{g/l}\cdot t)$ will be a solution to this modified equation.
This says the motion is periodic with constant amplitude (assuming frictionless behaviour), as the sine function is. More surprisingly, the period is found from $T = 2\pi/(\sqrt{g/l}) = 2\pi \sqrt{l/g}$. It depends on $l$ - longer "rods" take more time to swing back and forth - but does not depend on the how wide the pendulum is swinging between (provided $\theta_0$ is not so big the approximation of $\sin(x) \approx x$ fails). This latter fact may be surprising, though not to Galileo who discovered it.
How good is the approximation? Graphically we can see it is pretty good for the graphs we choose, but are there graphs out there for which the approximation is not so good? Of course. However, we can say this (the Lagrange form of a more general Taylor remainder theorem):
Let $f(x)$ be twice differentiable on $I=(a,b)$, and $a < c < b$. Then for any $x$ in $I$, there exists some value $\xi$ between $c$ and $x$ such that $~f(x) = f(c) + f'(c)(x-c) + f''(\xi)\cdot(x-c)^2/2.~$
That is, the error is basically a constant depending on the concavity of $f$ times a quadratic function centered at $c$.
For $\sin(x)$ at $c=0$ we get $\lvert\sin(x) - x\rvert = \lvert-\sin(\xi)\cdot x^2/2\rvert$. Since $\lvert\sin(\xi)\rvert \leq 1$, we must have this bound: $\lvert\sin(x) - x\rvert \leq x^2/2$.
Can we verify? Let's do so graphically:
h(x) = abs(sin(x) - x) g(x) = x^2/2 plot(h, -2, 2) plot!(g, -2, 2)
Similarly, for $f(x) = \log(1 + x)$ we have the following at $c=0$:
\[ ~ f'(x) = 1/(1+x), \quad f''(x) = -1/(1+x)^2. ~ \]
So, as $f(c)=0$ and $f'(c) = 1$, we have
\[ ~ \lvert f(x) - x\rvert \leq \lvert f''(\xi)\rvert \cdot \frac{x^2}{2} ~ \]
We see that $\lvert f''(x)\rvert$ is decreasing for $x > -1$. So if $-1 < x < c$ we have
\[ ~ \lvert f(x) - x\rvert \leq \lvert f''(x)\rvert \cdot \frac{x^2}{2} = \frac{x^2}{2(1+x)^2}. ~ \]
And for $c=0 < x$, we have
\[ ~ \lvert f(x) - x\rvert \leq \lvert f''(0)\rvert \cdot \frac{x^2}{2} = x^2/2. ~ \]
Plotting we verify:
h(x) = abs(log(1+x) - x) g(x) = x < 0 ? x^2/(2*(1+x)^2) : x^2/2 plot(h, -0.5, 2) plot!(g, -0.5, 2)
To see formally why the remainder is as it is, we recall the mean value theorem in the extended form of Cauchy. Suppose $c=0$ and let $h(x) = f(x) - (f(0) + f'(0) x)$ and $g(x) = x^2$. Then we have that there exists a $e$ with $0 < e < x$ such that
\[ ~ \text{error} = h(x) - h(0) = (g(x) - g(0)) \frac{h'(e)}{g'(e)} = x^2 \cdot \frac{1}{2} \cdot \frac{f'(e) - f'(0)}{e} = x^2 \cdot \frac{1}{2} \cdot f''(\xi). ~ \]
The value of $\xi$, from the mean value theorem applied to $f'(x)$, satisfies $0 < \xi < e < x$, so is in $[0,x]$.
SymPy can find the tangent line expression as a special case of its series function (which implements Taylor series). Here we see the answer provided for $e^{\sin(x)}$:
@vars x series(exp(sin(x)), x, 0, 2)
We see the expression $1 + x$ which comes from the fact that exp(sin(0)) is $1$, and the derivative exp(sin(0)) * cos(0) is also$1$. But what is the $\mathcal{O}(x^2)$?
We know the answer is precisely$f''(\xi)/2 \cdot x^2$, but were we only concerned about the scale as $x$ goes to zero we would say this is some well-behaved value (converging to $1 - f''(0)/2$) times $x^2$. The big "oh" notation says just that: were we take this term and divide by $x^2$ the limit - if it exists - would be bounded. A little "oh" (e.g., $\mathcal{o}(x^2)$) would mean that limit would be $0$.
Big "oh" and little "oh" give us a sense of how good an approximation is without being bogged down in the details of the exact value. As such they are useful guides in focusing on what is primary and what is secondary. Applying this to our case, we have this rough form of the tangent line approximation valid for functions having a second derivative:
\[ ~ f(x) = f(c) + f'(c)(x-c) + \mathcal{O}((x-c)^2). ~ \]
Suppose $f(x)$ and $g(x)$ are represented by their tangent lines about $c$, respectively:
\[ ~ f(x) = f(c) + f'(c)(x-c) + \mathcal{O}((x-c)^2), \quad g(x) = g(c) + g'(c)(x-c) + \mathcal{O}((x-c)^2). ~ \]
Consider the sum, after rearranging we have:
\[ ~ \begin{align} f(x) + g(x) &= f(c) + f'(c)(x-c) + \mathcal{O}((x-c)^2) + g(c) + g'(c)(x-c) + \mathcal{O}((x-c)^2)\\ &= (f(c) + g(c)) + (f'(c)+g'(c))(x-c) + \mathcal{O}((x-c)^2). \end{align} ~ \]
The two big "Oh" terms become just one as the sum of a constant times $(x-c)^2$ plus a constant time $(x-c)^2$ is just some other constant times $(x-c)^2$. What we can read off from this is the term multiplying $(x-c)$ is just the derivative of $f(x) + g(x)$ (from the sum rule), so this too is a tangent line approximation.
Is it a coincidence that s basic algebraic operation wit tangent lines approximations produces a tangent line approximation? Let's try multiplication:
\[ ~ \begin{align} f(x) \cdot g(x) &= [f(c) + f'(c)(x-c) + \mathcal{O}((x-c)^2)] \cdot [g(c) + g'(c)(x-c) + \mathcal{O}((x-c)^2)]\\ &=[(f(c) + f'(c)(x-c)] \cdot [g(c) + g'(c)(x-c)] + (f(c) + f'(c)(x-c) \cdot \mathcal{O}((x-c)^2)) + g(c) + g'(c)(x-c) \cdot \mathcal{O}((x-c)^2)) + [\mathcal{O}((x-c)^2))]^2\\ &= [(f(c) + f'(c)(x-c)] \cdot [g(c) + g'(c)(x-c)] + \mathcal{O}((x-c)^2)\\ &= f(c) \cdot g(c) + [f'(c)\cdot g(c) + f(c)\cdot g'(c)] \cdot (x-c) + [f'(c)\cdot g'(c) \cdot (x-c)^2 + \mathcal{O}((x-c)^2)] \\ &= f(c) \cdot g(c) + [f'(c)\cdot g(c) + f(c)\cdot g'(c)] \cdot (x-c) + \mathcal{O}((x-c)^2) \end{align} ~ \]
The big "oh" notation just sweeps up many things including any products of it and the term $f'(c)\cdot g'(c) \cdot (x-c)^2$. Again, we see from the product rule that this is just a tangent line approximation for $f(x) \cdot g(x)$.
In conclusion, basic mathematical operations involving tangent lines can be computed just using the tangent lines when the desired accuracy is at the tangent line level. This is even true for composition, though there the outer and inner functions may have different "$c$"s.
Knowing this can simplify the task of finding tangent line approximations of compound expressions.
For example, suppose we know that at $c=0$ we have these formula where $a \approx b$ is a shorthand for the more formal $a=b + \mathcal{O}(x^2)$:
\[ ~ \sin(x) \approx x, \quad e^x \approx 1 + x, \quad \text{and}\quad 1/(1+x) \approx 1 - x. ~ \]
Then we can immediately see these tangent line approximations about $x=0$:
\[ ~ e^x \cdot \sin(x) \approx (1+x) \cdot x = x + x^2 \approx x, ~ \]
and
\[ ~ \frac{\sin(x)}{e^x} \approx \frac{x}{1 + x} \approx x \cdot(1-x) = x-x^2 \approx x. ~ \]
Since $\sin(0) = 0$, we can use these to find the tangent line approximation of
\[ ~ e^{\sin(x)} \approx e^x \approx 1 + x. ~ \]
Note that $\sin(\exp(x))$ is approximately $\sin(1+x)$ but not approximately $1+x$, as the expansion for $\sin$ about $1$ is not simply $x$.
What is the right linear approximation for $\sqrt{1 + x}$ near $0$?
What is the right linear approximation for $(1 + x)^k$ near $0$?
What is the right linear approximation for $\cos(\sin(x))$ near $0$?
What is the right linear approximation for $\tan(x)$ near $0$?
What is the right linear approximation of $\sqrt{25 + x}$ near $x=0$?
Let $f(x) = \sqrt{x}$. Find the actual error in approximating $f(26)$ by the value of the tangent line at $(25, f(25))$ at $x=26$.
An estimate of some quantity was $12.34$ the actual value was $12$. What was the percentage error?
Find the percentage error in estimating $\sin(5^\circ)$ by $5 \pi/180$.
The side length of a square is measured roughly to be $2.0$ cm. The actual length $2.2$ cm. What is the difference in area (in absolute values) as estimated by a tangent line approximation.