Chapter 06

Derivative and Derivative Function: Instantaneous Slope, the Compass of Learning

Differentiation gives the instantaneous rate of change (slope) at a point. The derivative as a function is the basis for gradient descent and backprop in deep learning.

Math diagram by chapter

Select a chapter to see its diagram below. View the flow of basic math at a glance.

Left: Tangent line — the line that touches the curve at exactly one point. Its slope is the derivative there. Right: Line through two points — the line through two points on the curve. As the points get closer, this line approaches the tangent, and that limit is the derivative.

Pick one point (2, 4) on the curve $y=x^2$ . We want to measure the slope at this point.

The derivative is the slope of the tangent line. The limit of the "line through two points" slopes (as the points get closer) is the tangent slope.

What are differentiation and the derivative?

The derivative is the instantaneous rate of change at a point on the curve—how steep the graph is there. Geometrically it is the slope of the tangent line at that point. Just as a speedometer shows the speed at each instant, the derivative tells you how sensitively the output

y

responds when the input

x

changes by a tiny amount.

Mathematically, this slope is obtained by taking the limit (Limit) of the 'average slope between two very close points' and then sending the distance between the two points to zero. The formula is

f^{\prime}(a) = \lim_{h \to 0} \frac{f(a+h)-f(a)}{h}

(the denominator

h

is the distance between two points, the numerator is the change in

f

). The value obtained through this process is called the derivative coefficient and is written as

f^{\prime}(a)

. A large value means a steep slope; zero means flat.

The derivative function is the new function that assigns to each

x

the slope at that point. So instead of computing the slope at every point by hand, you plug

x

into the derivative formula—it is a "slope generator." The process of finding it is called differentiation.

Item $Below$
Description $Common differentiation formulas .$

Item $'$
Description $Means the derivative.$

Item	Description
Below	Common differentiation formulas.
$'$	Means the derivative.

Formula $Constant$
Description $derivative is 0$

Formula $x^n$
Description $n$

Formula $e^x$
Description $e^x$

Formula $a^x$
Description $a^x \ln a$

Formula $\ln x$
Description $1/x$

Formula $Log (base a)$
Description $1/(x \ln a)$

Formula $sin$
Description $derivative is cos$

Formula $cos$
Description $-\sin$

Formula $tan$
Description $1/\cos^2$

Formula $Sum/difference$
Description $differentiate each part and add or subtract$

Formula $Constant multiple$
Description $keep constant, differentiate the rest$

Formula $Product rule$
Description $(first')\timessecond + first\times(second')$

Formula $Quotient rule$
Description $(top'\timesbottom - top\timesbottom') / bottom²$

Formula $Intuition (product)$
Description $first derivative \times second + first \times second derivative. In the limit the two changes add up, so you get these two terms.$

Formula $Intuition (quotient)$
Description $Write the quotient as top \times (1/bottom) and use the product rule. The derivative of (1/bottom) gives the usual formula.$

Formula	Description
Constant	derivative is 0
Power $x^n$	bring down $n$ , exponent $n-1$
Exponential $e^x$	derivative is still $e^x$
Exponential $a^x$	$a^x \ln a$
Natural log $\ln x$	$1/x$
Log (base a)	$1/(x \ln a)$
sin	derivative is cos
cos	derivative is $-\sin$
tan	derivative is $1/\cos^2$
Sum/difference	differentiate each part and add or subtract
Constant multiple	keep constant, differentiate the rest
Product rule	(first′)×second + first×(second′)
Quotient rule	(top′×bottom − top×bottom′) / bottom²
Intuition (product)	first derivative × second + first × second derivative. In the limit the two changes add up, so you get these two terms.
Intuition (quotient)	Write the quotient as top × (1/bottom) and use the product rule. The derivative of (1/bottom) gives the usual formula.

Formula $Constant$
Numeric example $(5)'=0, (-3)'=0$
Solution step $derivative is 0$

Formula $x^n$
Numeric example $(x^3)'=3x^2$
Solution step $n$

Formula $e^x$
Numeric example $x=0$
Solution step $unchanged by differentiation$

Formula $a^x$
Numeric example $(2^x)'=2^x \ln 2$
Solution step $\ln a$

Formula $\ln x$
Numeric example $x=5$
Solution step $1/x$

Formula $sin$
Numeric example $x=0$
Solution step $sin \to cos$

Formula $cos$
Numeric example $x=0$
Solution step $-\sin$

Formula $Sum$
Numeric example $(x^2+x)'=2x+1$
Solution step $differentiate each term and add$

Formula $Constant multiple$
Numeric example $(5x^2)'=10x$
Solution step $keep constant, differentiate rest$

Formula $Product$
Numeric example $(x\cdot e^x)'=e^x(1+x)$
Solution step $(first')\timessecond + first\times(second')$

Formula $Quotient$
Numeric example $x/(x^2+1)$
Solution step $(top'\timesbottom-top\timesbottom')/bottom²$

Formula	Numeric example	Solution step
Constant	(5)'=0, (-3)'=0	derivative is 0
$x^n$	$(x^3)'=3x^2$ , at $x=2$ : 12	bring down $n$ , exponent $n-1$
$e^x$	at $x=0$ : 1; at $x=1$ : $e$	unchanged by differentiation
$a^x$	$(2^x)'=2^x \ln 2$	multiply by $\ln a$
$\ln x$	at $x=5$ : $1/5$	derivative is $1/x$
sin	at $x=0$ : 1	sin → cos
cos	at $x=0$ : 0	cos → $-\sin$
Sum	$(x^2+x)'=2x+1$	differentiate each term and add
Constant multiple	$(5x^2)'=10x$	keep constant, differentiate rest
Product	$(x\cdot e^x)'=e^x(1+x)$	(first′)×second + first×(second′)
Quotient	$x/(x^2+1)$ → at $x=1$ : 0	(top′×bottom−top×bottom′)/bottom²

In everyday terms, the derivative is a compass for finding the optimum. When you want to go from a mountain top to the lowest valley, you follow the slope under your feet downward. The derivative computes that slope in exact numbers. Where the slope is zero you are at a peak (maximum) or a valley floor (minimum), so the derivative is essential for finding minima and maxima.

Deep learning models must minimize the error (loss) between the correct answer and the prediction. The gradient is what tells us how to change each of the model's many weights (

w

) so that the error decreases. By differentiating, we learn whether slightly increasing a given weight would increase or decrease the error, and we update weights in the direction that reduces the error fastest.

Backpropagation applies this derivative idea in reverse for efficient learning. It works backward from the output, step by step, computing how much each step contributed to the final error. To differentiate through stacked functions we use differentiation of composite functions (chain rule); the basic derivative formulas in this chapter are exactly what make that work.

In general, the derivative is used for sensitivity analysis: how much does the whole result swing when one variable is nudged? In economics we ask how demand changes when price changes a little; in physics we measure how position changes over time (velocity). So anywhere we quantify how a small change in a cause affects the outcome, we use the derivative.

In AI training, every parameter update depends on derivative values. When we use libraries like PyTorch or TensorFlow, the loss is differentiated with respect to each weight in an instant. Moving the weights in the opposite direction of that derivative is gradient descent. The formulas in this chapter are the first key to understanding how AI gets smarter through computation.

To find the derivative, identify which rules apply (power, exponential, log, trig, product, quotient, chain), then apply them and simplify.

Example problems and solutions are in the table below.

Problem $f(x)=x^3-2x$
Solution $f^{\prime}(x)=3x^2-2$

Problem $g(x)=e^x \ln x$
Solution $e^x \ln x + e^x \cdot \frac{1}{x}$

Problem $h(x)=\frac{\sin x}{x}$
Solution $\frac{\cos x \cdot x - \sin x}{x^2}$

Problem	Solution
Ex 1. $f(x)=x^3-2x$	Power and sum: $f^{\prime}(x)=3x^2-2$
Ex 2. $g(x)=e^x \ln x$	Product rule: $e^x \ln x + e^x \cdot \frac{1}{x}$
Ex 3. $h(x)=\frac{\sin x}{x}$	Quotient rule: $\frac{\cos x \cdot x - \sin x}{x^2}$

Problem types and how to solve

Type $Power$
Formula $f(x)=x^n$
How to get $f^{\prime}(a)$ at $x=a$ $f^{\prime}(a) = n \cdot a^{n-1}$

Type $Linear$
Formula $f(x)=mx+b$
How to get $f^{\prime}(a)$ at $x=a$ $f^{\prime}(a) = m$

Type $Quadratic$
Formula $f(x)=a_2 x^2 + a_1 x + a_0$
How to get $f^{\prime}(a)$ at $x=a$ $f^{\prime}(a) = 2 a_2 \cdot a + a_1$

Type $Constant \times power$
Formula $f(x)=c \cdot x^n$
How to get $f^{\prime}(a)$ at $x=a$ $f^{\prime}(a) = c \cdot n \cdot a^{n-1}$

Type	Formula	How to get $f^{\prime}(a)$ at $x=a$
Power	$f(x)=x^n$	$f^{\prime}(a) = n \cdot a^{n-1}$ . Bring down the exponent, subtract 1, then plug in $a$ .
Linear	$f(x)=mx+b$	$f^{\prime}(a) = m$ . The slope is the derivative, so the answer is $m$ regardless of $a$ .
Quadratic	$f(x)=a_2 x^2 + a_1 x + a_0$	$f^{\prime}(a) = 2 a_2 \cdot a + a_1$ . Twice the $x^2$ coefficient times $a$ , plus the linear coefficient.
Constant × power	$f(x)=c \cdot x^n$	$f^{\prime}(a) = c \cdot n \cdot a^{n-1}$ . Keep $c$ , multiply by the derivative of $x^n$ .

Example (power)

For

f(x)=x^3

, find the derivative at

x=2

, i.e.

f^{\prime} (2)

Solution

f^{\prime}(x)=3x^2

, so

f^{\prime} (2) =3 \times 2^2 = 12

. → Answer 12

Example (linear)

For

f(x)=3x+1

, find

f^{\prime} (5)

Solution

The derivative of a linear function is the slope, so

f^{\prime}(x)=3

. Hence

f^{\prime} (5) =3

for any

a

. → Answer 3

Example (quadratic)

For

f(x)=x^2+2x+1

, find

f^{\prime} (3)

Solution

f^{\prime}(x)=2x+2

, so

f^{\prime} (3) =2 \times 3 + 2 = 8

. → Answer 8

Example (constant × power)

For

f(x)=2x^4

, find

f^{\prime} (1)

Solution

f^{\prime}(x)=2 \times 4 \times x^3 = 8x^3

, so

f^{\prime} (1) =8 \times 1 = 8

. → Answer 8

What are differentiation and the derivative?

y

responds when the input

x

changes by a tiny amount.

f^{\prime}(a) = \lim_{h \to 0} \frac{f(a+h)-f(a)}{h}

(the denominator

h

is the distance between two points, the numerator is the change in

f

). The value obtained through this process is called the derivative coefficient and is written as

f^{\prime}(a)

. A large value means a steep slope; zero means flat.

The derivative function is the new function that assigns to each

x

the slope at that point. So instead of computing the slope at every point by hand, you plug

x

into the derivative formula—it is a "slope generator." The process of finding it is called differentiation.

Item $Below$
Description $Common differentiation formulas .$

Item $'$
Description $Means the derivative.$

Item	Description
Below	Common differentiation formulas.
$'$	Means the derivative.

Formula $Constant$
Description $derivative is 0$

Formula $x^n$
Description $n$

Formula $e^x$
Description $e^x$

Formula $a^x$
Description $a^x \ln a$

Formula $\ln x$
Description $1/x$

Formula $Log (base a)$
Description $1/(x \ln a)$

Formula $sin$
Description $derivative is cos$

Formula $cos$
Description $-\sin$

Formula $tan$
Description $1/\cos^2$

Formula $Sum/difference$
Description $differentiate each part and add or subtract$

Formula $Constant multiple$
Description $keep constant, differentiate the rest$

Formula $Product rule$
Description $(first')\timessecond + first\times(second')$

Formula $Quotient rule$
Description $(top'\timesbottom - top\timesbottom') / bottom²$

Formula $Intuition (product)$
Description $first derivative \times second + first \times second derivative. In the limit the two changes add up, so you get these two terms.$

Formula $Intuition (quotient)$
Description $Write the quotient as top \times (1/bottom) and use the product rule. The derivative of (1/bottom) gives the usual formula.$

Formula	Description
Constant	derivative is 0
Power $x^n$	bring down $n$ , exponent $n-1$
Exponential $e^x$	derivative is still $e^x$
Exponential $a^x$	$a^x \ln a$
Natural log $\ln x$	$1/x$
Log (base a)	$1/(x \ln a)$
sin	derivative is cos
cos	derivative is $-\sin$
tan	derivative is $1/\cos^2$
Sum/difference	differentiate each part and add or subtract
Constant multiple	keep constant, differentiate the rest
Product rule	(first′)×second + first×(second′)
Quotient rule	(top′×bottom − top×bottom′) / bottom²
Intuition (product)	first derivative × second + first × second derivative. In the limit the two changes add up, so you get these two terms.
Intuition (quotient)	Write the quotient as top × (1/bottom) and use the product rule. The derivative of (1/bottom) gives the usual formula.

Formula $Constant$
Numeric example $(5)'=0, (-3)'=0$
Solution step $derivative is 0$

Formula $x^n$
Numeric example $(x^3)'=3x^2$
Solution step $n$

Formula $e^x$
Numeric example $x=0$
Solution step $unchanged by differentiation$

Formula $a^x$
Numeric example $(2^x)'=2^x \ln 2$
Solution step $\ln a$

Formula $\ln x$
Numeric example $x=5$
Solution step $1/x$

Formula $sin$
Numeric example $x=0$
Solution step $sin \to cos$

Formula $cos$
Numeric example $x=0$
Solution step $-\sin$

Formula $Sum$
Numeric example $(x^2+x)'=2x+1$
Solution step $differentiate each term and add$

Formula $Constant multiple$
Numeric example $(5x^2)'=10x$
Solution step $keep constant, differentiate rest$

Formula $Product$
Numeric example $(x\cdot e^x)'=e^x(1+x)$
Solution step $(first')\timessecond + first\times(second')$

Formula $Quotient$
Numeric example $x/(x^2+1)$
Solution step $(top'\timesbottom-top\timesbottom')/bottom²$

Formula	Numeric example	Solution step
Constant	(5)'=0, (-3)'=0	derivative is 0
$x^n$	$(x^3)'=3x^2$ , at $x=2$ : 12	bring down $n$ , exponent $n-1$
$e^x$	at $x=0$ : 1; at $x=1$ : $e$	unchanged by differentiation
$a^x$	$(2^x)'=2^x \ln 2$	multiply by $\ln a$
$\ln x$	at $x=5$ : $1/5$	derivative is $1/x$
sin	at $x=0$ : 1	sin → cos
cos	at $x=0$ : 0	cos → $-\sin$
Sum	$(x^2+x)'=2x+1$	differentiate each term and add
Constant multiple	$(5x^2)'=10x$	keep constant, differentiate rest
Product	$(x\cdot e^x)'=e^x(1+x)$	(first′)×second + first×(second′)
Quotient	$x/(x^2+1)$ → at $x=1$ : 0	(top′×bottom−top×bottom′)/bottom²

Deep learning models must minimize the error (loss) between the correct answer and the prediction. The gradient is what tells us how to change each of the model's many weights (

w

To find the derivative, identify which rules apply (power, exponential, log, trig, product, quotient, chain), then apply them and simplify.

Example problems and solutions are in the table below.

Problem $f(x)=x^3-2x$
Solution $f^{\prime}(x)=3x^2-2$

Problem $g(x)=e^x \ln x$
Solution $e^x \ln x + e^x \cdot \frac{1}{x}$

Problem $h(x)=\frac{\sin x}{x}$
Solution $\frac{\cos x \cdot x - \sin x}{x^2}$

Problem	Solution
Ex 1. $f(x)=x^3-2x$	Power and sum: $f^{\prime}(x)=3x^2-2$
Ex 2. $g(x)=e^x \ln x$	Product rule: $e^x \ln x + e^x \cdot \frac{1}{x}$
Ex 3. $h(x)=\frac{\sin x}{x}$	Quotient rule: $\frac{\cos x \cdot x - \sin x}{x^2}$

Problem types and how to solve

Type $Power$
Formula $f(x)=x^n$
How to get $f^{\prime}(a)$ at $x=a$ $f^{\prime}(a) = n \cdot a^{n-1}$

Type $Linear$
Formula $f(x)=mx+b$
How to get $f^{\prime}(a)$ at $x=a$ $f^{\prime}(a) = m$

Type $Quadratic$
Formula $f(x)=a_2 x^2 + a_1 x + a_0$
How to get $f^{\prime}(a)$ at $x=a$ $f^{\prime}(a) = 2 a_2 \cdot a + a_1$

Type $Constant \times power$
Formula $f(x)=c \cdot x^n$
How to get $f^{\prime}(a)$ at $x=a$ $f^{\prime}(a) = c \cdot n \cdot a^{n-1}$

Type	Formula	How to get $f^{\prime}(a)$ at $x=a$
Power	$f(x)=x^n$	$f^{\prime}(a) = n \cdot a^{n-1}$ . Bring down the exponent, subtract 1, then plug in $a$ .
Linear	$f(x)=mx+b$	$f^{\prime}(a) = m$ . The slope is the derivative, so the answer is $m$ regardless of $a$ .
Quadratic	$f(x)=a_2 x^2 + a_1 x + a_0$	$f^{\prime}(a) = 2 a_2 \cdot a + a_1$ . Twice the $x^2$ coefficient times $a$ , plus the linear coefficient.
Constant × power	$f(x)=c \cdot x^n$	$f^{\prime}(a) = c \cdot n \cdot a^{n-1}$ . Keep $c$ , multiply by the derivative of $x^n$ .

Example (power)

For

f(x)=x^3

, find the derivative at

x=2

, i.e.

f^{\prime} (2)

Solution

f^{\prime}(x)=3x^2

, so

f^{\prime} (2) =3 \times 2^2 = 12

. → Answer 12

Example (linear)

For

f(x)=3x+1

, find

f^{\prime} (5)

Solution

The derivative of a linear function is the slope, so

f^{\prime}(x)=3

. Hence

f^{\prime} (5) =3

for any

a

. → Answer 3

Example (quadratic)

For

f(x)=x^2+2x+1

, find

f^{\prime} (3)

Solution

f^{\prime}(x)=2x+2

, so

f^{\prime} (3) =2 \times 3 + 2 = 8

. → Answer 8

Example (constant × power)

For

f(x)=2x^4

, find

f^{\prime} (1)

Solution

f^{\prime}(x)=2 \times 4 \times x^3 = 8x^3

, so

f^{\prime} (1) =8 \times 1 = 8

. → Answer 8