Chapter 04

Limit and Epsilon-Delta (ε-δ)

A limit describes what happens when we get "arbitrarily close" to some value. Epsilon-delta is the precise way to define that idea and is the basis for derivatives and deep learning.

Math diagram by chapter

Select a chapter to see its diagram below. View the flow of basic math at a glance.

See the limit and ε-δ in the graph below.

Summary: Pick error ε (green); then there is distance δ (blue) so that whenever x is in the blue band, f(x) is in the green band. That is the idea of ε-δ.

How to read

Orange point = (x, f(x)) on the curve approaching (2, 4)
Green band = L±ε (allowed error for f(x))
Blue band = a±δ (if x is here, f(x) is in the green band)

What is a limit?

A limit means: "as

x

gets closer and closer to some number

a

, the value

f(x)

gets closer and closer to some number

L

." We write

\lim_{x \to a} f(x) = L

. We don't need

x

to actually reach

a

; we only care that near

a

f(x)

stays close to

L

In plain language: If you make

x

close enough to

a

, you can make

f(x)

as close to

L

as you want. Example: as

x

approaches

0

\frac{\sin x}{x}

approaches

1

(

\lim_{x \to 0} \frac{\sin x}{x} = 1

The epsilon ( $\varepsilon$ )–delta ( $\delta$ ) definition turns this "gets closer" into exact words: "No matter how small an error (

\varepsilon

) you choose, you can find a distance (

\delta

) so that if

x

is within

\delta

a

, then

f(x)

is within

\varepsilon

L

." It can look heavy at first, but it just means: if we get close enough, the result becomes as accurate as we want.

Derivatives measure the instantaneous rate of change at a point—how much the value changes when we move a tiny bit. That "tiny bit" is exactly a limit. So limits are the basis for understanding derivatives and slopes.

In deep learning, gradient descent updates parameters by a small step; "how much does the loss change when we change the parameter a little?" is the derivative (gradient), and limits sit behind that. A loose grasp of limits makes it easier to see why we use derivatives.

In AI, limits appear inside the math. A small learning rate makes each step behave like a limit of smaller steps, and the gradient used in backprop is the limit of "change in output over change in input" as the change goes to zero. You don't need to memorize epsilon-delta; feeling that we're dealing with very small changes is enough to follow the next chapters.

When looking at a limit, focus on where $x$ is going (e.g.

x \to 0

x \to \infty

) and what value $f(x)$ gets close to. Drawing the graph helps: you'll see

f(x)

bunch up near

L

around

a

Epsilon-delta proofs work by choosing $\varepsilon$ first, then finding a $\delta$ that works. In practice, understanding that "close enough gives as small an error as we want" is enough to move on to derivatives and continuity.

Example problems and solutions are in the table below.

Problem	Solution
Ex 1. $\lim_{x \to 2} (x^2 + 1)$	Solution: Substitute $x = 2$ : $2^2 + 1 = 5$ . Answer 5.
Ex 2. $\lim_{x \to \infty} \frac{1}{x}$	Solution: As $x$ grows, $\frac{1}{x} \to 0$ . Answer 0.
Ex 3. $\lim_{x \to 3} (2x - 1)$	Solution: Substitute $x = 3$ : $2 \times 3 - 1 = 5$ . Answer 5.