Chapter 04

Limits and ε-δ: Defining "Getting Arbitrarily Close"

A limit is the mathematical tool that lets us predict the state at a goal point without actually reaching it. Measuring the instantaneous velocity of a moving object, or the process of AI learning step by step toward the answer, all rest on this concept of limit.

Math diagram by chapter

Select a chapter to see its diagram below. View the flow of basic math at a glance.

See the limit and ε-δ in the graph below.

Summary: Pick error ε (green); then there is distance δ (blue) so that whenever x is in the blue band, f(x) is in the green band. That is the idea of ε-δ.

How to read

Orange point = (x, f(x)) on the curve approaching (2, 4)
Green band = L±ε (allowed error for f(x))
Blue band = a±δ (if x is here, f(x) is in the green band)

What is a limit?

A limit describes where the function

f(x)

is heading as

x

gets arbitrarily close to some value

a

. We write

\lim_{x \to a} f(x) = L

. The key point is that $x$ never actually equals $a$ —we only look at the trend in the immediate neighborhood of

a

Intuitive example: Imagine navigation where the distance to your destination shrinks from

100m

10m

1m

0.1m

. Even if the car never "exactly" stops at the destination, we know where it's going. In math we say that as

x

approaches

a

, the distance between

f(x)

and

L

converges to 0. When a limit does not exist: For

\frac{1}{x}

x \to 0

, the left limit (

-\infty

) and right limit (

+\infty

) differ, so no limit is defined. One-sided limits like

\frac{1}{x} \to 0

x \to \infty

are well defined.

The epsilon ( $\varepsilon$ )–delta ( $\delta$ ) definition is the precise mathematical agreement that defines a limit. Think of it as a bet:

(1) Someone challenges you: "Can you keep the error within

\varepsilon

(whether 0.1 or 0.0001)?"

(2) You prove: "If I keep

x

within some distance

\delta

a

, the result is guaranteed to fall within that error

\varepsilon

." So the core of epsilon-delta is: no matter how small an error ( $\varepsilon$ ) you demand, I can always find a range of $x$ ( $\delta$ ) that satisfies it.

Because it is the only way to define the instantaneous rate of change (derivative). "Instantaneous" means a time interval of zero—and we cannot divide by zero. By sending the interval arbitrarily close to zero (a limit), we can compute speed even from a single frozen frame.

It guarantees continuity and differentiability. If the AI's graph is broken or has a sharp spike, learning is impossible. The limit must exist and equal the function value for "continuity," and only then can we use differentiation to find the direction that reduces error. In other words, limits are the map that keeps AI from getting lost.

It is the theoretical foundation of gradient descent. When we say AI updates parameters by "a tiny amount," that "tiny amount" is justified mathematically by the limit. Tuning the learning rate and searching for the optimum is like implementing the idea of a limit in computation. Numerical differentiation also uses the limit principle when we take

h

very small (e.g.

0.0001

) to approximate the derivative.

When looking at a limit, focus on where $x$ is going (e.g.

x \to 0

x \to \infty

) and what value $f(x)$ gets close to. Drawing the graph helps: you'll see

f(x)

bunch up near

L

around

a

Epsilon-delta proofs work by choosing $\varepsilon$ first, then finding a $\delta$ that works. In practice, understanding that "close enough gives as small an error as we want" is enough to move on to derivatives and continuity.

Example problems and solutions are in the table below.

Problem $\lim_{x \to 2} (x^2 + 1)$
Solution $x = 2$

Problem $\lim_{x \to \infty} \frac{1}{x}$
Solution $x$

Problem $\lim_{x \to 3} (2x - 1)$
Solution $x = 3$

Problem	Solution
Ex 1. $\lim_{x \to 2} (x^2 + 1)$	Solution: Substitute $x = 2$ : $2^2 + 1 = 5$ . Answer 5.
Ex 2. $\lim_{x \to \infty} \frac{1}{x}$	Solution: As $x$ grows, $\frac{1}{x} \to 0$ . Answer 0.
Ex 3. $\lim_{x \to 3} (2x - 1)$	Solution: Substitute $x = 3$ : $2 \times 3 - 1 = 5$ . Answer 5.

Problem types and how to solve

Type $Polynomial limit$
Description $\lim_{x \to a} f(x)$
How to get the answer $x=a$

Type $Constant$
Description $\lim_{x \to a} c$
How to get the answer $c$

Type $Linear$
Description $\lim_{x \to a} (mx+b)$
How to get the answer $a$

Type $x\to\infty$
Description $\lim_{x \to \infty} f(x)$
How to get the answer $f(x)$

Type $ε-δ concept$
Description $Number asked in the definition$
How to get the answer $Enter the number that matches (e.g. 1=distance, 2=error).$

Type	Description	How to get the answer
Polynomial limit	$\lim_{x \to a} f(x)$ , $f$ polynomial	Polynomials are continuous; substitute $x=a$ . Limit = function value.
Constant	$\lim_{x \to a} c$	Constant is $c$ regardless of $x$ . Answer $c$ .
Linear	$\lim_{x \to a} (mx+b)$	Substitute $a$ → $ma+b$ .
x→∞	$\lim_{x \to \infty} f(x)$	See what $f(x)$ approaches as $x$ grows. $1/x$ , $1/x^2$ → 0. Use leading term.
ε-δ concept	Number asked in the definition	Enter the number that matches (e.g. 1=distance, 2=error).

Example (polynomial limit)

Find

\lim_{x \to 2} (x^2 + 1)

Solution

Polynomial is continuous; substitute

x=2

2^2+1=5

. → Answer 5

Example (linear)

Find

\lim_{x \to 3} (2x - 1)

Solution

Substitute

x=3

→

2\times 3-1=5

. → Answer 5

Example (constant)

Find

\lim_{x \to 0} 7

Solution

Constant function is 7 for all

x

. → Answer 7

Example (x→∞)

Find

\lim_{x \to \infty} \frac{1}{x}

Solution

x

grows,

1/x

approaches 0. → Answer 0

What is a limit?

A limit describes where the function

f(x)

is heading as

x

gets arbitrarily close to some value

a

. We write

\lim_{x \to a} f(x) = L

. The key point is that $x$ never actually equals $a$ —we only look at the trend in the immediate neighborhood of

a

Intuitive example: Imagine navigation where the distance to your destination shrinks from

100m

10m

1m

0.1m

. Even if the car never "exactly" stops at the destination, we know where it's going. In math we say that as

x

approaches

a

, the distance between

f(x)

and

L

converges to 0. When a limit does not exist: For

\frac{1}{x}

x \to 0

, the left limit (

-\infty

) and right limit (

+\infty

) differ, so no limit is defined. One-sided limits like

\frac{1}{x} \to 0

x \to \infty

are well defined.

The epsilon ( $\varepsilon$ )–delta ( $\delta$ ) definition is the precise mathematical agreement that defines a limit. Think of it as a bet:

(1) Someone challenges you: "Can you keep the error within

\varepsilon

(whether 0.1 or 0.0001)?"

(2) You prove: "If I keep

x

within some distance

\delta

a

, the result is guaranteed to fall within that error

\varepsilon

." So the core of epsilon-delta is: no matter how small an error ( $\varepsilon$ ) you demand, I can always find a range of $x$ ( $\delta$ ) that satisfies it.

h

very small (e.g.

0.0001

) to approximate the derivative.

When looking at a limit, focus on where $x$ is going (e.g.

x \to 0

x \to \infty

) and what value $f(x)$ gets close to. Drawing the graph helps: you'll see

f(x)

bunch up near

L

around

a

Example problems and solutions are in the table below.

Problem $\lim_{x \to 2} (x^2 + 1)$
Solution $x = 2$

Problem $\lim_{x \to \infty} \frac{1}{x}$
Solution $x$

Problem $\lim_{x \to 3} (2x - 1)$
Solution $x = 3$

Problem	Solution
Ex 1. $\lim_{x \to 2} (x^2 + 1)$	Solution: Substitute $x = 2$ : $2^2 + 1 = 5$ . Answer 5.
Ex 2. $\lim_{x \to \infty} \frac{1}{x}$	Solution: As $x$ grows, $\frac{1}{x} \to 0$ . Answer 0.
Ex 3. $\lim_{x \to 3} (2x - 1)$	Solution: Substitute $x = 3$ : $2 \times 3 - 1 = 5$ . Answer 5.

Problem types and how to solve

Type $Polynomial limit$
Description $\lim_{x \to a} f(x)$
How to get the answer $x=a$

Type $Constant$
Description $\lim_{x \to a} c$
How to get the answer $c$

Type $Linear$
Description $\lim_{x \to a} (mx+b)$
How to get the answer $a$

Type $x\to\infty$
Description $\lim_{x \to \infty} f(x)$
How to get the answer $f(x)$

Type $ε-δ concept$
Description $Number asked in the definition$
How to get the answer $Enter the number that matches (e.g. 1=distance, 2=error).$

Type	Description	How to get the answer
Polynomial limit	$\lim_{x \to a} f(x)$ , $f$ polynomial	Polynomials are continuous; substitute $x=a$ . Limit = function value.
Constant	$\lim_{x \to a} c$	Constant is $c$ regardless of $x$ . Answer $c$ .
Linear	$\lim_{x \to a} (mx+b)$	Substitute $a$ → $ma+b$ .
x→∞	$\lim_{x \to \infty} f(x)$	See what $f(x)$ approaches as $x$ grows. $1/x$ , $1/x^2$ → 0. Use leading term.
ε-δ concept	Number asked in the definition	Enter the number that matches (e.g. 1=distance, 2=error).

Example (polynomial limit)

Find

\lim_{x \to 2} (x^2 + 1)

Solution

Polynomial is continuous; substitute

x=2

2^2+1=5

. → Answer 5

Example (linear)

Find

\lim_{x \to 3} (2x - 1)

Solution

Substitute

x=3

→

2\times 3-1=5

. → Answer 5

Example (constant)

Find

\lim_{x \to 0} 7

Solution

Constant function is 7 for all

x

. → Answer 7

Example (x→∞)

Find

\lim_{x \to \infty} \frac{1}{x}

Solution

x

grows,

1/x

approaches 0. → Answer 0