Chapter 03

Logarithm: From Multiplication to Addition, the Language of Loss Design

A logarithm answers 'how many times we multiply the base to get this number?' It is the inverse of exponentiation and is used with exponentials in loss and probability in deep learning.

Math diagram by chapter

Select a chapter to see its diagram below. View the flow of basic math at a glance.

Logarithm is the inverse of exponent. $y = \log_2 x$ means $2^y = x$ . Below are the graphs of $y = \log_2 x$ and its inverse $y = 2^x$ .

Example: $\log_2 1 = 0$ , $\log_2 2 = 1$ , $\log_2 4 = 2$ , $\log_2 8 = 3$ (when $2^y = x$ , $y$ is $\log_2 x$ )

Purple: $y=\log_2 x$ , Teal: $y=2^x$

What is the logarithm?

The logarithm is like "running exponentiation backward." In

2^3 = 8

, when you see the result 8 and ask "how many times did we multiply 2 to get 8?", that count

(3) is the logarithm:

\log_2 8 = 3

. Here 2 is the base and 8 is the argument.

Think of it as counting digits.

100 = 10^2

\log_{10} 100 = 2

;

1000 = 10^3

\log_{10} 1000 = 3

. When the number grows 10×, the log value only goes up by 1. So log acts as a filter that turns explosively large numbers into much gentler ones. Basic properties:

\log_a 1 = 0

(base to the 0th power is 1),

\log_a a = 1

(base to the 1st power is itself).

The magic of log is that it turns multiplication into addition:

\log_a(b \times c) = \log_a b + \log_a c

. For computers, multiplication is costlier than addition and can overflow or underflow; taking the log turns that multiplication into a safer, simpler addition.

The argument condition ( $x>0$ ) matters: log of 0 or a negative number is undefined. So in AI code we often add a tiny constant (

\epsilon

, epsilon) to avoid

\log(0)

errors. The natural log (

\ln

, base

e

) keeps differentiation tidy and is the standard in deep learning.

Avoiding underflow is essential. If AI multiplies probability

0.1

a hundred times (

0.1^{100}

), the computer may treat it as zero. Taking the log gives

\log(0.1^{100}) = 100 \times \log(0.1) = -100

—a meaningful number the computer can still handle.

It is the ruler for information (entropy). The rarer an event, the larger (in absolute value) its log. A rare event (e.g. "sun rises in the west") carries high information; an obvious one ("morning comes") carries almost none. AI uses this log-based measure to see how much surprising information was learned.

It penalizes mistakes harshly. For

y=\ln x

with

0<x<1

, as

x

approaches 0,

y

plunges toward

-\infty

. If the model predicts probability 0.9 for the correct answer, loss is small; if it predicts 0.01, the log gives a huge penalty. So the model is pushed to fix wrong answers clearly.

The cross-entropy loss is a prime example. In classification we minimize

-\log p

for the correct class. That is the mathematical way to say "make the probability of the correct answer as close to 1 as possible (and its log close to 0)."

Maximum likelihood estimation (MLE) uses it: "maximize the probability of observing this data" means maximizing a product of probabilities. Taking the log turns that into maximizing a sum, which is easier to differentiate and numerically more stable.

Example $\log_2 8$
Value $2^3=8$

Example $\log_2 4$
Value $2$

Example $\log_3 9$
Value $2$

Example	Value
$\log_2 8$	3 ( $2^3=8$ )
$\log_2 4$	2
$\log_3 9$	2

The log is an integer only when the argument is a power of the base.

Operations frequently used with logarithms (often in AI loss and probability):

Operation $Log sum$
Formula $\log_a b + \log_a c = \log_a(b \cdot c)$
Note $product \to sum$

Operation $Log difference$
Formula $\log_a b - \log_a c = \log_a(b/c)$
Note $quotient \to difference$

Operation $Power$
Formula $\log_a(b^n) = n \cdot \log_a b$
Note $exponent out front$

Operation	Formula	Note
Log sum	$\log_a b + \log_a c = \log_a(b \cdot c)$	product → sum
Log difference	$\log_a b - \log_a c = \log_a(b/c)$	quotient → difference
Power	$\log_a(b^n) = n \cdot \log_a b$	exponent out front

Example $Log sum$
Calculation $\log_2 2 + \log_2 4 = 1 + 2 = 3$

Example $Log difference$
Calculation $\log_2 8 - \log_2 2 = 3 - 1 = 2$

Example	Calculation
Log sum	$\log_2 2 + \log_2 4 = 1 + 2 = 3$
Log difference	$\log_2 8 - \log_2 2 = 3 - 1 = 2$

In the problems below, find log values, arguments, log sums, or log differences.