Everyone's AI
Machine learningAI Papers
Loading...

Learn

🏅My achievements

Chapter 02

Exponents and Exponential Functions: The Math of Growth and Activation

Exponentiation is repeated multiplication of the same base; an exponential function fixes the base and uses the exponent as the variable. Used in activation and loss design in deep learning.

Math diagram by chapter

Select a chapter to see its diagram below. View the flow of basic math at a glance.

0123024681012xy(x=0.0, y=1.0)

Example: 20=12^0=120=1, 21=22^1=221=2, 22=42^2=422=4, 23=82^3=823=8

What are exponent and exponential function?

An exponent is how many times a number (the base) is multiplied by itself. Like the fact that folding a piece of paper 42 times would reach the moon, repeated multiplication (not addition) makes values grow explosively (exponential growth).
An exponential function puts that repeated power in a variable: y=axy = a^xy=ax. In polynomials the variable is in the base (x2x^2x2); in exponentials the variable is in the exponent. That means growth proportional to current size. If a>1a>1a>1, the value shoots up as xxx increases (exponential growth); if 0<a<10<a<10<a<1, it quickly approaches 0 as xxx increases (exponential decay). Radioactive decay and half-life in interest calculations are examples of exponential decay.
The natural constant eee (about 2.718…) is the most important base in math and AI. The function y=exy=e^xy=ex is the only one that stays unchanged when differentiated. That property dramatically reduces the amount of calculus in deep learning.
In AI, exponentials are the building blocks of activation functions. Linear computation (ax+bax+bax+b) alone cannot solve complex problems; exponentials are used to bend the signal (nonlinearity) or smoothly squeeze values between 0 and 1.
Because the output is always positive. The graph of y=axy=a^xy=ax always lies above the xxx-axis: for any real xxx, y>0y>0y>0. AI cannot say "the probability is -50%," so exponentials are essential when we need outputs to be positive (e.g. probabilities or positive scores).
They amplify small differences. Inputs 1 and 2 differ by 1, but 101=1010^1=10101=10 and 102=10010^2=100102=100 differ by 90. AI uses this to sharply separate similar data and classify with confidence.
Efficient differentiation: Backprop is a long chain of derivatives. The exponential exe^xex keeps the same shape when differentiated (or stays in a simple form), which is crucial for fast, stable training.
Used in the softmax function. When AI chooses one out of 1000 images, it applies exe^xex to each score. Slightly higher scores get much larger values and lower ones shrink toward 0, so the model can say "this is the answer with 99% confidence."
The sigmoid function y=11+e−xy = \frac{1}{1+e^{-x}}y=1+e−x1​ squeezes the input into (0, 1). The output never exceeds 1 or goes below 0, so the neuron acts like an on/off switch.
  • Expression202^020
  • Value1
  • Expression212^121
  • Value2
  • Expression222^222
  • Value4
  • Expression232^323
  • Value8
  • Expression242^424
  • Value16
  • Expression323^232
  • Value9
  • Expression333^333
  • Value27
ExpressionValue
202^0201
212^1212
222^2224
232^3238
242^42416
323^2329
333^33327
In the visual below, y=2xy = 2^xy=2x gives 111 for x=0x=0x=0, 222 for x=1x=1x=1, 444 for x=2x=2x=2, 888 for x=3x=3x=3. Use it to see how base and exponent relate.
Problem types and how to solve them
  • TypeFind value
  • Descriptionax=?a^x = ?ax=?
  • How to get the answerMultiply base aaa by itself xxx times. E.g. 23=82^3 = 823=8.
  • TypeFind exponent
  • Descriptiona?=valuea^? = \text{value}a?=value
  • How to get the answer"How many times do we multiply aaa to get this value?" That count is the answer. E.g. 2?=8⇒32^? = 8 \Rightarrow 32?=8⇒3.
  • TypeCompare
  • DescriptionWhich is larger: 1) ama^{m}am, 2) bnb^{n}bn?
  • How to get the answer
    Compute each, then compare. If
    (1) is larger enter 1, if
    (2) enter 2.
  • TypeProduct, same base
  • Descriptionap×aq=a?a^p \times a^q = a^?ap×aq=a?
  • How to get the answerAdd exponents: ?=p+q? = p + q?=p+q. (Rule: ap⋅aq=ap+qa^p \cdot a^q = a^{p+q}ap⋅aq=ap+q)
  • TypeQuotient, same base
  • Descriptionap÷aq=a?a^p \div a^q = a^?ap÷aq=a? (p≥qp \ge qp≥q)
  • How to get the answerSubtract exponents: ?=p−q? = p - q?=p−q. (Rule: ap/aq=ap−qa^p / a^q = a^{p-q}ap/aq=ap−q)
  • TypePower of power
  • Description(ap)q=?(a^p)^q = ?(ap)q=?
  • How to get the answerMultiply exponents: ?=ap×q? = a^{p \times q}?=ap×q. (Rule: (ap)q=apq(a^p)^q = a^{pq}(ap)q=apq)
TypeDescriptionHow to get the answer
Find valueax=?a^x = ?ax=?Multiply base aaa by itself xxx times. E.g. 23=82^3 = 823=8.
Find exponenta?=valuea^? = \text{value}a?=value"How many times do we multiply aaa to get this value?" That count is the answer. E.g. 2?=8⇒32^? = 8 \Rightarrow 32?=8⇒3.
CompareWhich is larger: 1) ama^{m}am, 2) bnb^{n}bn?
Compute each, then compare. If
(1) is larger enter 1, if
(2) enter 2.
Product, same baseap×aq=a?a^p \times a^q = a^?ap×aq=a?Add exponents: ?=p+q? = p + q?=p+q. (Rule: ap⋅aq=ap+qa^p \cdot a^q = a^{p+q}ap⋅aq=ap+q)
Quotient, same baseap÷aq=a?a^p \div a^q = a^?ap÷aq=a? (p≥qp \ge qp≥q)Subtract exponents: ?=p−q? = p - q?=p−q. (Rule: ap/aq=ap−qa^p / a^q = a^{p-q}ap/aq=ap−q)
Power of power(ap)q=?(a^p)^q = ?(ap)q=?Multiply exponents: ?=ap×q? = a^{p \times q}?=ap×q. (Rule: (ap)q=apq(a^p)^q = a^{pq}(ap)q=apq)