Everyone's AI
Machine learningAI Papers
Loading...

Learn

🏅My achievements

Ch.03

Matrices and Data: Structural Representation of Many Vectors

Math diagram by chapter

Select a chapter to see its diagram below. View the flow of intermediate math at a glance.

Scalar11Vector51732-299Matrix11058299-834783D Tensorstack of 2D slices11058299-8347811058299-8347811058299-834784D Tensorstack of 3D blocks5D Tensorstack of 4D blocks
Purple columns highlight in turn. Each column is a vector of the same length; placing three columns next to each other forms one matrix. The right panel shows m×nm\times nm×n and which column is active. Reading by rows gives one line per sample (a common data convention).
A matrix is a rectangular grid of numbers—one sheet. In machine learning, one row is often one sample (one person, one image) and one column is one feature. This chapter connects vectors (Ch.01) and dot products (Ch.02) to how they appear many times at once inside a matrix, and sets up matrix multiplication and linear layers (Ch.04).

Matrices and data batches: putting many vectors on one sheet

If a vector is numbers in a line, a matrix stacks several such lines into a rectangle. Size m×nm\times nm×n means mmm rows meet nnn columns. Notation varies (sometimes rows are samples, sometimes columns)—always check the shape first.
Think of a matrix as one spreadsheet: each cell is a number; a whole column can be one feature vector; a whole row can be one record. The same table changes meaning depending on which direction you read.
Core facts:
1. Shape: m×nm\times nm×n means mmm rows and nnn columns of real entries.
2. Entries: the value at row iii, column jjj is aija_{ij}aij​.
3. Transpose: ATA^{\mathsf T}AT is n×mn\times mn×m with (AT)ji=aij(A^{\mathsf T})_{ji}=a_{ij}(AT)ji​=aij​.
4. Columns as vectors: columns aj∈Rm\mathbf{a}_j\in\mathbb{R}^maj​∈Rm can be written A=[a1 ⋯ an]A=[\mathbf{a}_1\ \cdots\ \mathbf{a}_n]A=[a1​ ⋯ an​].
5. Add / scale: for the same shape, (A+B)ij=aij+bij(A+B)_{ij}=a_{ij}+b_{ij}(A+B)ij​=aij​+bij​ and (cA)ij=c aij(cA)_{ij}=c\,a_{ij}(cA)ij​=caij​.
This chapter focuses on reading stacked vectors safely and matching dimensions before full matrix multiplication.
In deep learning, weights are often matrices (or 2D slices of tensors). One layer’s linear map is “many dot products at once”; batching stacks samples along a row/column. In machine learning, the design matrix stacks feature vectors into one data table.
One‑line summary: a matrix bundles many vectors on one sheet; whether rows or columns are samples follows convention. The transpose swaps axes to match dimensions. Row dot products from Ch.02 become the coordinates of AuA\mathbf{u}Au. Next: matrix multiplication and linear maps.
Ch.01 gave vectors; Ch.02 gave dot products for one interaction. Ch.03 extends that interaction to whole tables. Matrices are the language of losses, gradients, and weight updates.
Real data is usually many samples × many features. Stating the shape m×nm\times nm×n makes the layout explicit; wrong shapes silently break code.
Training data is often a design matrix; linear models are written as matrix–vector products. Logistic/softmax, linear SVM, and matrix‑factorization recommendations all use batched vector operations.
Columns span a subspace (column space); fitting data to a lower dimension is projection to a subspace (later chapters).
The table below lists symbols and dimension rules for problem solving. Worked patterns illustrate typical steps.
  • Symbolm×nm\times nm×n
  • Meaningmmm rows and nnn columns
  • Symbolaija_{ij}aij​
  • Meaningentry at row iii, column jjj
  • SymbolATA^{\mathsf T}AT
  • Meaningtranspose: (AT)ji=aij(A^{\mathsf T})_{ji}=a_{ij}(AT)ji​=aij​
  • Symbolcolumn aj\mathbf{a}_jaj​
  • Meaningcolumn jjj of AAA as a vector
  • Symbolsame shape
  • MeaningA+BA+BA+B only if dimensions match
  • SymbolAuA\mathbf{u}Au (preview)
  • Meaningvector of row–u\mathbf{u}u dot products
SymbolMeaning
m×nm\times nm×nmmm rows and nnn columns
aija_{ij}aij​entry at row iii, column jjj
ATA^{\mathsf T}ATtranspose: (AT)ji=aij(A^{\mathsf T})_{ji}=a_{ij}(AT)ji​=aij​
column aj\mathbf{a}_jaj​column jjj of AAA as a vector
same shapeA+BA+BA+B only if dimensions match
AuA\mathbf{u}Au (preview)vector of row–u\mathbf{u}u dot products
Details
① Shape Always check dimensions before add/multiply.
② Transpose Swaps sample/feature axes when needed.
③ Row/column view Meaning depends on the problem setup.
④ Ch.02 link Each row dotted with u\mathbf{u}u gives one coordinate of AuA\mathbf{u}Au.

Practice problems

Below are 10 problems sampled from a bank of 60 (4 easy · 3 medium · 3 hard; order easy→medium→hard). Each item is multiple choice—pick the option number.

Which property matches a zero matrix?
1 / 10