English
A vector is an object that can be added to other vectors and scaled by numbers. That sentence sounds like a definition, and it is — but the power of linear algebra comes from recognizing how many different things satisfy it. Geometric arrows in the plane are vectors. Lists of pixel intensities in an image are vectors. Functions on an interval are vectors. The state of a quantum system is a vector. Probability distributions are vectors. Once you recognize that all of these objects share the same algebraic structure, every theorem about abstract vector spaces applies to all of them simultaneously. Linear algebra is the mathematics of this shared structure, and the reason it appears everywhere in computer science is that the data CS manipulates — images, embeddings, states, distributions — almost always has it.
A linear transformation is a function between vector spaces that respects addition and scaling: T(u + v) = T(u) + T(v) and T(cv) = cT(v). Matrices are how linear transformations are computed once you choose a coordinate system. The matrix is not the transformation; it is a representation of the transformation in a particular basis, and the same transformation has different matrix representations in different bases. This distinction — between the intrinsic mathematical object and its coordinate representation — is the conceptual move that separates a deep understanding of linear algebra from a working familiarity with matrix arithmetic. A neural network weight matrix represents a linear transformation; understanding what the transformation does geometrically — how it stretches, rotates, and projects the input space — is what makes the mechanics of neural networks legible rather than opaque.
Linear algebra is the mathematical foundation with the broadest reach across computer science. Machine learning is built from it: training is optimization over parameters, inference is matrix multiplication, attention is inner products. Computer graphics uses it for every transformation of 3D geometry. Scientific computing solves large linear systems as its primary operation. Quantum computing is linear algebra over complex vector spaces. Post-quantum cryptography is built on the hardness of problems in integer lattices, which are discrete subgroups of vector spaces. The investment in this subject pays dividends in more downstream CS subjects than any other single mathematical topic.
Prerequisites: Calculus (§2.4) and Probability (§2.5) are independent. Discrete mathematics (§2.2) is useful but not required. Chapter 5 (Artificial Intelligence) requires this section.
From Navigation Tables to the Language of Machine Learning
Linear algebra’s history is unusual: the machinery came centuries before the theory that unified it. Gaussian elimination — the systematic procedure for solving systems of linear equations — appears in a Chinese mathematical text, the Nine Chapters on the Mathematical Art, compiled around 200 BCE. The procedure was rediscovered in seventeenth-century Europe and systematized by Gauss in the nineteenth century, who used it for the least-squares adjustment of astronomical and geodetic observations. Gauss did not have the concept of a matrix; he had a method for solving linear systems. The matrix as an object in its own right, with its own algebra, was introduced by Arthur Cayley in 1858. Cayley defined matrix multiplication and proved that square matrices satisfy their own characteristic polynomial — what we now call the Cayley-Hamilton theorem — without having a general concept of eigenvalue or a theory of vector spaces.
The abstract concept of a vector space arrived in the 1880s. Hermann Grassmann had been developing a general “algebra of extensions” since 1844, arguing that oriented line segments, areas, and volumes were all instances of the same kind of object, obeying the same algebraic laws. Giuseppe Peano gave the first modern axiomatic definition of a vector space in 1888. But the abstraction was slow to catch on; most mathematicians continued to work with specific coordinate representations. It was only in the early twentieth century, through the work of mathematicians including David Hilbert and Hermann Weyl, that the abstract approach became standard — and immediately proved its value by showing that polynomials, functions, and geometric vectors were all the same kind of object, subject to the same theorems. The fundamental theorem of linear algebra — that the rank plus the nullity of a linear map equals the dimension of the domain — holds for all vector spaces simultaneously, which would not have been visible if one worked only with coordinate arrays.
The computational turn began with the computer. In 1947, John von Neumann and Herman Goldstine published a systematic analysis of the numerical errors that arise when Gaussian elimination is performed in finite-precision arithmetic — one of the earliest papers on numerical analysis. They showed that the accuracy of the computed solution depends not on the size of the matrix but on a quantity they called the condition number: roughly, the ratio of the largest to the smallest singular value. A matrix with high condition number is ill-conditioned; small errors in the input or in arithmetic produce large errors in the output. This insight was foundational. Numerical linear algebra, the study of how to perform linear algebraic computations reliably in finite-precision arithmetic, grew from this analysis into a discipline of its own.
Through the 1950s and 1960s, practical algorithms for the computational problems of linear algebra were developed. The QR algorithm, discovered by Francis (1961) and Kublanovskaya (1961) independently, computes all eigenvalues of a matrix stably and efficiently; it remains the standard method used in numerical software today. The singular value decomposition (SVD) had been known in theoretical form since the late nineteenth century, but Golub and Reinsch’s 1970 paper gave the first numerically stable algorithm for computing it. The LINPACK and LAPACK libraries, developed through the 1970s and 1980s, codified these algorithms into reusable software that became the foundation for all subsequent scientific computing.
The SVD deserves special attention because its history and its CS applications are tightly coupled. Mathematically, SVD decomposes any matrix M into M = UΣVᵀ, where U and V are orthogonal and Σ is diagonal with non-negative entries. Computationally, it reveals the intrinsic geometry of a linear map: U gives the output directions, V gives the input directions, and the singular values in Σ give the scaling in each direction. When Deerwester et al. applied SVD to term-document matrices in 1990, producing latent semantic analysis, they showed that low-rank approximation — keeping only the largest singular values — could extract semantic structure from text. This was a direct precursor to word embeddings and, more distantly, to the attention mechanisms in transformers. The attention matrix in a transformer is computed by projections whose structure the SVD illuminates: the effective rank of the attention matrix, the directions of maximum variance, and the relationship between keys and queries all become visible through singular value analysis.
The deep learning era transformed linear algebra from a computational tool into something closer to a native language. A forward pass through a neural network is a composition of matrix multiplications and pointwise nonlinearities. The gradient of the loss with respect to parameters is computed by matrix calculus — specifically by the chain rule applied to these matrix compositions. When Vaswani et al. introduced the transformer architecture in 2017, the core operation — scaled dot-product attention — was Q·Kᵀ/√d, a matrix multiplication followed by a softmax, followed by multiplication by V. The entire architecture is linear algebra at scale, executed on hardware designed specifically for matrix operations. GPUs are matrix multiplication engines; TPUs optimize for a narrower class of tensor operations; the entire infrastructure of modern AI is built around the linear algebraic operations that vector spaces support.
The contemporary frontier that is less visible but equally consequential is lattice cryptography. A lattice is a discrete subgroup of a vector space: a grid of points generated by integer linear combinations of a basis. Problems involving lattices — finding the shortest vector in a lattice, solving linear systems over integers — are believed to be computationally hard even for quantum computers. NIST’s 2022 selection of CRYSTALS-Kyber and CRYSTALS-Dilithium for standardization, followed by the 2024 publication of ML-KEM and ML-DSA standards derived from them, made lattice problems the foundation of the first standardized post-quantum cryptographic systems. Integer linear algebra, long the province of number theorists, now underpins the security of internet communication.
Transformations, Decompositions, and the Geometry of High Dimensions
Linear algebra for CS organizes itself around three interconnected concerns that appear, in various forms, across every downstream application.
Structure: What Matrices Reveal About Transformations
The rank of a matrix is the dimension of its image — the space of outputs it can produce. A rank-deficient matrix maps many inputs to the same output, which means it loses information. In machine learning, the effective rank of a weight matrix determines the expressive capacity of that layer; low-rank weight matrices have been found throughout large pretrained models, which is the mathematical basis for LoRA and other low-rank adaptation techniques. In data compression and approximation, the Eckart-Young theorem says that the best rank-k approximation to a matrix in the Frobenius norm is obtained by keeping the k largest singular values — this is PCA, and it underlies most dimensionality reduction in ML.
The four fundamental subspaces of a matrix M — the column space, null space, row space, and left null space — partition the input and output spaces into orthogonal complements. A vector in the null space maps to zero; adding it to an input does not change the output. A vector in the row space contributes to the output. Understanding which directions are amplified, preserved, and annihilated by a linear map is the geometric content of the rank-nullity theorem: dim(column space) + dim(null space) = number of columns.
The eigendecomposition and singular value decomposition give this geometric picture numerical content. Eigenvalues and eigenvectors describe the directions a square matrix merely scales — if Av = λv, then v is preserved in direction and scaled by λ. The spectral theorem says that for symmetric matrices (including covariance matrices, graph Laplacians, and Hessians of loss functions), the eigenvectors form an orthonormal basis and the eigenvalues are real. This is what makes principal component analysis possible: the covariance matrix is symmetric, its eigenvectors are orthogonal, and the eigenvector with the largest eigenvalue is the direction of maximum variance in the data. The eigenvalues of the Hessian of a loss function describe the curvature of the loss landscape — large eigenvalues correspond to steep directions where a small step causes a large change in loss; small eigenvalues correspond to flat directions. This is why poorly conditioned optimization landscapes are hard to navigate: gradient descent must take small steps to avoid the steep directions, making progress in the flat directions painfully slow.
Orthogonality: The Geometry of Angle and Projection
An inner product on a vector space assigns a scalar to every pair of vectors, measuring their “alignment.” In Euclidean space, the inner product of u and v is uᵀv; normalized by the lengths of the vectors, this gives the cosine of the angle between them. Orthogonality — zero inner product — is the algebraic expression of perpendicularity: two vectors that share no component along each other’s direction.
Orthogonality is central to ML because it governs similarity and independence. Cosine similarity, the standard measure of proximity in embedding spaces, is the normalized inner product. The key claim of word2vec and its successors is that related concepts have high cosine similarity; that is, their embedding vectors point in similar directions in a high-dimensional space. Orthogonality between the query and key vectors in attention means the token has no influence; high inner product means strong attention. The geometry of how vectors relate through their inner products determines the behavior of nearly every modern ML system.
The geometry of high-dimensional spaces is counterintuitive in ways that matter. In d dimensions, two random unit vectors have expected inner product of order 1/√d — most vectors are nearly orthogonal. The volume of a d-dimensional ball is concentrated near its surface, not its center. The distance to the nearest neighbor in a point cloud becomes almost indistinguishable from the distance to the farthest neighbor as d grows. These are not pathological corner cases; they are the normal behavior of high-dimensional geometry, and they directly affect embedding design, approximate nearest-neighbor search, and the analysis of overparameterized models.
Projection is the practical expression of orthogonality. The projection of a vector b onto a subspace V is the closest point in V to b. This is least-squares: when an overdetermined linear system Ax = b has no exact solution, the least-squares solution minimizes ‖b - Ax‖², which is the projection of b onto the column space of A. The normal equations AᵀAx = Aᵀb give the solution; the QR decomposition of A provides a numerically stable way to compute it. Every regression problem in statistics, every linear layer’s output, is a projection in this sense.
Numerical Stability: The Gap Between Mathematics and Computation
Exact linear algebra and floating-point linear algebra are different subjects. A matrix that is mathematically nonsingular can be numerically singular — its computed inverse wildly inaccurate — if it is ill-conditioned. The condition number κ(A) = σ_max/σ_min is the ratio of the largest to the smallest singular value; for a well-conditioned matrix, small perturbations in the input produce small perturbations in the output; for an ill-conditioned matrix, the amplification factor is κ(A).
The canonical example is solving Ax = b by Gaussian elimination. The algorithm is mathematically correct — it finds the unique solution when one exists — but the intermediate computations involve dividing by pivots that can be small, amplifying rounding errors. Partial pivoting (choosing the largest available pivot) controls this, and it is why numerical software implementations of Gaussian elimination always pivot. LU decomposition with partial pivoting is the standard method for solving general linear systems; QR decomposition is more stable and is preferred for least-squares problems; Cholesky decomposition is fastest and most stable for symmetric positive definite matrices.
The softmax function in neural networks is a case where numerical stability matters immediately. Naively, softmax(x)_i = exp(x_i)/Σexp(x_j) overflows when the x_j are large. The stable implementation subtracts the maximum before exponentiation: exp(x_i - max(x))/Σexp(x_j - max(x)). This does not change the mathematical result but prevents overflow. Every ML framework implements this internally, but understanding why reveals something about the relationship between mathematical specifications and computational implementations.
What Studying This Changes
Linear algebra changes what a practitioner can see when they look at a computation.
The most immediate change is geometric reading. A weight matrix is not an array of numbers — it is a linear transformation that does something specific to the space of inputs: it rotates, scales, and projects. A practitioner who has internalized the SVD can look at the singular values of a weight matrix and immediately read off: how many effective dimensions does this matrix have? Are there near-zero singular values indicating redundant structure? Is the matrix nearly rank-deficient? These questions have answers, and the answers predict model behavior in ways that treating the matrix as an opaque array of parameters does not.
The second change is decomposition fluency. Many algorithms that appear distinct are the same linear algebra operation. PCA is SVD. Least-squares is QR. Linear regression is a projection. Spectral clustering is an eigendecomposition of the graph Laplacian. Once the underlying operation is recognized, analysis transfers: insights about PCA apply to latent semantic analysis and low-rank approximation of attention matrices. The apparently vast landscape of ML techniques collapses, for someone fluent in linear algebra, into a smaller number of structural moves applied in different contexts.
The third change is the ability to read mathematical arguments across fields. Linear algebra is the common language of physics, statistics, control theory, signal processing, and computer science. A practitioner fluent in it can read papers from any of these fields without a translation step. The quantum state formalism, the Kalman filter, the Riccati equation, and scaled dot-product attention are all statements in the same language. Fluency in this language is what makes intellectual cross-pollination possible.
The fourth change is numerical awareness. The distinction between a mathematically correct algorithm and a numerically stable one is invisible to someone who has not studied numerical linear algebra. Solving a linear system by inverting the matrix is correct mathematics and bad computation; QR decomposition is better; for symmetric positive definite systems, Cholesky is better still. These choices matter not in contrived cases but in routine practice, and the ability to make them requires understanding condition numbers, backward stability, and how floating-point arithmetic differs from exact arithmetic.
Resources
Books and Texts
The canonical entry is Gilbert Strang’s Introduction to Linear Algebra (6th ed., 2023), best used alongside his MIT 18.06 lectures (free on MIT OpenCourseWare and YouTube). Strang organizes the subject around the four fundamental subspaces and builds geometric intuition throughout; the lectures are worth watching even if you have read the book, because the emphasis differs in ways that deepen understanding. His approach is unusual in that computation and geometry are developed together rather than separately — every matrix operation is accompanied by its geometric meaning. For most learners this is the right starting point, and many practitioners return to it as a reference throughout their careers.
Sheldon Axler’s Linear Algebra Done Right (4th ed., 2024, free online) takes a different route: it builds linear algebra on linear maps rather than matrices, and minimizes the role of determinants (which Axler treats as a late consequence rather than a foundational tool). The approach produces structural clarity that Strang’s computational emphasis does not emphasize — the student who works through Axler understands why the spectral theorem holds, not just that it holds. The two books are genuinely complementary: Strang for computational intuition and geometric feeling, Axler for structural depth. Learners aiming for ML research benefit from reading both.
For numerical linear algebra — the subject of how computations actually behave on machines — Trefethen and Bau’s Numerical Linear Algebra (SIAM, 1997) is the canonical reference. It covers conditioning, backward stability, QR factorization, the QR algorithm for eigenvalues, and SVD computation with both theoretical depth and computational precision. The writing is unusually clear for numerical mathematics. Most ML practitioners benefit from reading the first few chapters (on conditioning and stability) and returning to the rest when specific numerical questions arise.
For the connection to machine learning, Deisenroth, Faisal, and Ong’s Mathematics for Machine Learning (Cambridge, 2020, free at mml-book.com) develops linear algebra through Chapters 2 and 3 with explicit ML connections — PCA, Gaussian processes, regression — and develops matrix calculus in Chapter 5, which most linear algebra texts do not cover. It is not as rigorous as Axler or as comprehensive as Strang, but it is the most direct path from linear algebra to ML applications.
For geometric intuition before formal study, 3Blue1Brown’s Essence of Linear Algebra (YouTube, free) is the best available resource. The animated visualizations show what linear transformations do to space — stretching, rotating, projecting — in ways that static text cannot match. It is not a substitute for rigorous study but is the right first encounter for any learner whose prior exposure was procedural.
For advanced matrix theory, Horn and Johnson’s Matrix Analysis (2nd ed., 2012) and Golub and Van Loan’s Matrix Computations (4th ed.) are the comprehensive references — too encyclopedic to read cover to cover, but indispensable when a specific result needs the definitive treatment.
The Matrix Cookbook (Petersen and Pedersen, free, 2012) is a reference document, not a textbook: a compendium of matrix derivative identities for when you need to differentiate a scalar with respect to a matrix. It does not teach matrix calculus but is the standard practical reference for computing such gradients.
| Book | Role | Type |
|---|---|---|
| Strang, Introduction to Linear Algebra (6th ed.) + MIT 18.06 (free) | Canonical entry; geometric and computational | Entry |
| Axler, Linear Algebra Done Right (4th ed., free online) | Structural depth; linear maps over matrices | Depth |
| Trefethen & Bau, Numerical Linear Algebra | Numerical computation and stability | Reference |
| Deisenroth, Faisal & Ong, Mathematics for Machine Learning Ch. 2–5 (free) | ML-specific bridge; matrix calculus | Entry |
| Horn & Johnson, Matrix Analysis (2nd ed.) | Advanced matrix theory | Reference |
| Golub & Van Loan, Matrix Computations (4th ed.) | Computational algorithms reference | Reference |
| Petersen & Pedersen, The Matrix Cookbook (free) | Matrix derivative reference | Reference |
| Halmos, Finite-Dimensional Vector Spaces | Classical theoretical depth | Depth |
Courses and Lectures
MIT 18.06 (Gilbert Strang’s Linear Algebra) is available in full on MIT OpenCourseWare with lectures, problem sets, and exams. Several versions exist, the most recent filmed in 2011; all are worth using. The Spring 2020 iteration with Strang teaching is available on YouTube. This is the single most-watched linear algebra course in the world and for good reason: the pedagogical approach is exemplary.
MIT 18.065 (Matrix Methods in Data Analysis, Signal Processing, and Machine Learning) is Strang’s more advanced course connecting linear algebra to contemporary applications — SVD, low-rank approximation, optimization, and neural networks. It presupposes 18.06 and extends it directly into ML territory. Full lectures are available free on YouTube.
3Blue1Brown’s Essence of Linear Algebra (YouTube, free, 15 videos) is not a course in the conventional sense but a visual essay on the geometry of linear algebra. Watch it before or alongside a formal course, not after.
fast.ai’s Computational Linear Algebra (Rachel Thomas, free) covers the subject from a numerical and machine-learning perspective, including implementations in Python. Practical complement to the more theoretical MIT courses.
Linear Algebra: Foundations to Frontiers (LAFF, UT Austin, on edX) links linear algebra to matrix software development and algorithms for matrix computation. It is most useful after an initial exposure to vectors, matrices, and systems of equations; its strength is showing that linear algebra is not only a collection of formulas but an implementation discipline behind high-performance numerical software.
| Course | Platform | Type |
|---|---|---|
| MIT 18.06 Linear Algebra (Strang, free) | MIT OCW / YouTube | Entry |
| MIT 18.065 Matrix Methods (Strang, free) | MIT OCW / YouTube | Depth |
| 3Blue1Brown, Essence of Linear Algebra (free) | YouTube | Practice |
| fast.ai Computational Linear Algebra (Rachel Thomas, free) | fast.ai / YouTube | Practice |
| LAFF, Linear Algebra: Foundations to Frontiers | edX / UT Austin | Practice |
Practice, Tools, and Code
3Blue1Brown (mentioned above) is the primary visualization resource. The videos on linear transformations, dot products, cross products, eigenvectors, and the Gram-Schmidt process are the most effective visualizations of these concepts available anywhere.
Immersive Linear Algebra is an interactive visual book whose figures can be manipulated directly in the browser. It is not a replacement for Strang, Axler, or numerical linear algebra, but it is excellent for building geometric intuition about vectors, matrices, determinants, eigenvectors, and singular value decomposition before the formal machinery becomes dense.
Implementing linear algebra from scratch in NumPy is more educational than reading about it. A productive sequence: implement matrix multiplication from loops, verify it against NumPy, time both; implement the Gram-Schmidt process; implement power iteration to find the dominant eigenvector; implement the compact SVD from scratch using the QR algorithm. These implementations are short (each under 50 lines) and the process of making them correct and numerically stable teaches more about the subject than equivalent reading time.
NumPy’s linear algebra documentation (numpy.linalg) and SciPy’s documentation are the standard references for computational practice. Understanding which function to use for which problem — np.linalg.solve vs. np.linalg.lstsq, np.linalg.eigh vs. np.linalg.eig — requires understanding the mathematical distinctions between symmetric and general matrices, between full-rank and rank-deficient cases, between the symmetric eigenvalue problem and the general one.
| Resource | Platform | Type |
|---|---|---|
| Immersive Linear Algebra (free) | immersivemath.com | Practice |
| NumPy / SciPy linear algebra documentation (free) | numpy.org / scipy.org | Reference |
| Scratch implementations in NumPy | Local | Practice |
Traps
| Trap | Why it misleads | Better response |
|---|---|---|
| Learning procedures without geometric meaning | Row reduction, matrix inversion, and determinant computation can all be learned as mechanical procedures without any sense of what they are doing to space. This procedural knowledge fails when novel combinations arise — when you need to understand why a matrix is ill-conditioned, or what SVD reveals about a weight matrix, or why the gradient of a matrix expression has the form it does. The gap between procedural and geometric understanding is large and invisible to the learner until they hit the wall. | Before learning any procedure, ask what it does geometrically. What does matrix multiplication do to vectors? What does the determinant measure? What do eigenvectors preserve? 3Blue1Brown’s series answers these questions visually; return to it as a geometric reference when a procedure’s meaning becomes unclear. |
| Treating matrices as the fundamental object | Matrices are coordinate representations of linear maps. The map exists independently of the coordinates; the matrix does not. Changing the basis changes the matrix but not the underlying transformation. A learner who thinks in matrices rather than maps is confused by every change-of-basis argument and misses the insight that two very different-looking matrices can represent the same transformation. | Study Axler after Strang. Axler’s approach — building everything on linear maps and introducing matrices as a computational tool — forces the distinction. Read chapter 3 of Axler specifically on the relationship between linear maps and their matrix representations. |
| Skipping numerical linear algebra | Exact linear algebra and floating-point linear algebra are different subjects, and the differences are not exotic. Solving a linear system by inverting the matrix is correct mathematics and poor computation. Naive softmax overflows. Gram-Schmidt orthogonalization is numerically unstable; Householder reflections are stable. These differences arise in routine ML engineering and are invisible to practitioners who have not learned the numerical perspective. | Read at minimum the first two chapters of Trefethen and Bau, which develop condition number and backward stability. Then implement softmax both naively and with the log-sum-exp trick, observe the difference, and understand why. |
| Not extending from matrices to tensors | Standard linear algebra courses treat matrices (2D arrays) as the primary object. Deep learning operates on batches of sequences of vectors — 3D or 4D tensors — and the operations on them (batched matrix multiplication, attention, convolution) extend matrix operations in ways that are not automatic. Practitioners who never make this extension are perpetually confused by framework code. | Study NumPy’s broadcasting semantics and einsum notation deliberately. Implement batched matrix multiplication from scratch. Implement the attention operation — Q·Kᵀ/√d followed by softmax and multiplication by V — in NumPy without a framework. |
| Confusing SVD with eigendecomposition | Both decompose a matrix into simpler components, but they apply to different situations and reveal different structure. SVD works for any matrix; eigendecomposition requires a square matrix and may produce complex eigenvalues if the matrix is not symmetric. Applying eigendecomposition where SVD is appropriate, or treating singular values as eigenvalues, produces incorrect analyses. | Learn both decompositions from their geometric motivation first. SVD: any linear map decomposes into a rotation of the input space, an axis-aligned scaling, and a rotation of the output space. Eigendecomposition: a square map has directions it merely scales, if they exist. The geometric pictures clarify when each applies. |
| Deferring implementation | Linear algebra understood only through reading develops slowly. The correspondence between mathematical objects and numerical computation — what matrix multiplication actually does entry by entry, how the QR algorithm iteratively builds eigenvectors — is built through implementing, not reading. | Implement matrix multiplication, Gram-Schmidt, power iteration, and a thin SVD from scratch in NumPy. Each takes under an hour and teaches more than the equivalent reading time. The implementations are short enough that making them correct and efficient is achievable; the process is what teaches. |
中文
向量是一个可以与其他向量相加、也可以被数字缩放的对象。这句话听起来像一个定义，它确实是定义——但线性代数的力量，来自于认识到有多少不同的东西都满足它。平面中的几何箭头是向量。图像中的像素强度列表是向量。区间上的函数是向量。量子系统的状态是向量。概率分布也是向量。一旦你认识到这些对象共享同一种代数结构，那么关于抽象向量空间的每一个定理，就会同时适用于它们。线性代数就是研究这种共享结构的数学；它之所以在计算机科学中无处不在，是因为 CS 操作的数据——图像、嵌入、状态、分布——几乎总是具有这种结构。
线性变换是向量空间之间的一种函数，它保持加法和缩放：T(u + v) = T(u) + T(v)，T(cv) = cT(v)。一旦你选择了坐标系，矩阵就是计算线性变换的方式。矩阵不是变换本身；它是在某个特定基下对变换的表示，同一个变换在不同基下会有不同的矩阵表示。这一区分——内在数学对象与坐标表示之间的区分——是把对线性代数的深刻理解，与对矩阵运算的工作性熟悉区分开的概念动作。神经网络的权重矩阵表示一个线性变换；理解这个变换在几何上做了什么——它如何拉伸、旋转和投影输入空间——才能让神经网络的机制变得可读，而不是不透明。
线性代数是计算机科学中覆盖范围最广的数学基础。机器学习建立在它之上：训练是在参数上的优化，推理是矩阵乘法，注意力是内积。计算机图形学用它完成每一次 3D 几何变换。科学计算把求解大型线性系统作为核心操作。量子计算是在复向量空间上的线性代数。后量子密码学建立在整数格中困难问题的基础上，而整数格是向量空间中的离散子群。在线性代数上的投入，会在后续 CS 主题中产生比任何单一数学主题都更多的回报。
前置知识：微积分（§2.4）和概率论（§2.5）彼此独立。离散数学（§2.2）有帮助，但不是必需。第五章（人工智能）要求掌握本节。
从导航表到机器学习的语言
线性代数的历史很特别：具体工具比统一它们的理论早出现了几个世纪。高斯消元——系统求解线性方程组的过程——出现在约公元前 200 年成书的中国数学典籍《九章算术》中。这个过程在十七世纪的欧洲被重新发现，并在十九世纪由 Gauss 系统化；Gauss 用它对天文和大地测量观测进行最小二乘调整。Gauss 当时并没有矩阵的概念；他拥有的是求解线性系统的方法。矩阵作为一种独立对象，并拥有自己的代数，是 Arthur Cayley 在 1858 年引入的。Cayley 定义了矩阵乘法，并证明了方阵满足自身的特征多项式——也就是我们今天所说的 Cayley-Hamilton 定理——但当时还没有一般的特征值概念，也没有向量空间理论。
抽象的向量空间概念出现在 1880 年代。Hermann Grassmann 从 1844 年起就在发展一种一般性的“扩张代数”，主张有向线段、面积和体积都是同一类对象的实例，服从相同的代数规律。Giuseppe Peano 在 1888 年给出了第一个现代公理化向量空间定义。但这个抽象并没有很快被接受；多数数学家仍然使用具体的坐标表示。直到二十世纪初，经过 David Hilbert、Hermann Weyl 等数学家的工作，抽象方法才成为标准，并立刻证明了自己的价值：多项式、函数和几何向量都可以被看成同一类对象，服从同一批定理。线性代数基本定理——一个线性映射的秩加上零空间维数等于定义域维数——同时适用于所有向量空间；如果只处理坐标数组，这一点并不会显现出来。
计算转向始于计算机。1947 年，John von Neumann 和 Herman Goldstine 发表了一篇系统分析：当高斯消元在有限精度算术中执行时，会产生怎样的数值误差。这是数值分析最早的论文之一。他们表明，计算所得解的准确性并不取决于矩阵大小，而取决于一个他们称为条件数的量：粗略说，就是最大奇异值与最小奇异值之比。条件数高的矩阵是病态的；输入或算术中的小误差会在输出中变成大误差。这个洞见具有奠基意义。数值线性代数——研究如何在有限精度算术中可靠执行线性代数计算——正是从这一分析中成长为一门独立学科。
整个 1950 年代和 1960 年代，人们发展出了一批用于线性代数计算问题的实用算法。QR 算法由 Francis（1961）和 Kublanovskaya（1961）独立发现，可以稳定且高效地计算矩阵的所有特征值；它至今仍是数值软件中使用的标准方法。奇异值分解（SVD）早在十九世纪末就已经以理论形式为人所知，但 Golub 和 Reinsch 在 1970 年的论文给出了第一个计算 SVD 的数值稳定算法。LINPACK 和 LAPACK 库在 1970 年代和 1980 年代发展出来，把这些算法固化为可复用软件，并成为后来所有科学计算的基础。
SVD 值得特别关注，因为它的历史与它在 CS 中的应用紧密耦合。在数学上，SVD 会把任意矩阵 M 分解为 M = UΣVᵀ，其中 U 和 V 是正交矩阵，Σ 是带有非负元素的对角矩阵。在计算上，它揭示了一个线性映射的内在几何：U 给出输出方向，V 给出输入方向，Σ 中的奇异值给出每个方向上的缩放。当 Deerwester 等人在 1990 年把 SVD 应用于词项—文档矩阵，产生潜在语义分析时，他们证明了低秩近似——只保留最大的奇异值——可以从文本中抽取语义结构。这是词嵌入的直接先驱，也更间接地通向了 Transformer 中的注意力机制。Transformer 中的注意力矩阵由投影计算而来，而 SVD 能照亮这些投影的结构：注意力矩阵的有效秩、最大方差方向，以及 key 与 query 之间的关系，都可以通过奇异值分析变得可见。
深度学习时代把线性代数从一种计算工具，转变成了更接近原生语言的东西。神经网络的一次前向传播，是矩阵乘法和逐点非线性的组合。损失函数相对于参数的梯度，是通过矩阵微积分计算出来的——具体说，就是把链式法则应用到这些矩阵组合上。当 Vaswani 等人在 2017 年引入 Transformer 架构时，其核心操作——缩放点积注意力——就是 Q·Kᵀ/√d，先进行矩阵乘法，再接 softmax，再乘以 V。整个架构都是大规模线性代数，并运行在专门为矩阵运算设计的硬件之上。GPU 是矩阵乘法引擎；TPU 针对更窄的一类张量运算进行优化；现代 AI 的整个基础设施，都围绕向量空间所支持的线性代数操作构建起来。
当代更不显眼但同样重要的前沿，是格密码学。格是向量空间中的离散子群：由某组基的整数线性组合生成的点阵。涉及格的问题——例如在格中寻找最短向量，或者在整数上求解线性系统——被认为即使对量子计算机来说也很难。NIST 在 2022 年选择 CRYSTALS-Kyber 和 CRYSTALS-Dilithium 作为标准化对象，随后又在 2024 年发布由它们衍生而来的 ML-KEM 和 ML-DSA 标准，这使格问题成为第一批标准化后量子密码系统的基础。长期属于数论研究范围的整数线性代数，如今支撑着互联网通信安全。
变换、分解与高维几何
面向 CS 的线性代数围绕三个相互连接的问题组织自身；它们会以不同形式出现在每一个后续应用中。
结构：矩阵揭示了变换的什么
矩阵的秩，是其像空间的维度——也就是它能够产生的输出空间的维度。一个秩亏矩阵会把许多输入映射到同一个输出，这意味着它丢失了信息。在机器学习中，一个权重矩阵的有效秩决定了该层的表达能力；人们已经在大型预训练模型中发现大量低秩权重矩阵，这正是 LoRA 和其他低秩适配技术的数学基础。在数据压缩和近似中，Eckart-Young 定理说，在 Frobenius 范数下，一个矩阵的最佳 rank-k 近似可以通过保留最大的 k 个奇异值得到——这就是 PCA，也构成了机器学习中多数降维方法的基础。
矩阵 M 的四个基本子空间——列空间、零空间、行空间和左零空间——把输入空间和输出空间划分为正交补。零空间中的向量会被映射为零；把它加到输入上，不会改变输出。行空间中的向量会对输出产生贡献。理解一个线性映射会放大、保留和消除哪些方向，就是秩—零度定理的几何内容：dim(column space) + dim(null space) = 列数。
特征分解和奇异值分解会把这种几何图景赋予数值内容。特征值和特征向量描述的是一个方阵只是进行缩放的方向——如果 Av = λv，那么 v 的方向被保留，只被 λ 缩放。谱定理说，对于对称矩阵（包括协方差矩阵、图拉普拉斯矩阵和损失函数的 Hessian 矩阵），特征向量构成一组标准正交基，特征值都是实数。这正是主成分分析成为可能的原因：协方差矩阵是对称的，它的特征向量彼此正交，而具有最大特征值的特征向量就是数据中方差最大的方向。损失函数 Hessian 矩阵的特征值描述了损失景观的曲率——大特征值对应陡峭方向，在这些方向上迈出一小步就会导致损失大幅变化；小特征值对应平坦方向。这就是为什么病态优化景观难以导航：梯度下降必须采取小步长，以避免陡峭方向上的不稳定，这使它在平坦方向上的进展痛苦缓慢。
正交性：角度与投影的几何
向量空间上的内积，会为每一对向量分配一个标量，用来衡量它们的“对齐程度”。在欧几里得空间中，u 和 v 的内积是 uᵀv；如果除以两个向量的长度，就得到二者夹角的余弦。正交性——内积为零——是垂直性的代数表达：两个向量在彼此方向上没有任何分量。
正交性对机器学习至关重要，因为它支配相似性和独立性。余弦相似度是嵌入空间中衡量接近程度的标准方法，它就是归一化内积。word2vec 及其后继方法的核心主张是：相关概念具有高余弦相似度，也就是说，它们的嵌入向量在高维空间中指向相近方向。注意力机制中 query 向量与 key 向量正交，意味着该 token 没有影响；内积高，则意味着强注意力。向量如何通过内积相互关联的几何，决定了几乎每一个现代机器学习系统的行为。
高维空间的几何具有反直觉特征，而且这些特征很重要。在 d 维中，两个随机单位向量的期望内积是 1/√d 的量级——大多数向量几乎彼此正交。d 维球体的体积集中在表面附近，而不是中心。随着 d 增大，一个点云中最近邻的距离会变得几乎无法与最远邻的距离区分开。这些并不是病态的边角情况；它们就是高维几何的正常行为，并且会直接影响嵌入设计、近似最近邻搜索，以及对过参数化模型的分析。
投影是正交性的实践表达。向量 b 到子空间 V 上的投影，是 V 中离 b 最近的点。这就是最小二乘：当一个超定线性系统 Ax = b 没有精确解时，最小二乘解会最小化 ‖b - Ax‖²，也就是把 b 投影到 A 的列空间上。正规方程 AᵀAx = Aᵀb 给出这个解；A 的 QR 分解则提供了一种数值稳定的计算方式。统计学中的每一个回归问题、每一个线性层的输出，都可以在这个意义上理解为投影。
数值稳定性：数学与计算之间的距离
精确线性代数和浮点线性代数是不同学科。一个在数学上非奇异的矩阵，如果是病态的，就可能在数值上表现得像奇异矩阵——它的计算逆会严重不准确。条件数 κ(A) = σ_max/σ_min 是最大奇异值与最小奇异值之比；对于条件良好的矩阵，输入中的小扰动只会在输出中产生小扰动；对于病态矩阵，放大因子就是 κ(A)。
经典例子是用高斯消元求解 Ax = b。这个算法在数学上是正确的——当唯一解存在时，它会找到这个解——但中间计算需要除以主元，而主元可能很小，从而放大舍入误差。部分选主元（选择可用的最大主元）可以控制这一点，这也是为什么数值软件中的高斯消元实现总是会选主元。带部分选主元的 LU 分解是求解一般线性系统的标准方法；QR 分解更加稳定，更适合最小二乘问题；Cholesky 分解在对称正定矩阵上最快也最稳定。
神经网络中的 softmax 函数是数值稳定性立刻变得重要的一个案例。朴素实现中，softmax(x)_i = exp(x_i)/Σexp(x_j)，当 x_j 很大时会溢出。稳定实现会在指数运算前减去最大值：exp(x_i - max(x))/Σexp(x_j - max(x))。这不会改变数学结果，但能防止溢出。每个机器学习框架都会在内部这样实现；但理解为什么要这样做，会揭示数学定义与计算实现之间的关系。
学习这一部分会改变什么
线性代数会改变实践者在观察一个计算时能够看见什么。
最直接的变化，是几何阅读能力。一个权重矩阵不是一组数字；它是一个线性变换，会对输入空间做某件具体的事：旋转、缩放和投影。一个内化了 SVD 的实践者，可以看一个权重矩阵的奇异值，并立刻读出：这个矩阵有多少有效维度？是否存在接近零的奇异值，暗示冗余结构？这个矩阵是否近似秩亏？这些问题有答案，而这些答案能够以把矩阵视为不透明参数数组所无法做到的方式预测模型行为。
第二个变化，是分解流利度。许多看起来不同的算法，本质上是同一个线性代数操作。PCA 是 SVD。最小二乘是 QR。线性回归是投影。谱聚类是图拉普拉斯矩阵的特征分解。一旦识别出底层操作，分析就可以迁移：关于 PCA 的洞见可以应用到潜在语义分析，也可以应用到注意力矩阵的低秩近似。对于精通线性代数的人来说，机器学习技术中看似庞大的景观会坍缩为少数几种结构性动作，只是它们被应用在不同语境中。
第三个变化，是跨领域阅读数学论证的能力。线性代数是物理、统计、控制理论、信号处理和计算机科学的共同语言。精通它的实践者可以阅读这些领域的论文，而不需要转换语言。量子态形式主义、Kalman 滤波器、Riccati 方程和缩放点积注意力，都是同一种语言中的陈述。正是这种语言流利度，使智识上的跨领域传粉成为可能。
第四个变化，是数值意识。对于没有学过数值线性代数的人来说，“数学上正确的算法”和“数值上稳定的算法”之间的区别是不可见的。通过求逆矩阵来解线性系统，是正确的数学，但糟糕的计算；QR 分解更好；对于对称正定系统，Cholesky 更好。这些选择并不只在刻意构造的案例中重要，而是在日常实践中就很重要；要做出这些选择，需要理解条件数、后向稳定性，以及浮点算术与精确算术的不同。
资源
书籍与文本
经典入口是 Gilbert Strang 的 Introduction to Linear Algebra（第 6 版，2023），最好与他的 MIT 18.06 讲座（可在 MIT OpenCourseWare 和 YouTube 免费获取）一起使用。Strang 围绕四个基本子空间组织这门学科，并始终建立几何直觉；即使已经读过教材，讲座仍然值得看，因为讲座的强调方式不同，会加深理解。他的方法很特别：计算和几何不是分开发展的，而是一起发展——每一个矩阵操作都会伴随它的几何意义。对多数学习者来说，这是正确起点；许多实践者在整个职业生涯中也会不断回到它，把它作为参考。
Sheldon Axler 的 Linear Algebra Done Right（第 4 版，2024，免费在线）采取了另一条路线：它把线性代数建立在线性映射上，而不是矩阵上，并尽量降低行列式的角色（Axler 把行列式视为后来的结果，而不是基础工具）。这种方法带来了 Strang 的计算强调不突出的一种结构清晰性——读完 Axler 的学生理解的是谱定理为什么成立，而不只是知道它成立。两本书确实互补：Strang 提供计算直觉和几何感觉，Axler 提供结构深度。目标是机器学习研究的学习者，读二者都会受益。
对于数值线性代数——研究计算在机器上实际如何表现的学科——Trefethen 和 Bau 的 Numerical Linear Algebra（SIAM，1997）是经典参考。它覆盖条件数、后向稳定性、QR 分解、用于特征值的 QR 算法，以及 SVD 计算，兼具理论深度和计算精确性。对于数值数学书来说，它写得异常清楚。多数机器学习实践者都应该阅读前几章（条件数和稳定性），并在遇到具体数值问题时回到后续章节。
对于与机器学习的连接，Deisenroth、Faisal 和 Ong 的 Mathematics for Machine Learning（Cambridge，2020，可在 mml-book.com 免费获取）在第 2、3 章中通过明确的机器学习连接发展线性代数——PCA、高斯过程、回归——并在第 5 章发展矩阵微积分，这是多数线性代数教材不覆盖的内容。它不如 Axler 严谨，也不如 Strang 全面，但它是从线性代数通向机器学习应用的最直接路径。
对于正式学习前的几何直觉，3Blue1Brown 的 Essence of Linear Algebra（YouTube，免费）是目前最好的资源。动画可视化会展示线性变换对空间做了什么——拉伸、旋转、投影——而这是静态文本无法匹配的。它不能替代严谨学习，但对于那些过去只接触过程序化线性代数的学习者来说，这是正确的第一接触。
对于高级矩阵理论，Horn 和 Johnson 的 Matrix Analysis（第 2 版，2012）以及 Golub 和 Van Loan 的 Matrix Computations（第 4 版）是综合性参考——它们过于百科全书化，不适合从头读到尾，但当需要某个具体结果的权威处理时，它们不可替代。
Matrix Cookbook（Petersen 和 Pedersen，免费，2012）是参考文档，不是教材：它汇集了矩阵求导恒等式，适用于你需要对一个标量相对于矩阵求导的时候。它不教授矩阵微积分，但它是计算这类梯度的标准实践参考。
| 书籍 | 作用 | 类型 |
|---|---|---|
| Strang, Introduction to Linear Algebra（第 6 版）+ MIT 18.06（免费） | 经典入口；几何与计算 | 入门 |
| Axler, Linear Algebra Done Right（第 4 版，免费在线） | 结构深度；以线性映射而非矩阵为中心 | 深入 |
| Trefethen & Bau, Numerical Linear Algebra | 数值计算与稳定性 | 参考 |
| Deisenroth, Faisal & Ong, Mathematics for Machine Learning Ch. 2–5（免费） | 面向 ML 的桥梁；矩阵微积分 | 入门 |
| Horn & Johnson, Matrix Analysis（第 2 版） | 高级矩阵理论 | 参考 |
| Golub & Van Loan, Matrix Computations（第 4 版） | 计算算法参考 | 参考 |
| Petersen & Pedersen, The Matrix Cookbook（免费） | 矩阵求导参考 | 参考 |
| Halmos, Finite-Dimensional Vector Spaces | 经典理论深度 | 深入 |
课程与讲座
MIT 18.06（Gilbert Strang 的 Linear Algebra）在 MIT OpenCourseWare 上完整提供，包括讲座、习题和考试。它有多个版本，最近一次录制于 2011 年；每个版本都值得使用。Strang 2020 春季学期的版本也可以在 YouTube 上观看。这是全世界观看人数最多的线性代数课程，理由很充分：它的教学方式堪称典范。
MIT 18.065（Matrix Methods in Data Analysis, Signal Processing, and Machine Learning）是 Strang 更高级的课程，把线性代数连接到当代应用——SVD、低秩近似、优化和神经网络。它预设学习者已经掌握 18.06，并直接延伸到机器学习领域。完整讲座可在 YouTube 免费获取。
3Blue1Brown 的 Essence of Linear Algebra（YouTube，免费，共 15 个视频）不是传统意义上的课程，而是一篇关于线性代数几何的视觉论文。应当在正式课程之前或同时观看，而不是之后才看。
fast.ai 的 Computational Linear Algebra（Rachel Thomas，免费）从数值和机器学习视角讲解这门学科，并包含 Python 实现。它是对更偏理论的 MIT 课程的实践补充。
Linear Algebra: Foundations to Frontiers（LAFF，UT Austin，edX）把线性代数连接到矩阵软件开发和矩阵计算算法。它最适合在已经初步接触向量、矩阵和方程组之后使用；它的强项在于展示线性代数不只是一组公式，也是一门高性能数值软件背后的实现学科。
| 课程 | 平台 | 类型 |
|---|---|---|
| MIT 18.06 Linear Algebra（Strang，免费） | MIT OCW / YouTube | 入门 |
| MIT 18.065 Matrix Methods（Strang，免费） | MIT OCW / YouTube | 深入 |
| 3Blue1Brown, Essence of Linear Algebra（免费） | YouTube | 实践 |
| fast.ai Computational Linear Algebra（Rachel Thomas，免费） | fast.ai / YouTube | 实践 |
| LAFF, Linear Algebra: Foundations to Frontiers | edX / UT Austin | 实践 |
实践、工具与代码
3Blue1Brown（上文已提到）是主要的可视化资源。关于线性变换、点积、叉积、特征向量和 Gram-Schmidt 过程的视频，是目前最有效的相关概念可视化材料。
Immersive Linear Algebra 是一本交互式视觉书，书中的图形可以直接在浏览器中操作。它不能替代 Strang、Axler 或数值线性代数，但在正式工具变得密集之前，它非常适合建立对向量、矩阵、行列式、特征向量和奇异值分解的几何直觉。
用 NumPy 从零实现线性代数，比只是阅读相关内容更有教育价值。一个有成效的顺序是：用循环实现矩阵乘法，与 NumPy 结果核对，并对二者计时；实现 Gram-Schmidt 过程；实现幂迭代来寻找主特征向量；使用 QR 算法从零实现紧 SVD。这些实现都很短（每个不到 50 行），而让它们正确且数值稳定的过程，比同等时间的阅读更能教会你这门学科。
NumPy 的线性代数文档（numpy.linalg）和 SciPy 文档，是计算实践中的标准参考。理解哪个问题该用哪个函数——np.linalg.solve 还是 np.linalg.lstsq，np.linalg.eigh 还是 np.linalg.eig——需要理解数学区别：对称矩阵与一般矩阵，满秩与秩亏情形，对称特征值问题与一般特征值问题。
| 资源 | 平台 | 类型 |
|---|---|---|
| Immersive Linear Algebra（免费） | immersivemath.com | 实践 |
| NumPy / SciPy 线性代数文档（免费） | numpy.org / scipy.org | 参考 |
| NumPy 从零实现练习 | 本地 | 实践 |
陷阱
| 陷阱 | 为什么会误导 | 更好的回应 |
|---|---|---|
| 学会过程，却没有几何意义 | 行化简、矩阵求逆和行列式计算都可以作为机械过程被学会，而完全不知道它们对空间做了什么。当遇到新的组合时，这种程序性知识就会失效——比如需要理解为什么一个矩阵是病态的、SVD 揭示了权重矩阵的什么、或者为什么矩阵表达式的梯度具有某种形式。程序性理解与几何理解之间的差距很大，而且在学习者撞墙之前往往不可见。 | 学任何过程之前，先问它在几何上做了什么。矩阵乘法对向量做了什么？行列式度量什么？特征向量保留了什么？3Blue1Brown 的系列会用视觉方式回答这些问题；当某个过程的意义变得不清楚时，应回到它，把它当作几何参考。 |
| 把矩阵当成根本对象 | 矩阵是线性映射的坐标表示。映射独立于坐标而存在；矩阵不是。改变基会改变矩阵，但不会改变底层变换。一个只按矩阵而非映射思考的学习者，会被每个换基论证困住，也会错过这样一个洞见：两个看起来非常不同的矩阵，可能表示同一个变换。 | 在 Strang 之后学习 Axler。Axler 的方法——把一切建立在线性映射上，并把矩阵作为计算工具引入——会迫使学习者区分二者。特别阅读 Axler 第 3 章，理解线性映射与其矩阵表示之间的关系。 |
| 跳过数值线性代数 | 精确线性代数和浮点线性代数是不同学科，而且差异并不冷门。通过求逆矩阵来解线性系统，是正确的数学，但糟糕的计算。朴素 softmax 会溢出。Gram-Schmidt 正交化在数值上不稳定；Householder 反射是稳定的。这些差异会出现在日常 ML 工程中，而没有学过数值视角的实践者看不见它们。 | 至少阅读 Trefethen 和 Bau 的前两章，理解条件数和后向稳定性。然后分别实现朴素 softmax 和使用 log-sum-exp 技巧的 softmax，观察差别，并理解原因。 |
| 没有从矩阵延伸到张量 | 标准线性代数课程把矩阵（二维数组）作为主要对象。深度学习操作的是一批序列向量——三维或四维张量——而对它们进行的操作（批量矩阵乘法、注意力、卷积）是矩阵操作的扩展，但这种扩展并不是自动的。没有完成这一延伸的实践者，会长期被框架代码困住。 | 有意识地学习 NumPy 的广播语义和 einsum 记号。从零实现批量矩阵乘法。用 NumPy 在不使用框架的情况下实现注意力操作——Q·Kᵀ/√d，接 softmax，再乘以 V。 |
| 混淆 SVD 与特征分解 | 二者都会把矩阵分解成更简单的组成部分，但它们适用于不同情形，揭示的结构也不同。SVD 适用于任意矩阵；特征分解要求方阵，而且如果矩阵不是对称的，还可能产生复特征值。在适合 SVD 的地方使用特征分解，或者把奇异值当成特征值，都会产生错误分析。 | 先从几何动机学习两种分解。SVD：任意线性映射都可以分解为输入空间的一次旋转、一次轴对齐缩放，以及输出空间的一次旋转。特征分解：如果存在，一个方阵会有若干只是被缩放的方向。几何图景会澄清什么时候该用哪一种。 |
| 推迟实现 | 只通过阅读理解线性代数，会发展得很慢。数学对象与数值计算之间的对应关系——矩阵乘法逐项到底做了什么，QR 算法如何迭代地构造特征向量——是通过实现建立起来的，而不是通过阅读建立起来的。 | 用 NumPy 从零实现矩阵乘法、Gram-Schmidt、幂迭代和薄 SVD。每个实现都不到一小时，但它们教会的东西比同等阅读时间更多。这些实现足够短，因此让它们正确且高效是可达成的；真正产生学习的是这个过程。 |