English
Linear algebra studies vector spaces and transformations that preserve addition and scaling. Its computational importance comes from two uses: representing many quantities together and understanding how a transformation mixes or preserves them. A matrix is a coordinate representation of a linear map, not merely a rectangular container of numbers.
An image, a collection of measurements, or an embedding can be represented by a vector, but the interpretation supplies additional constraints. Probability vectors have nonnegative entries summing to one; they lie within a vector space without themselves forming a vector space. Adding two distributions does not produce another normalized distribution. Likewise, numbering categories does not make their averages meaningful.
Basic algebra, functions, and coordinate geometry suffice to begin. Work with small matrices and pictures before relying on numerical libraries, then use those libraries to connect the exact mathematics to finite-precision computation. Calculus is useful later for optimization but is not needed to understand span, rank, orthogonality, or linear systems.
From Navigation Tables to the Language of Machine Learning
Solving simultaneous equations predates modern matrix notation by many centuries. Elimination procedures appear in the Chinese Nine Chapters on the Mathematical Art. Astronomy, surveying, and other measurement practices later created a recurring problem: observations are noisy, and more equations may be available than can be satisfied exactly. Least squares turns this inconsistency into an optimization problem by minimizing squared residuals.
Nineteenth-century work on determinants, matrices, and vector spaces gradually separated transformations from particular systems of coordinates. This changed the question from “how do I solve these equations?” to “what structure does this whole family of transformations possess?” Eigenvalues, invariant subspaces, and decompositions made that structure available for both reasoning and computation.
Electronic computers amplified another distinction: an exact algebraic identity is not automatically a reliable numerical recipe. Scientific computing required error analysis, effective storage layouts, and reusable implementations. The development of libraries such as LINPACK and LAPACK brought mathematical algorithms together with machine-aware kernels; LAPACK’s first public release was in 1992. This is part of the history of software as well as mathematics: trustworthy infrastructure allowed users to solve domain problems without rebuilding every numerical method.
Graphics, signal processing, statistics, and machine learning draw on this shared infrastructure for different reasons. Their use of matrices does not make all their models linear. Neural networks, for example, compose affine maps with nonlinear operations; removing those nonlinearities would collapse a chain of affine layers into a single affine map.
Transformations, Decompositions, and the Geometry of High Dimensions
Keep three levels in view: the transformation, its coordinates, and the algorithm used to compute with those coordinates. Changing a basis changes a matrix without changing the underlying map. Changing an algorithm can change numerical error without changing the exact mathematical answer. Many apparent contradictions disappear once these levels are separated.
Structure: What Matrices Reveal About Transformations
For a real matrix A with n columns, an input x has n coordinates. The column space contains all reachable outputs Ax; the null space contains input changes that produce zero output. The rank is the dimension of the column space, and rank plus nullity equals n. A map that sends (x, y) to x + y has rank one: adding (1, -1) to the input leaves the output unchanged. Knowing only the sum cannot tell us how much each input contributed.
The other two fundamental subspaces complete this picture. The row space is perpendicular to the null space and contains the input directions that the map can distinguish. The left null space is perpendicular to the column space in the output space. Thus an observation outside the column space cannot be produced exactly by Ax; least squares finds the closest attainable output in Euclidean distance. These subspaces distinguish information lost in the input from discrepancies the model cannot explain in the output.
Decompositions express a matrix as a sequence of simpler transformations. In QR, Q supplies orthonormal directions and R gives coefficients relative to them; this is useful for solving least-squares problems. The singular value decomposition, A = UΣVᵀ, describes a change of input coordinates, nonnegative scaling along selected directions, and a change of output coordinates. Small singular values mean that a change in some input direction produces only a small output change. Recovering that input change from a noisy observation can therefore amplify the noise.
For a square matrix, an eigenvector satisfies Av = λv: applying the map changes its scale but preserves its direction. Repeated application gives A^k v = λ^k v, so eigenvalues help describe growth, decay, and oscillation in repeated transformations. Arbitrary matrices need not possess a basis of eigenvectors; the SVD applies to rectangular matrices as well.
Truncating the SVD gives a best approximation of a specified lower rank in the usual spectral or Frobenius norm. Whether it preserves useful information depends on the task and the units of the features. PCA applies this idea to centered data, choosing directions of greatest variance; high variance is not automatically high predictive relevance. LoRA instead constrains an adaptation’s weight update to low rank. It does not require the pretrained weight matrix itself to have low rank.
Orthogonality: The Geometry of Angle and Projection
An inner product provides a way to discuss lengths and angles. Orthogonal directions have zero inner product, which makes their contributions separable in geometric calculations. Orthogonality is not the same as statistical independence. Even zero covariance does not generally imply independence without additional assumptions.
Least squares gives a useful geometric picture: the fitted vector Ax is the orthogonal projection of the observation vector b onto the column space of A, and the residual is perpendicular to that space. The coefficient vector x is a set of coordinates, not that projection itself. Nor is every matrix multiplication a projection; projection maps have special structure.
Dot products also appear in attention, but the rest of the computation matters. In standard softmax attention, a zero query-key score does not assign zero weight: two unmasked scores (0, 0) produce weights (1/2, 1/2). The value vectors therefore still contribute. This follows directly from the attention formula.
High-dimensional geometric claims likewise need assumptions. Independent random unit vectors uniformly drawn from a sphere have dot product with mean zero and root-mean-square size 1/√d; they tend toward orthogonality as dimension grows. Learned embeddings need not follow that distribution. Geometry is a model to check against the data, not a property granted merely by having many coordinates.
Numerical Stability: The Gap Between Mathematics and Computation
A mathematical formula and the sequence of floating-point operations used to evaluate it can behave differently. For example, softmax assigns weights proportional to exp(x_i). With inputs (1000, 999), directly evaluating these exponentials overflows in ordinary double precision. Subtracting the largest input first gives (0, -1), whose softmax weights are approximately (0.731, 0.269). Subtracting the same constant from all inputs leaves the mathematical ratio unchanged because a common exponential factor cancels. This is an example of improving the calculation while preserving the function.
Conditioning concerns the problem: how much can its answer change when its input changes slightly? Stability concerns the algorithm: how much error does the computational procedure introduce? A backward-stable solver produces the exact solution of a nearby problem, but that solution can still be far from the desired one if the problem is ill-conditioned.
For example, let A = diag(1, ε) with small positive ε. Solving Ax = b gives x₂ = b₂/ε, so an absolute perturbation δ in b₂ becomes δ/ε in the recovered second component. A better implementation cannot remove this sensitivity from the original question. Better measurements, a reformulated model, or regularization may help, but they change the available information or the problem being solved.
Prefer solving a linear system to explicitly forming an inverse merely to multiply by it. QR often provides a safer least-squares route than forming the normal equations, which can square the condition number for a full-column-rank matrix. Pivoting, scaling, and appropriate factorizations address particular numerical risks; none guarantees accuracy for every problem. Inspect residuals and sensitivity together: a small residual alone does not establish a small solution error.
What Studying This Changes
Linear algebra makes an array operation interpretable. Ask what each axis represents, which map the multiplication implements, what information its null space discards, and which metric makes the comparison meaningful. Rescaling a feature from meters to millimeters can change a distance-based method even though the observations describe the same objects.
A compact exercise is to fit a line to a small set of noisy points. Write the design matrix, explain the column-space projection, solve with QR or SVD, then make two columns nearly dependent and perturb the data. Comparing coefficients, predictions, and residuals reveals why “the solver returned numbers” is only the beginning of numerical understanding.
Resources
Books and Texts
The canonical entry is Gilbert Strang’s Introduction to Linear Algebra (6th ed., 2023), best used alongside his MIT 18.06 lectures (free on MIT OpenCourseWare and YouTube). Strang organizes the subject around the four fundamental subspaces and builds geometric intuition throughout; the lectures are worth watching even if you have read the book, because the emphasis differs in ways that deepen understanding. His approach is unusual in that computation and geometry are developed together rather than separately — every matrix operation is accompanied by its geometric meaning. For most learners this is the right starting point, and many practitioners return to it as a reference throughout their careers.
Sheldon Axler’s Linear Algebra Done Right (4th ed., 2024, free online) takes a different route: it builds linear algebra on linear maps rather than matrices, and minimizes the role of determinants (which Axler treats as a late consequence rather than a foundational tool). The approach produces structural clarity that Strang’s computational emphasis does not emphasize — the student who works through Axler understands why the spectral theorem holds, not just that it holds. The two books are genuinely complementary: Strang for computational intuition and geometric feeling, Axler for structural depth. Learners aiming for ML research benefit from reading both.
For numerical linear algebra — the subject of how computations actually behave on machines — Trefethen and Bau’s Numerical Linear Algebra (SIAM, 1997) is the canonical reference. It covers conditioning, backward stability, QR factorization, the QR algorithm for eigenvalues, and SVD computation with both theoretical depth and computational precision. The writing is unusually clear for numerical mathematics. After learning matrix factorizations, read the lectures on conditioning, stability, and backward error analysis; use the remaining lectures when their numerical questions arise.
For the connection to machine learning, Deisenroth, Faisal, and Ong’s Mathematics for Machine Learning (Cambridge, 2020, free at mml-book.com) develops linear algebra through Chapters 2 and 3 with explicit ML connections — PCA, Gaussian processes, regression — and develops matrix calculus in Chapter 5, which most linear algebra texts do not cover. It is not as rigorous as Axler or as comprehensive as Strang, but it is the most direct path from linear algebra to ML applications.
For geometric intuition before formal study, 3Blue1Brown’s Essence of Linear Algebra (YouTube, free) is the best available resource. The animated visualizations show what linear transformations do to space — stretching, rotating, projecting — in ways that static text cannot match. It is not a substitute for rigorous study but is the right first encounter for any learner whose prior exposure was procedural.
For advanced matrix theory, Horn and Johnson’s Matrix Analysis (2nd ed., 2012) and Golub and Van Loan’s Matrix Computations (4th ed.) are the comprehensive references — too encyclopedic to read cover to cover, but indispensable when a specific result needs the definitive treatment.
The Matrix Cookbook (Petersen and Pedersen, free, 2012) is a reference document, not a textbook: a compendium of matrix derivative identities for when you need to differentiate a scalar with respect to a matrix. It does not teach matrix calculus but is the standard practical reference for computing such gradients.
| Book | Role | Type |
|---|---|---|
| Strang, Introduction to Linear Algebra (6th ed.) + MIT 18.06 (free) | Canonical entry; geometric and computational | Entry |
| Axler, Linear Algebra Done Right (4th ed., free online) | Structural depth; linear maps over matrices | Depth |
| Trefethen & Bau, Numerical Linear Algebra | Numerical computation and stability | Reference |
| Deisenroth, Faisal & Ong, Mathematics for Machine Learning Ch. 2–5 (free) | ML-specific bridge; matrix calculus | Entry |
| Horn & Johnson, Matrix Analysis (2nd ed.) | Advanced matrix theory | Reference |
| Golub & Van Loan, Matrix Computations (4th ed.) | Computational algorithms reference | Reference |
| Petersen & Pedersen, The Matrix Cookbook (free) | Matrix derivative reference | Reference |
| Halmos, Finite-Dimensional Vector Spaces | Classical theoretical depth | Depth |
Courses and Lectures
MIT 18.06 (Gilbert Strang’s Linear Algebra) is available in full on MIT OpenCourseWare with lectures, problem sets, and exams. Several versions exist, the most recent filmed in 2011; all are worth using. The Spring 2020 iteration with Strang teaching is available on YouTube. This is the single most-watched linear algebra course in the world and for good reason: the pedagogical approach is exemplary.
MIT 18.065 (Matrix Methods in Data Analysis, Signal Processing, and Machine Learning) is Strang’s more advanced course connecting linear algebra to contemporary applications — SVD, low-rank approximation, optimization, and neural networks. It presupposes 18.06 and extends it directly into ML territory. Full lectures are available free on YouTube.
3Blue1Brown’s Essence of Linear Algebra (YouTube, free, 15 videos) is not a course in the conventional sense but a visual essay on the geometry of linear algebra. Watch it before or alongside a formal course, not after.
fast.ai’s Computational Linear Algebra (Rachel Thomas, free) covers the subject from a numerical and machine-learning perspective, including implementations in Python. Practical complement to the more theoretical MIT courses.
Linear Algebra: Foundations to Frontiers (LAFF, UT Austin, on edX) links linear algebra to matrix software development and algorithms for matrix computation. It is most useful after an initial exposure to vectors, matrices, and systems of equations; its strength is showing that linear algebra is not only a collection of formulas but an implementation discipline behind high-performance numerical software.
| Course | Platform | Type |
|---|---|---|
| MIT 18.06 Linear Algebra (Strang, free) | MIT OCW / YouTube | Entry |
| MIT 18.065 Matrix Methods (Strang, free) | MIT OCW / YouTube | Depth |
| 3Blue1Brown, Essence of Linear Algebra (free) | YouTube | Practice |
| fast.ai Computational Linear Algebra (Rachel Thomas, free) | fast.ai / YouTube | Practice |
| LAFF, Linear Algebra: Foundations to Frontiers | edX / UT Austin | Practice |
Practice, Tools, and Code
3Blue1Brown (mentioned above) is the primary visualization resource. The videos on linear transformations, dot products, cross products, eigenvectors, and the Gram-Schmidt process are the most effective visualizations of these concepts available anywhere.
Immersive Linear Algebra is an interactive visual book whose figures can be manipulated directly in the browser. It is not a replacement for Strang, Axler, or numerical linear algebra, but it is excellent for building geometric intuition about vectors, matrices, determinants, eigenvectors, and singular value decomposition before the formal machinery becomes dense.
Implementing linear algebra from scratch in NumPy is more educational than reading about it. A productive sequence: implement matrix multiplication from loops, verify it against NumPy, time both; implement the Gram-Schmidt process; implement power iteration to find the dominant eigenvector; implement the compact SVD from scratch using the QR algorithm. These implementations are short (each under 50 lines) and the process of making them correct and numerically stable teaches more about the subject than equivalent reading time.
NumPy’s linear algebra documentation (numpy.linalg) and SciPy’s documentation are the standard references for computational practice. Understanding which function to use for which problem — np.linalg.solve vs. np.linalg.lstsq, np.linalg.eigh vs. np.linalg.eig — requires understanding the mathematical distinctions between symmetric and general matrices, between full-rank and rank-deficient cases, between the symmetric eigenvalue problem and the general one.
| Resource | Platform | Type |
|---|---|---|
| Immersive Linear Algebra (free) | immersivemath.com | Practice |
| NumPy / SciPy linear algebra documentation (free) | numpy.org / scipy.org | Reference |
| Scratch implementations in NumPy | Local | Practice |
Traps
| Trap | Why it misleads | Better response |
|---|---|---|
| Learning matrix arithmetic without maps | Correct multiplication can conceal a mistaken representation. | State the input space, output space, dimensions, and meaning of each axis. |
| Treating every numeric encoding as geometry | Category codes and differently scaled features do not automatically support meaningful distances. | Justify the representation and metric before interpreting proximity. |
| Confusing conditioning with stability | An accurate algorithm can solve an intrinsically sensitive problem. | Perturb the data and distinguish problem sensitivity from computational error. |
| Calling every linear operation a projection | General maps can rotate, shear, scale, or discard information. | Check the defining property of the operation and identify its null and column spaces. |
| Equating orthogonality with independence or zero attention | These statements belong to different mathematical models. | Follow the full statistical or computational definition, including normalization. |
| Trusting a decomposition without checking its purpose | Best low-rank approximation is defined by a norm, not by the user’s task. | Compare reconstruction error with the information the downstream task needs. |
中文
线性代数研究向量空间，以及保持加法和数乘的变换。它在计算中的重要性来自两种用途：把许多量组织在一起，以及理解变换如何混合或保留这些量。矩阵是线性映射在选定坐标下的表示，不只是一个长方形的数字容器。
图像、测量数据或嵌入都可以表示为向量，但具体含义还会带来额外约束。概率向量的各项非负且总和为一；它们位于向量空间之内，自身却不构成向量空间。两个概率分布直接相加，并不会得到另一个归一化分布。同样，给类别编号，也不会使编号的平均值自动获得意义。
具备基础代数、函数和坐标几何知识就可以开始。先用小矩阵和图形建立理解，再利用数值库连接精确数学与有限精度计算。微积分在后续优化中很有用，但理解张成空间、秩、正交性和线性方程组并不需要它。
从导航表到机器学习的语言
联立方程的求解比现代矩阵记号早了许多世纪。中国《九章算术》中已经出现消元过程。此后，天文、测量等实践反复遇到另一个问题：观测带有噪声，方程可能多到无法同时精确满足。最小二乘法通过最小化残差平方和，把这种不一致转化为优化问题。
十九世纪关于行列式、矩阵与向量空间的研究，逐渐把变换本身与特定坐标系分开。问题从“怎样解这组方程”，扩展为“这一类变换具有怎样的结构”。特征值、不变子空间和各种分解，使这些结构既能用于推理，也能用于计算。
电子计算机放大了另一种区别：精确的代数恒等式，不会自动成为可靠的数值计算步骤。科学计算需要误差分析、有效的存储布局和可复用的实现。LINPACK、LAPACK 等库的发展，把数学算法与适应机器特点的运算内核结合起来；LAPACK 首次公开发布于 1992 年。这既是数学史，也是软件史：可靠的基础设施让使用者能处理领域问题，而不必重新实现每一种数值方法。
图形学、信号处理、统计与机器学习出于不同原因使用这些共同基础。使用矩阵，并不意味着模型整体是线性的。例如，神经网络把仿射映射与非线性运算组合起来；如果去掉非线性部分，一串仿射层就会合并成一个仿射映射。
变换、分解与高维几何
同时留意三个层次：变换本身、它的坐标表示，以及利用这些坐标进行计算的算法。换基会改变矩阵，却不改变底层映射。换算法可能改变数值误差，却不改变精确的数学答案。分清这些层次，许多表面矛盾就会消失。
结构：矩阵揭示了变换的什么
对于有 n 列的实矩阵 A，输入 x 有 n 个坐标。列空间包含所有可达到的输出 Ax；零空间包含所有产生零输出的输入变化。秩是列空间的维数，秩与零空间维数之和等于 n。把 (x, y) 映射为 x + y 的变换，其秩为一：给输入加上 (1, -1)，输出保持不变。只知道总和，就无法确定两个输入各自贡献了多少。
另外两个基本子空间补充了这幅图景。行空间垂直于零空间，包含映射能够区分的输入方向。左零空间则在输出空间中垂直于列空间。因此，列空间之外的观测，不可能由 Ax 精确产生；最小二乘会按欧氏距离寻找最接近的可达到输出。这些子空间帮助区分：哪些输入信息丢失了，哪些输出差异无法由模型解释。
矩阵分解把一个矩阵表示成一串较简单的变换。在 QR 中，Q 提供标准正交方向，R 给出相对于这些方向的系数，适合用于最小二乘求解。奇异值分解 A = UΣVᵀ 描述了输入坐标的变换、沿选定方向的非负缩放，以及输出坐标的变换。很小的奇异值意味着：沿某个输入方向的变化，只产生很小的输出变化。因此，从带噪观测中恢复这个输入变化，可能会放大噪声。
对于方阵，特征向量满足 Av = λv：映射改变它的倍数，却保持其方向。反复应用得到 A^k v = λ^k v，因此特征值有助于描述重复变换中的增长、衰减与振荡。任意矩阵未必具有由特征向量组成的基；SVD 则也适用于矩形矩阵。
截断 SVD 在通常的谱范数或 Frobenius 范数下，给出指定较低秩的最佳近似。但它是否保留有用信息，取决于任务和特征单位。PCA 将这一思路用于中心化数据，选择方差最大的方向；方差大不等于预测价值高。LoRA 则把适配过程中的权重更新限制为低秩，并不要求预训练权重矩阵本身具有低秩。
正交性：角度与投影的几何
内积使我们能够讨论长度和角度。正交方向的内积为零，因此在几何计算中，它们的贡献可以分开处理。正交不等于统计独立；即使协方差为零，缺少额外条件时也通常不能推出独立。
最小二乘提供了一个直观图像：拟合向量 Ax 是观测向量 b 在 A 的列空间上的正交投影，残差则垂直于该空间。系数向量 x 是一组坐标，并不是投影向量本身。也不是每次矩阵乘法都是投影；投影映射具有特殊结构。
注意力机制也使用点积，但还必须考虑后续计算。在标准 softmax 注意力中，查询与键的分数为零，不表示权重为零：两个未被掩蔽的分数 (0, 0) 会产生权重 (1/2, 1/2)，对应的值向量仍会参与输出。这直接来自注意力公式。
高维几何结论同样有前提。从单位球面独立、均匀抽取的随机单位向量，其点积均值为零，均方根大小为 1/√d；维数增加时，它们趋于正交。学习得到的嵌入未必服从这种分布。几何性质需要用数据检验，不能只因为坐标很多就自动成立。
数值稳定性：数学与计算之间的距离
数学公式与求它的浮点运算步骤，可能表现不同。例如，softmax 分配的权重与 exp(x_i) 成比例。输入为 (1000, 999) 时，直接计算指数会在普通双精度中溢出。先减去最大的输入，得到 (0, -1)，就能算出约为 (0.731, 0.269) 的 softmax 权重。所有输入减去同一个常数，不改变数学上的比例，因为公共的指数因子会约去。这是在保持函数不变的同时，改善计算方式的例子。
条件性描述问题本身：输入稍有变化，答案可能改变多少？稳定性描述算法：计算步骤本身引入多大误差？后向稳定的求解器给出某个邻近问题的精确解，但如果原问题是病态的，这个解仍可能离所需答案很远。
例如，令 A = diag(1, ε)，其中 ε 是很小的正数。解 Ax = b 得到 x₂ = b₂/ε，因此 b₂ 中大小为 δ 的绝对扰动，会在恢复出的第二个分量中变成 δ/ε。更好的实现无法消除原问题自身的这种敏感性。改善测量、重新建模或正则化可能有帮助，但它们改变了可用信息，或改变了正在求解的问题。
如果目的只是用逆矩阵乘一个向量，应优先直接解线性方程组，而不是显式构造逆矩阵。对于最小二乘，QR 通常比构造正规方程更稳妥；对列满秩矩阵，正规方程可能把条件数平方。选主元、缩放和适当的矩阵分解各自应对特定风险，没有哪一种保证所有问题都准确。应同时检查残差和敏感性：残差小，本身不能保证解的误差小。
学习这一部分会改变什么
线性代数使数组运算变得可以解释。追问每个轴代表什么，乘法实现什么映射，零空间丢弃哪些信息，以及什么度量使比较有意义。把一个特征的单位从米换成毫米，可能改变基于距离的方法，即使观测描述的仍是同样的对象。
一个紧凑的练习是给少量带噪点拟合直线。写出设计矩阵，解释列空间上的投影，用 QR 或 SVD 求解，再让两列变得近乎相关并扰动数据。比较系数、预测值和残差，就能看见为什么“求解器返回了数字”只是理解数值计算的起点。
资源
书籍与文本
经典入口是 Gilbert Strang 的 Introduction to Linear Algebra（第 6 版，2023），最好与他的 MIT 18.06 讲座（可在 MIT OpenCourseWare 和 YouTube 免费获取）一起使用。Strang 围绕四个基本子空间组织这门学科，并始终建立几何直觉；即使已经读过教材，讲座仍然值得看，因为讲座的强调方式不同，会加深理解。他的方法很特别：计算和几何不是分开发展的，而是一起发展——每一个矩阵操作都会伴随它的几何意义。对多数学习者来说，这是正确起点；许多实践者在整个职业生涯中也会不断回到它，把它作为参考。
Sheldon Axler 的 Linear Algebra Done Right（第 4 版，2024，免费在线）采取了另一条路线：它把线性代数建立在线性映射上，而不是矩阵上，并尽量降低行列式的角色（Axler 把行列式视为后来的结果，而不是基础工具）。这种方法带来了 Strang 的计算强调不突出的一种结构清晰性——读完 Axler 的学生理解的是谱定理为什么成立，而不只是知道它成立。两本书确实互补：Strang 提供计算直觉和几何感觉，Axler 提供结构深度。目标是机器学习研究的学习者，读二者都会受益。
对于数值线性代数——研究计算在机器上实际如何表现的学科——Trefethen 和 Bau 的 Numerical Linear Algebra（SIAM，1997）是经典参考。它覆盖条件数、后向稳定性、QR 分解、用于特征值的 QR 算法，以及 SVD 计算，兼具理论深度和计算精确性。对于数值数学书来说，它写得异常清楚。学过矩阵分解之后，可以阅读条件性、稳定性与后向误差分析的相关讲次；其余部分则按具体数值问题查阅。
对于与机器学习的连接，Deisenroth、Faisal 和 Ong 的 Mathematics for Machine Learning（Cambridge，2020，可在 mml-book.com 免费获取）在第 2、3 章中通过明确的机器学习连接发展线性代数——PCA、高斯过程、回归——并在第 5 章发展矩阵微积分，这是多数线性代数教材不覆盖的内容。它不如 Axler 严谨，也不如 Strang 全面，但它是从线性代数通向机器学习应用的最直接路径。
对于正式学习前的几何直觉，3Blue1Brown 的 Essence of Linear Algebra（YouTube，免费）是目前最好的资源。动画可视化会展示线性变换对空间做了什么——拉伸、旋转、投影——而这是静态文本无法匹配的。它不能替代严谨学习，但对于那些过去只接触过程序化线性代数的学习者来说，这是正确的第一接触。
对于高级矩阵理论，Horn 和 Johnson 的 Matrix Analysis（第 2 版，2012）以及 Golub 和 Van Loan 的 Matrix Computations（第 4 版）是综合性参考——它们过于百科全书化，不适合从头读到尾，但当需要某个具体结果的权威处理时，它们不可替代。
Matrix Cookbook（Petersen 和 Pedersen，免费，2012）是参考文档，不是教材：它汇集了矩阵求导恒等式，适用于你需要对一个标量相对于矩阵求导的时候。它不教授矩阵微积分，但它是计算这类梯度的标准实践参考。
| 书籍 | 作用 | 类型 |
|---|---|---|
| Strang, Introduction to Linear Algebra（第 6 版）+ MIT 18.06（免费） | 经典入口；几何与计算 | 入门 |
| Axler, Linear Algebra Done Right（第 4 版，免费在线） | 结构深度；以线性映射而非矩阵为中心 | 深入 |
| Trefethen & Bau, Numerical Linear Algebra | 数值计算与稳定性 | 参考 |
| Deisenroth, Faisal & Ong, Mathematics for Machine Learning Ch. 2–5（免费） | 面向 ML 的桥梁；矩阵微积分 | 入门 |
| Horn & Johnson, Matrix Analysis（第 2 版） | 高级矩阵理论 | 参考 |
| Golub & Van Loan, Matrix Computations（第 4 版） | 计算算法参考 | 参考 |
| Petersen & Pedersen, The Matrix Cookbook（免费） | 矩阵求导参考 | 参考 |
| Halmos, Finite-Dimensional Vector Spaces | 经典理论深度 | 深入 |
课程与讲座
MIT 18.06（Gilbert Strang 的 Linear Algebra）在 MIT OpenCourseWare 上完整提供，包括讲座、习题和考试。它有多个版本，最近一次录制于 2011 年；每个版本都值得使用。Strang 2020 春季学期的版本也可以在 YouTube 上观看。这是全世界观看人数最多的线性代数课程，理由很充分：它的教学方式堪称典范。
MIT 18.065（Matrix Methods in Data Analysis, Signal Processing, and Machine Learning）是 Strang 更高级的课程，把线性代数连接到当代应用——SVD、低秩近似、优化和神经网络。它预设学习者已经掌握 18.06，并直接延伸到机器学习领域。完整讲座可在 YouTube 免费获取。
3Blue1Brown 的 Essence of Linear Algebra（YouTube，免费，共 15 个视频）不是传统意义上的课程，而是一篇关于线性代数几何的视觉论文。应当在正式课程之前或同时观看，而不是之后才看。
fast.ai 的 Computational Linear Algebra（Rachel Thomas，免费）从数值和机器学习视角讲解这门学科，并包含 Python 实现。它是对更偏理论的 MIT 课程的实践补充。
Linear Algebra: Foundations to Frontiers（LAFF，UT Austin，edX）把线性代数连接到矩阵软件开发和矩阵计算算法。它最适合在已经初步接触向量、矩阵和方程组之后使用；它的强项在于展示线性代数不只是一组公式，也是一门高性能数值软件背后的实现学科。
| 课程 | 平台 | 类型 |
|---|---|---|
| MIT 18.06 Linear Algebra（Strang，免费） | MIT OCW / YouTube | 入门 |
| MIT 18.065 Matrix Methods（Strang，免费） | MIT OCW / YouTube | 深入 |
| 3Blue1Brown, Essence of Linear Algebra（免费） | YouTube | 实践 |
| fast.ai Computational Linear Algebra（Rachel Thomas，免费） | fast.ai / YouTube | 实践 |
| LAFF, Linear Algebra: Foundations to Frontiers | edX / UT Austin | 实践 |
实践、工具与代码
3Blue1Brown（上文已提到）是主要的可视化资源。关于线性变换、点积、叉积、特征向量和 Gram-Schmidt 过程的视频，是目前最有效的相关概念可视化材料。
Immersive Linear Algebra 是一本交互式视觉书，书中的图形可以直接在浏览器中操作。它不能替代 Strang、Axler 或数值线性代数，但在正式工具变得密集之前，它非常适合建立对向量、矩阵、行列式、特征向量和奇异值分解的几何直觉。
用 NumPy 从零实现线性代数，比只是阅读相关内容更有教育价值。一个有成效的顺序是：用循环实现矩阵乘法，与 NumPy 结果核对，并对二者计时；实现 Gram-Schmidt 过程；实现幂迭代来寻找主特征向量；使用 QR 算法从零实现紧 SVD。这些实现都很短（每个不到 50 行），而让它们正确且数值稳定的过程，比同等时间的阅读更能教会你这门学科。
NumPy 的线性代数文档（numpy.linalg）和 SciPy 文档，是计算实践中的标准参考。理解哪个问题该用哪个函数——np.linalg.solve 还是 np.linalg.lstsq，np.linalg.eigh 还是 np.linalg.eig——需要理解数学区别：对称矩阵与一般矩阵，满秩与秩亏情形，对称特征值问题与一般特征值问题。
| 资源 | 平台 | 类型 |
|---|---|---|
| Immersive Linear Algebra（免费） | immersivemath.com | 实践 |
| NumPy / SciPy 线性代数文档（免费） | numpy.org / scipy.org | 参考 |
| NumPy 从零实现练习 | 本地 | 实践 |
陷阱
| 陷阱 | 为什么会误导 | 更好的做法 |
|---|---|---|
| 只学矩阵运算，不理解映射 | 乘法做对了，也可能使用了错误的表示。 | 说明输入空间、输出空间、维度及各个轴的含义。 |
| 把所有数字编码都当作几何 | 类别编号和尺度不同的特征不会自动支持有意义的距离。 | 解释表示与度量的依据，再讨论远近。 |
| 混淆条件性与稳定性 | 准确的算法也可能在求解一个本来就敏感的问题。 | 扰动数据，区分问题敏感性与计算误差。 |
| 把每种线性运算都叫作投影 | 一般映射可以旋转、剪切、缩放或丢失信息。 | 检查运算的定义性质，识别零空间与列空间。 |
| 把正交等同于独立或零注意力 | 这些命题属于不同的数学模型。 | 沿完整的统计或计算定义推理，包括归一化步骤。 |
| 使用分解却不检查目的 | 最佳低秩近似由范数定义，不是由用户任务定义。 | 比较重建误差与后续任务需要保留的信息。 |