English
Calculus studies change and accumulation; analysis makes their limiting arguments precise. Their role in computer science is especially visible when discrete machines model continuous quantities: time, position, probability densities, or adjustable parameters. A derivative describes local sensitivity, and an integral combines contributions over a domain. These are complementary operations under the conditions of the fundamental theorem of calculus.
The bridge to computation introduces a second question: how well does a finite calculation approximate the mathematical object? An optimization theorem may describe exact real arithmetic while an implementation uses rounded numbers and stops after a finite number of steps. Learning this subject means keeping the model, the approximation, and the implementation connected.
Begin with functions, algebra, and graphs. Single-variable calculus supplies the initial intuition; linear algebra then makes multivariable derivatives intelligible. More advanced analysis becomes useful as the questions demand it. General programming, discrete algorithms, and elementary probability need not wait until this entire sequence is complete.
From Infinitesimals to Automatic Differentiation
Calculus grew from several connected problems: tangents to curves, areas, rates of motion, and the relation between a changing quantity and its accumulated effect. Newton and Leibniz developed influential seventeenth-century formulations, building on earlier work. Their achievement was to connect and systematize methods that could be used across problems; the subsequent history includes clarifying why those methods work and where they fail.
Nineteenth-century analysis made limits, continuity, and convergence more precise. This was practically consequential. A sequence of increasingly fine approximations need not approach the desired answer in the way intuition suggests, and a convergent infinite process may be too slow to be useful. The distinction between a limiting result and an effective error bound remains central to numerical computation.
Numerical methods existed before electronic computers. Faster machines changed the scale of simulations and the importance of accumulated rounding, stability, and algorithm choice. Simply transferring a hand-calculation method to a machine could reproduce its weaknesses at much greater speed. Reliable numerical software became a discipline in its own right.
Automatic differentiation opened another connection: a program built from differentiable operations can carry the information needed to differentiate their composition. Forward and reverse accumulation apply the chain rule systematically; neural-network backpropagation is a prominent use of reverse accumulation. This development drew on numerical analysis and optimization as well as AI, rather than originating as a single invention inside modern deep learning. The automatic differentiation survey explains these connections.
Derivatives, Integration, and the Structures That Emerge
The same small collection of ideas supports many applications: local linear approximation, composition, accumulation, and controlled limiting processes. Learn to recognize which operation a problem requires before selecting a formula. Predicting how a parameter change affects an output is a differentiation question; totaling a varying contribution is an integration question; recovering a trajectory from its rate law is a differential-equation question.
Derivatives and the Calculus of Composed Functions
A derivative is a local linear approximation. For a scalar function of several variables, f(x + h) ≈ f(x) + ∇f(x)·h when the displacement is small and the function is differentiable. The gradient represents sensitivity in the chosen Euclidean coordinates. For a vector-valued function, the Jacobian is the corresponding linear map. Checking input and output dimensions often catches errors before any calculation begins.
Composition explains the chain rule. For f(w) = (wx - y)^2, hold the data x, y fixed: the outer square contributes 2(wx - y), and the inner expression contributes x, giving 2x(wx - y). In a computational graph, contributions from multiple paths must be added. Reverse accumulation is efficient when a scalar objective depends on many parameters, but storing or recomputing intermediate values has a cost.
Automatic differentiation evaluates these derivative rules without choosing the finite-difference step used in (f(x+h)-f(x))/h. It still incurs floating-point error and follows the operations and control flow of the implemented computation. At nondifferentiable points, library conventions may supply a selected value rather than a classical derivative. A correct derivative of the code does not establish that the code models the intended phenomenon.
Gradient descent takes a step opposite the gradient. This is the direction of greatest first-order decrease under a small fixed Euclidean step-length constraint; an unconstrained linear approximation generally has no minimum. The step size and curvature matter. Newton’s method instead solves a system involving the Hessian to choose a step; outside suitable conditions, that step need not decrease the objective. Derivatives supply local information, not a universal guarantee of finding a global optimum.
Integrals and Accumulation
Integration combines contributions over a domain. Integrating velocity over time gives displacement; integrating a probability density over an interval gives its probability. A density must be nonnegative and integrate to one, but its value at a point need not be at most one. For continuous distributions, probabilities concern regions, not the density value alone.
Numerical quadrature approximates an integral with weighted evaluations. Monte Carlo integration instead uses sampling: under an appropriate sampling distribution and weighting, the estimate is an average with a probabilistic error. For independent samples with finite variance, its standard error falls as 1/√n. This avoids a simple exponential dependence on dimension in the convergence exponent, but the variance and cost of obtaining useful samples can still make the problem difficult.
Rendering illustrates the distinction between the model and the approximation. Outgoing light includes emission and reflected contributions integrated over directions. A renderer samples those contributions; more samples reduce sampling noise under the estimator’s assumptions, but do not repair a wrong material model or systematic bias in the implementation.
Differential Equations and the Dynamics of Continuous Systems
A differential equation specifies how a state changes rather than listing the state at every time. With an initial condition, a suitable equation can determine a trajectory. Existence and uniqueness require conditions on the equation; neither follows merely from writing a derivative symbol.
Discretizing time produces an algorithm whose behavior must be checked separately. For y′ = -1000y, the exact solution decays. Explicit Euler gives y[k+1] = (1 - 1000h)y[k]; its iterates decay only when 0 < h < 0.002. A step that seems modest can therefore make the numerical solution oscillate and grow while the exact solution shrinks.
This example illustrates the stability restriction behind stiffness: rapidly decaying components can force an explicit method to take very small steps even when the solution of interest changes slowly. Stiffness is not simply exponential growth of perturbations. Implicit methods can improve stability but require solving equations at each step. Simulation thus joins analysis, algorithms, and linear algebra; choosing the equation is only part of choosing the computation.
Analysis: When the Intuition Breaks Down
Analysis asks which limiting operations are justified and what their conclusions actually guarantee. Pointwise convergence, uniform convergence, and convergence in probability answer different questions. Interchanging a limit with an integral or derivative requires hypotheses; a graph that looks convergent is not enough. Measure theory and functional analysis extend these tools when elementary definitions no longer suffice.
For a beginner, the most useful discipline is to attach conditions and a scale to each claim. Does an error bound hold after a finite number of iterations? Does it depend on smoothness, dimension, or a condition number? Is convergence guaranteed from any starting point or only nearby? These questions explain why a correct mathematical theorem can coexist with a disappointing computation, without requiring every learner to complete advanced analysis first.
What Studying This Changes
Calculus lets you read an optimization or simulation as a mathematical construction. Instead of treating a learning rate as an unexplained knob, connect it to local sensitivity and curvature. Instead of treating a numerical trajectory as ground truth, compare step sizes, error estimates, and invariants such as conserved quantities where appropriate.
Try deriving a small gradient by hand, computing it by automatic differentiation, and checking it with finite differences away from nondifferentiable points. Then vary the finite-difference step to observe truncation and roundoff effects. The disagreement between methods is an opportunity to identify their assumptions, not merely a reason to trust whichever tool is more sophisticated.
Resources
Books and Texts
For geometric intuition before formal study, 3Blue1Brown’s Essence of Calculus (YouTube, free, 11 episodes) builds understanding of derivatives, integrals, and the fundamental theorem geometrically and visually. It does not teach computation but provides the right conceptual framing to make computation meaningful rather than mechanical. Most learners should watch this series before or alongside a formal calculus course.
For the standard foundational treatment, Stewart’s Calculus: Early Transcendentals (9th ed., 2020) is the comprehensive reference that most university calculus courses use. It is thorough, example-rich, and reliable. Most learners do not need to work through all of it: for CS purposes, the critical content is derivatives, the chain rule, partial derivatives, gradients, and enough integration to understand probability densities and expected values. The heavy coverage of integration techniques (trigonometric substitution, partial fractions) is not a CS priority. MIT OpenCourseWare 18.01 and 18.02 provide free alternatives to Stewart for single-variable and multivariable calculus respectively, with complete problem sets and exams.
For multivariable calculus with mathematical care, Hubbard and Hubbard’s Vector Calculus, Linear Algebra, and Differential Forms (5th ed.) develops both subjects simultaneously in a unified geometric framework. It is more demanding than Stewart but produces deeper understanding of the relationships between linear algebra and calculus — particularly relevant for understanding the Jacobian and how gradients transform. Marsden and Tromba’s Vector Calculus is a more conventional alternative at similar depth.
For the ML-specific calculus content — matrix calculus, Jacobians, the relationship between the chain rule and backpropagation — Deisenroth, Faisal, and Ong’s Mathematics for Machine Learning Chapter 5 (free at mml-book.com) is the most direct treatment available. Most calculus courses do not develop matrix calculus; this chapter does. It is the right companion to Strang’s linear algebra for ML practitioners.
For rigorous foundations, Spivak’s Calculus (4th ed.) develops single-variable calculus from the real number axioms through the Riemann integral with unusual clarity and mathematical seriousness. It is demanding — closer to a first analysis course than a standard calculus text — but it is the right choice for learners who want to understand why the theorems hold, not just how to use them. Abbott’s Understanding Analysis (2nd ed.) is more accessible and covers similar analysis content with more pedagogical care; it is the recommended starting point for learners approaching real analysis for the first time. Tao’s Analysis I and Analysis II (free online) develop analysis from first principles with exceptional precision.
For differential equations, Strogatz’s Nonlinear Dynamics and Chaos (2nd ed., 2018) is the most accessible rigorous treatment of ODEs from a dynamical systems perspective — it emphasizes geometric thinking (phase portraits, stability, bifurcations) over solution techniques, which is what CS practitioners most need. Specialized ODE and PDE references belong later, in scientific computing, simulation, graphics, robotics, or domain-specific study rather than in this foundation section.
For automatic differentiation specifically, Karpathy’s micrograd (GitHub, free, ~150 lines of Python) implements scalar reverse-mode autodiff from scratch. Working through it makes the chain rule concrete in code. Baydin et al.’s “Automatic Differentiation in Machine Learning: a Survey” (2018, free) covers the full theory including forward mode, reverse mode, and higher-order differentiation.
For numerical methods — the computational implementation of calculus — Trefethen and Bau’s Numerical Linear Algebra (§2.3 reference) covers the linear algebra side; the numerical ODE and PDE content is treated in §6.5 Scientific Computing and Simulation.
| Book | Role | Type |
|---|---|---|
| Stewart, Calculus: Early Transcendentals (9th ed.) | Standard comprehensive reference | Entry |
| MIT OCW 18.01 + 18.02 (free) | Free alternative with complete problem sets | Entry |
| Hubbard & Hubbard, Vector Calculus, Linear Algebra, and Differential Forms | Unified multivariable treatment | Depth |
| Marsden & Tromba, Vector Calculus | Multivariable calculus at depth | Depth |
| Deisenroth, Faisal & Ong, Mathematics for Machine Learning Ch. 5 (free) | Matrix calculus; chain rule for ML | Entry |
| Spivak, Calculus (4th ed.) | Rigorous single-variable with proofs | Depth |
| Abbott, Understanding Analysis (2nd ed.) | Accessible real analysis entry | Entry |
| Tao, Analysis I and Analysis II (free online) | Real analysis from first principles | Depth |
| Strogatz, Nonlinear Dynamics and Chaos (2nd ed.) | ODEs; dynamical systems; geometric view | Depth |
| Rudin, Principles of Mathematical Analysis | Classical real analysis reference | Auxiliary |
Courses and Lectures
MIT 18.01 (Single Variable Calculus, free on MIT OCW) and 18.02 (Multivariable Calculus) are the standard free courses with complete problem sets, exams, and lecture videos. Several instructors have taught 18.02 on OCW; Denis Auroux’s version is particularly well-regarded.
3Blue1Brown’s Essence of Calculus (YouTube, free) — already mentioned in Books — functions as both a reference and a companion. Return to specific episodes when the geometric intuition behind a concept is unclear.
MIT 18.06 (Strang’s Linear Algebra, §2.3) should be studied alongside or before 18.02 for the connections between multivariable calculus and linear algebra — gradients are linear maps, the Hessian is a symmetric bilinear form — to be visible.
Karpathy’s Neural Networks: Zero to Hero (YouTube, free) builds a complete language model from scratch, implementing automatic differentiation, neural network layers, and training loops step by step. This is not a calculus course, but it makes the chain rule and backpropagation concrete in a way that abstract treatment cannot.
| Course | Platform | Type |
|---|---|---|
| MIT 18.01 Single Variable Calculus (free) | MIT OCW | Entry |
| MIT 18.02 Multivariable Calculus (free) | MIT OCW | Entry |
| Karpathy, Neural Networks: Zero to Hero (free) | YouTube / GitHub | Practice |
Practice, Tools, and Code
3Blue1Brown (mentioned above) is the primary visualization resource. The episodes on derivatives, the chain rule, integrals, Taylor series, and the essence of calculus are the best geometric introductions to each concept.
Paul’s Online Math Notes (free) is a reliable procedural reference for Calculus I-III and differential equations. It is not CS-specific and not especially conceptual, but it is useful when a learner needs a clear worked example, a formula refresher, or extra practice on a technique before returning to the CS-facing material.
Implementing automatic differentiation from scratch is the most important calculus-related coding project for ML practitioners. Karpathy’s micrograd (~150 lines) implements scalar autodiff; extending it to support tensors (as in tinygrad) reveals how production frameworks work. After implementing autodiff, implement the backward pass of a two-layer neural network manually in NumPy, comparing with PyTorch’s computed gradient to verify correctness.
SymPy (Python) and Wolfram Alpha (web) compute symbolic derivatives and integrals, which is useful for checking manual calculations. More importantly, SymPy can be used to explore how the chain rule decomposes complex expressions — running it on neural network loss functions and inspecting the symbolic gradient teaches matrix calculus more concretely than reading about it.
| Resource | Platform | Type |
|---|---|---|
| 3Blue1Brown, Essence of Calculus (free) | YouTube | Practice |
| Paul’s Online Math Notes (free) | tutorial.math.lamar.edu | Reference |
| Karpathy, micrograd (free) | GitHub | Practice |
| SymPy (free) | Python / sympy.org | Practice |
| Wolfram Alpha (free tier) | wolframalpha.com | Practice |
Traps
| Trap | Why it misleads | Better response |
|---|---|---|
| Memorizing derivatives without local approximation | Symbol manipulation does not explain sensitivity. | Predict how a small input change affects the output, then check the prediction. |
| Treating the gradient as a global solution | It describes first-order behavior near one point. | Check step size, curvature, constraints, and the conditions of any convergence claim. |
| Calling automatic differentiation exact without qualification | Floating-point arithmetic, branches, and nondifferentiable operations still matter. | Identify the function implemented and compare derivative methods on a small example. |
| Equating smaller steps with unlimited accuracy | More steps can increase cost and accumulated roundoff. | Examine both approximation error and computational error. |
| Confusing stiffness with an unstable physical system | A stable continuous system can impose severe numerical stability constraints. | Compare the exact and discrete dynamics, as in the Euler decay example. |
| Making advanced analysis a prerequisite for all computing | Different tasks require different depths of continuous mathematics. | Learn enough to state the assumptions of the method in use, then deepen the theory as needed. |
中文
微积分研究变化与累积；数学分析则使其中的极限论证变得精确。当离散机器模拟时间、位置、概率密度或可调参数等连续量时，它们在计算机科学中的作用尤其明显。导数描述局部敏感性，积分汇总一个区域内的贡献。在微积分基本定理的适用条件下，两者互相联系。
从数学走向计算，还会产生第二个问题：有限的计算在多大程度上近似了数学对象？一个优化定理可能讨论精确实数运算，实现却使用舍入后的数值，并在有限步后停止。学习这一领域，需要把模型、近似与实现联系起来。
从函数、代数与图像开始。单变量微积分提供最初直觉，线性代数随后帮助理解多变量导数。当具体问题需要时，再深入分析。一般编程、离散算法和初等概率，不必等这条学习路线全部完成才开始。
从无穷小量到自动微分
微积分源于几个相互关联的问题：曲线的切线、面积、运动速率，以及变化量与其累积效果之间的关系。Newton 和 Leibniz 在前人工作的基础上，发展出影响深远的十七世纪表述。他们的重要贡献，是把能跨问题使用的方法连接起来并加以系统化；此后的历史也包括澄清这些方法为什么有效、在哪里失效。
十九世纪的数学分析使极限、连续性与收敛的含义更加精确。这具有实际后果：一列不断细化的近似，不一定按直觉预期趋近所需答案；一个收敛的无限过程，也可能慢得无法使用。极限结论与有效误差界之间的区别，至今仍是数值计算的核心。
数值方法早于电子计算机。更快的机器改变了模拟规模，也放大了累积舍入、稳定性与算法选择的重要性。直接把手算方法搬上机器，可能只是以更高速度重复其弱点。可靠的数值软件因此形成了自身的研究与实践。
自动微分建立了另一种联系：由可微运算构成的程序，也可以携带对其组合求导所需的信息。前向和反向累积系统地应用链式法则；神经网络反向传播是反向累积的一个重要用途。这一发展同时来自数值分析、优化和 AI，并不是现代深度学习内部某一次独立发明的产物。自动微分综述解释了这些联系。
导数、积分与由此出现的结构
许多应用由同一小组观念支撑：局部线性近似、组合、累积，以及受到控制的极限过程。选择公式前，先判断问题需要哪种操作。预测参数变化怎样影响输出，是求导问题；汇总不断变化的贡献，是积分问题；从变化率规律恢复轨迹，则是微分方程问题。
导数与复合函数的微积分
导数是一种局部线性近似。对于多变量标量函数，在位移很小且函数可微时，f(x + h) ≈ f(x) + ∇f(x)·h。梯度在所选欧氏坐标下表示敏感性。对于向量值函数，Jacobian 矩阵表示相应的线性映射。检查输入输出维度，往往能在计算开始前发现错误。
函数组合解释了链式法则。以 f(w) = (wx - y)^2 为例，把数据 x, y 固定：外层平方贡献 2(wx - y)，内层表达式贡献 x，相乘得到 2x(wx - y)。在计算图中，多条路径的贡献必须相加。当一个标量目标依赖许多参数时，反向累积很高效，但存储或重算中间值也有成本。
自动微分计算这些导数规则，无需像有限差分 (f(x+h)-f(x))/h 那样选择步长。它仍有浮点误差，并且遵循实现中的运算和控制流。在不可微点，库的约定可能提供一个选定值，而非经典意义上的导数。正确求出了代码的导数，并不意味着代码正确描述了目标现象。
梯度下降沿梯度反方向迈步。在欧氏步长固定且很小的约束下，这是使一阶近似下降最多的方向；没有约束的线性近似一般没有最小值。步长和曲率都很重要。Newton 方法则求解包含 Hessian 的方程组来选择步子；缺少适当条件时，这一步未必使目标下降。导数提供局部信息，并不普遍保证找到全局最优解。
积分与累积
积分汇总一个区域内的贡献。对速度按时间积分得到位移；对概率密度在一个区间内积分得到该区间的概率。密度必须非负，且总积分为一，但它在某一点的值不必小于等于一。对于连续分布，概率描述区域，不能直接把密度值当成概率。
数值求积用加权的函数值近似积分。Monte Carlo 积分则使用采样：在合适的采样分布和权重下，估计值是一个具有概率误差的平均数。对独立且方差有限的样本，标准误差按 1/√n 下降。其收敛指数没有直接随维度呈指数恶化，但方差大小与获取有效样本的成本，仍可能使问题很困难。
渲染展示了模型与近似的区别。出射光包含自发光，以及对各方向反射贡献的积分。渲染器对这些贡献采样；在估计方法的前提成立时，增加样本可以减少采样噪声，但无法修正错误的材质模型或实现中的系统性偏差。
微分方程与连续系统的动力学
微分方程规定状态怎样变化，而不必列出每个时刻的状态。给定初始条件，适当的方程可以确定一条轨迹。解的存在性与唯一性依赖方程满足的条件，并不会因为写下导数符号就自动成立。
把时间离散化后，会得到一个必须单独检查其行为的算法。对 y′ = -1000y，精确解不断衰减。显式 Euler 方法给出 y[k+1] = (1 - 1000h)y[k]；只有在 0 < h < 0.002 时，迭代结果才会衰减。因此，一个看起来并不大的步长，也可能使数值解振荡增长，而精确解却在缩小。
这个例子展示了刚性背后的稳定性限制：快速衰减的分量，可能迫使显式方法使用极小步长，即使我们关心的解变化很慢。刚性并不等于扰动指数增长。隐式方法可以改善稳定性，但每一步都需要解方程。因此，模拟把分析、算法与线性代数连接起来；选择方程，只是选择计算方式的一部分。
分析：当直觉失效时
分析追问哪些极限操作有根据，以及结论实际保证了什么。逐点收敛、一致收敛和依概率收敛回答不同问题。交换极限与积分或求导的次序需要前提；图像看起来在收敛，并不足以支持这种交换。当初等定义不够用时，测度论与泛函分析会扩展这些工具。
对初学者，最有用的习惯是为每个结论补上条件和尺度。误差界是否适用于有限次迭代？它是否依赖光滑性、维数或条件数？收敛是否从任意起点都得到保证，还是只在附近成立？这些问题能解释为什么正确的数学定理可以与不理想的计算结果同时存在，而不必要求每位学习者先学完高等分析。
学习这一部分会改变什么
微积分让你能够把优化或模拟读成一种数学构造。学习率不再只是含义不明的旋钮，而是与局部敏感性和曲率相关。数值轨迹也不再自动等于真实结果：应比较步长、误差估计，并在适用时检查守恒量等不变性质。
可以手算一个小函数的梯度，再用自动微分计算，并在远离不可微点的位置用有限差分核对。随后改变差分步长，观察截断误差与舍入误差。方法之间的差异，是识别各自前提的机会，而不只是选择相信更复杂工具的理由。
资源
书籍与文本
对于正式学习前的几何直觉，3Blue1Brown 的 Essence of Calculus（YouTube，免费，共 11 集）会用几何和视觉方式建立对导数、积分和微积分基本定理的理解。它不教授计算技巧，但会提供正确的概念框架，使后续计算变得有意义，而不是机械操作。多数学习者都应该在正式微积分课程之前或同时观看这个系列。
对于标准基础处理，Stewart 的 Calculus: Early Transcendentals（第 9 版，2020）是多数大学微积分课程使用的综合参考书。它内容完整、例子丰富、可靠。多数学习者不需要从头到尾学完；就 CS 目的而言，关键内容是导数、链式法则、偏导数、梯度，以及足够理解概率密度和期望值的积分。它对积分技巧的大量覆盖（三角代换、部分分式等）并不是 CS 优先事项。MIT OpenCourseWare 的 18.01 和 18.02 分别提供单变量和多变量微积分的免费替代方案，并带有完整习题和考试。
对于具有数学谨慎性的多变量微积分，Hubbard 和 Hubbard 的 Vector Calculus, Linear Algebra, and Differential Forms（第 5 版）会在一个统一的几何框架中同时发展向量微积分和线性代数。它比 Stewart 要求更高，但能更深入理解线性代数与微积分之间的关系——尤其适合理解 Jacobian，以及梯度如何变换。Marsden 和 Tromba 的 Vector Calculus 是同等深度下更传统的替代选择。
对于机器学习所需的微积分内容——矩阵微积分、Jacobian、链式法则与反向传播之间的关系——Deisenroth、Faisal 和 Ong 的 Mathematics for Machine Learning 第 5 章（可在 mml-book.com 免费获取）是目前最直接的处理。多数微积分课程并不发展矩阵微积分；这一章会做。对 ML 实践者来说，它是 Strang 线性代数的正确伴读材料。
对于严谨基础，Spivak 的 Calculus（第 4 版）从实数公理一直发展到 Riemann 积分，以异常清晰和严肃的数学态度处理单变量微积分。它要求很高——更接近第一门分析课程，而不是标准微积分教材——但对于想理解定理为什么成立，而不只是如何使用定理的学习者，它是正确选择。Abbott 的 Understanding Analysis（第 2 版）更易进入，并以更强的教学意识覆盖类似的分析内容；对于第一次接触实分析的学习者，它是推荐起点。Tao 的 Analysis I 和 Analysis II（免费在线）则以极高精确性从第一原则发展分析。
对于微分方程，Strogatz 的 Nonlinear Dynamics and Chaos（第 2 版，2018）是从动力系统视角进入 ODE 的最易读且严谨的处理。它强调几何思考（相图、稳定性、分岔），而不是解法技巧，而这正是 CS 实践者最需要的。专门的 ODE 和 PDE 参考书应放到后续科学计算、仿真、图形学、机器人学或领域特定学习中，而不是放在这个基础章节中。
对于自动微分本身，Karpathy 的 micrograd（GitHub，免费，约 150 行 Python）从零实现了标量反向模式自动微分。完整学一遍它，会让链式法则在代码中具体化。Baydin 等人的 “Automatic Differentiation in Machine Learning: a Survey”（2018，免费）覆盖完整理论，包括前向模式、反向模式和高阶微分。
对于数值方法——微积分的计算实现——Trefethen 和 Bau 的 Numerical Linear Algebra（§2.3 参考）覆盖线性代数部分；数值 ODE 和 PDE 内容会在 §6.5 Scientific Computing and Simulation 中处理。
| 书籍 | 作用 | 类型 |
|---|---|---|
| Stewart, Calculus: Early Transcendentals（第 9 版） | 标准综合参考 | 入门 |
| MIT OCW 18.01 + 18.02（免费） | 带完整习题的免费替代方案 | 入门 |
| Hubbard & Hubbard, Vector Calculus, Linear Algebra, and Differential Forms | 统一的多变量处理 | 深入 |
| Marsden & Tromba, Vector Calculus | 深入多变量微积分 | 深入 |
| Deisenroth, Faisal & Ong, Mathematics for Machine Learning Ch. 5（免费） | 矩阵微积分；面向 ML 的链式法则 | 入门 |
| Spivak, Calculus（第 4 版） | 带证明的严谨单变量微积分 | 深入 |
| Abbott, Understanding Analysis（第 2 版） | 易进入的实分析入口 | 入门 |
| Tao, Analysis I and Analysis II（免费在线） | 从第一原则展开实分析 | 深入 |
| Strogatz, Nonlinear Dynamics and Chaos（第 2 版） | ODE；动力系统；几何视角 | 深入 |
| Rudin, Principles of Mathematical Analysis | 经典实分析参考 | 辅助 |
课程与讲座
MIT 18.01（Single Variable Calculus，可在 MIT OCW 免费获取）和 18.02（Multivariable Calculus）是标准免费课程，带完整习题、考试和讲课视频。OCW 上有多位教师讲授 18.02 的版本；Denis Auroux 的版本尤其受到好评。
3Blue1Brown 的 Essence of Calculus（YouTube，免费）——前面在书籍部分已经提到——既可作为参考，也可作为伴随材料。当某个概念背后的几何直觉不清楚时，可以回到具体集数。
MIT 18.06（Strang 的 Linear Algebra，§2.3）应与 18.02 同时或先于 18.02 学习，这样多变量微积分与线性代数之间的连接才会显现出来——梯度是线性映射，Hessian 是对称双线性形式。
Karpathy 的 Neural Networks: Zero to Hero（YouTube，免费）从零构建完整语言模型，逐步实现自动微分、神经网络层和训练循环。它不是微积分课程，但它以抽象处理无法做到的方式，让链式法则和反向传播变得具体。
| 课程 | 平台 | 类型 |
|---|---|---|
| MIT 18.01 Single Variable Calculus（免费） | MIT OCW | 入门 |
| MIT 18.02 Multivariable Calculus（免费） | MIT OCW | 入门 |
| Karpathy, Neural Networks: Zero to Hero（免费） | YouTube / GitHub | 实践 |
实践、工具与代码
3Blue1Brown（上文已提到）是主要的可视化资源。关于导数、链式法则、积分、Taylor 级数和微积分本质的几集，是这些概念最好的几何入门。
Paul’s Online Math Notes（免费）是 Calculus I–III 和微分方程的可靠程序性参考。它并不面向 CS，也不特别强调概念，但当学习者需要一个清楚的完整例题、公式回顾，或在回到面向 CS 的材料之前额外练习某个技巧时，它很有用。
从零实现自动微分，是 ML 实践者最重要的微积分相关编码项目。Karpathy 的 micrograd（约 150 行）实现了标量自动微分；把它扩展到支持张量（类似 tinygrad）会揭示生产级框架如何工作。在实现自动微分之后，再用 NumPy 手动实现一个两层神经网络的 backward pass，并与 PyTorch 计算出的梯度比较，以验证正确性。
SymPy（Python）和 Wolfram Alpha（网页）可以计算符号导数和积分，适合检查手算结果。更重要的是，SymPy 可以用来探索链式法则如何分解复杂表达式——把它应用到神经网络损失函数上，并检查符号梯度，会比单纯阅读更具体地教会矩阵微积分。
| 资源 | 平台 | 类型 |
|---|---|---|
| 3Blue1Brown, Essence of Calculus（免费） | YouTube | 实践 |
| Paul’s Online Math Notes（免费） | tutorial.math.lamar.edu | 参考 |
| Karpathy, micrograd（免费） | GitHub | 实践 |
| SymPy（免费） | Python / sympy.org | 实践 |
| Wolfram Alpha（免费层） | wolframalpha.com | 实践 |
陷阱
| 陷阱 | 为什么会误导 | 更好的做法 |
|---|---|---|
| 只记导数公式，不理解局部近似 | 符号操作本身不能解释敏感性。 | 预测输入小变化如何影响输出，再核对预测。 |
| 把梯度当成全局解法 | 它描述某一点附近的一阶行为。 | 检查步长、曲率、约束与收敛结论的条件。 |
| 不加限定地说自动微分精确 | 浮点运算、分支与不可微操作仍然重要。 | 识别实际实现的函数，用小例子比较求导方法。 |
| 认为步长越小就能无限提高精度 | 更多步骤可能增加成本和累积舍入。 | 同时考察近似误差与计算误差。 |
| 把刚性混同于物理系统不稳定 | 稳定的连续系统也可能施加苛刻的数值稳定性约束。 | 像 Euler 衰减例子那样，比较精确动力学与离散动力学。 |
| 把高等分析设为全部计算学习的前提 | 不同任务需要不同深度的连续数学。 | 先学到能说清所用方法的前提，再按需要深入理论。 |