English
Calculus is the mathematics of continuous change. Where algebra deals with static relationships between quantities, calculus deals with how quantities vary — how fast a surface cools, how quickly a population grows, how a physical system evolves in time. The two founding operations are the derivative, which measures the instantaneous rate of change of a function, and the integral, which accumulates infinitely many infinitely small quantities into a finite sum. The fundamental theorem of calculus says these are inverses of each other: to find how much something has accumulated, find a function whose rate of change is the accumulation rate, and evaluate it at the endpoints. This single insight, together with the systematic rules for computing derivatives and integrals, became the working language of physics, engineering, and economics for three centuries.
For computer science, the relevant parts of calculus are more selective than this history might suggest. A systems programmer or database engineer may go years without needing it. But where it is needed, the dependence is deep. The central operation of machine learning — training by gradient descent — is an application of multivariable calculus. Every parameter update requires computing how the loss changes as each parameter changes, propagating that computation through potentially hundreds of composed functions via the chain rule. Computer graphics simulates light transport through integrals and physical dynamics through differential equations. Scientific computing exists primarily to solve differential equations that have no closed-form solution. Robotics uses control theory, which is calculus applied to dynamical systems. The subject is not uniformly required, but for the subfields that require it, it is irreplaceable.
Mathematical analysis is calculus made rigorous. The rules for derivatives and integrals that Newton and Leibniz developed are correct, but it took two centuries to put them on a logically sound foundation — to say precisely what a limit is, when a function can be differentiated, which functions can be integrated. Real analysis is that foundation, and it matters for CS in places where informal reasoning about approximation breaks down: in machine learning theory, in numerical analysis, in the functional analysis that underlies quantum computing.
Prerequisites: Linear Algebra (§2.3) is independent. Probability (§2.5) is independent. Calculus is prerequisite to Chapter 5 (Artificial Intelligence) and several applied sections.
From Infinitesimals to Automatic Differentiation
The problem calculus solves is ancient. Archimedes, around 250 BCE, computed the area under a parabola by exhaustion: divide the region into thin slices, sum their areas, and show that the sum approaches the exact area as the slices become infinitely thin. He had the idea of integration without the general machinery. Kepler, computing the area swept by a planet in its elliptical orbit to verify his second law in 1609, used similar approximations. But neither had a systematic method that worked for arbitrary functions.
The systematic method arrived twice, independently and nearly simultaneously. Isaac Newton developed what he called the “method of fluxions” between 1665 and 1666, working in isolation in Lincolnshire while Cambridge was closed by plague — the most productive two years in the history of mathematics. He used his method to derive Kepler’s laws from his law of gravitation, showing that the same mathematics that described falling apples described planetary orbits. Gottfried Wilhelm Leibniz developed his differential calculus independently between 1673 and 1676 and published it in 1684, introducing the notation — dy/dx for the derivative, ∫ for the integral — that mathematicians still use. The bitter priority dispute that followed divided European and British mathematics for a generation, with British mathematicians loyally using Newton’s inferior notation while Continental mathematicians advanced on Leibniz’s.
What neither man could explain rigorously was what an “infinitesimal” actually was. Their calculations proceeded by treating infinitely small quantities dx as nonzero when dividing by them and zero when they were inconveniently large — a procedure that worked magnificently in practice and made no logical sense. The philosopher George Berkeley mocked this in 1734: “And what are these fluxions? The velocities of evanescent increments. And what are these evanescent increments? They are neither finite quantities, nor quantities infinitely small, nor yet nothing. May we not call them ghosts of departed quantities?” Berkeley’s critique was mathematically correct. The rules worked; the justification did not exist.
The rigorous foundation came a century later. Augustin-Louis Cauchy developed the limit in the 1820s, giving a precise meaning to “approaching” a value without the problematic infinitesimals. Karl Weierstrass completed the project in the 1850s and 1860s with the epsilon-delta definition — the version taught in analysis courses today — which made limits, continuity, and convergence into precise mathematical concepts with unambiguous truth conditions. Bernhard Riemann defined the integral in 1854 as the limit of sums of rectangles, specifying exactly which functions could be integrated. Henri Lebesgue extended this in 1902 to a more powerful integral that handles functions Riemann’s definition cannot. These developments transformed calculus from an immensely powerful heuristic into mathematics in the rigorous sense.
The rigorization had a paradoxical consequence: it revealed how many things could go wrong. Functions that seemed intuitively continuous could fail to be differentiable; series that seemed intuitively convergent could diverge; limits that seemed to commute could fail to commute. Mathematical analysis is the discipline of understanding these subtleties and proving theorems about them carefully. For most engineering applications, the subtleties do not arise and informal reasoning is adequate. For machine learning theory — particularly the convergence analysis of optimization algorithms and the generalization theory of overparameterized models — the subtleties matter, and the rigor of analysis is needed to reason correctly about them.
The intersection of calculus with computer science began seriously in the 1940s, when the first computers were used to solve differential equations arising in ballistics, fluid dynamics, and nuclear physics. Finite difference methods — approximating derivatives by ratios of differences and integrating by summing small steps — were the primary tool. The field of numerical analysis developed to understand when these approximations were accurate and when they catastrophically diverged. The key concept, again, is conditioning: some differential equations are stiff, meaning that tiny perturbations grow exponentially, making numerical methods unstable unless they are carefully designed. Understanding stiffness requires understanding the structure of the differential equation, which requires analysis.
Automatic differentiation emerged from a different need. By the 1960s, programmers were computing derivatives of complex functions by hand and encoding them explicitly — a tedious process prone to errors. Seppo Linnainmaa’s 1970 master’s thesis showed that the chain rule could be applied mechanically to any computation expressed as a composition of elementary operations, computing exact derivatives as a byproduct of the forward computation without symbolic manipulation and without finite-difference approximation. This is automatic differentiation. The insight had been discovered earlier and was rediscovered several times, but Linnainmaa’s work established it clearly. The reverse mode of automatic differentiation — which computes the gradient of a scalar-valued function with respect to all its inputs in one backward pass — is what backpropagation implements. When Rumelhart, Hinton, and Williams described backpropagation in their 1986 Nature paper, they were applying reverse-mode automatic differentiation to neural networks. When you call .backward() in PyTorch, you are running Linnainmaa’s algorithm.
The development of deep learning in the 2010s made calculus a central professional tool for a much larger population of practitioners than any previous application. The chain rule, applied through hundreds of layers to tensors with millions of entries, is the mathematical operation that makes it possible to train large neural networks. The gradient descent algorithm, analyzed and modified through decades of optimization theory, is the reason those networks learn anything. Understanding what backpropagation is computing, when gradients vanish or explode, and why certain architectures are easier to train than others requires calculus — not the ability to compute indefinite integrals by hand, but the conceptual understanding of derivatives, the chain rule, and how they behave when composed.
Derivatives, Integration, and the Structures That Emerge
Calculus for CS practitioners organizes around three clusters of ideas, each appearing in multiple downstream contexts.
Derivatives and the Calculus of Composed Functions
The derivative of a function f at a point x measures the instantaneous rate of change: how much f(x) changes per unit change in x, at that specific point. Geometrically, it is the slope of the tangent line. Computationally, it is the limit of the difference quotient (f(x+h) - f(x))/h as h approaches zero. The derivative is itself a function, assigning to each x the slope at that point — and this function can in turn be differentiated, giving the second derivative, which measures how the rate of change itself changes.
The chain rule computes the derivative of a composition. If z = f(y) and y = g(x), then dz/dx = (dz/dy)(dy/dx). This one rule makes calculus applicable to complex systems built from simple parts: the derivative of any composed function is a product of the derivatives of its parts, evaluated at the appropriate points. In a neural network with L layers, the gradient of the loss with respect to the first layer’s weights is a product of L Jacobians, one for each layer, multiplied by the gradient of the loss with respect to the final layer’s output. Backpropagation is the efficient computation of this product by starting from the output and working backward, avoiding redundant computation.
For functions of multiple variables — and every neural network is a function of millions of variables — the derivative becomes a vector: the gradient ∇f = (∂f/∂x₁, …, ∂f/∂xₙ), where each partial derivative ∂f/∂xᵢ measures the rate of change with respect to xᵢ while holding the other variables fixed. The gradient points in the direction of steepest ascent; gradient descent moves opposite to it. For vector-valued functions, the derivative is a matrix — the Jacobian J where J_ij = ∂f_i/∂x_j. Backpropagation computes vector-Jacobian products (not full Jacobians), which is why it scales to networks with millions of parameters.
Taylor’s theorem says that any smooth function near a point can be approximated as a polynomial. The first-order approximation is f(x + δ) ≈ f(x) + ∇f(x)ᵀδ — a linear function. Gradient descent is implicitly making this approximation and taking a step that minimizes it; the step must be small enough that the approximation is valid. The second-order approximation includes the Hessian H = ∂²f/∂xᵢ∂xⱼ, the matrix of second derivatives that describes the curvature of f. Newton’s method uses the second-order approximation, taking the step that minimizes it exactly; it converges much faster than gradient descent in regions of high curvature, at the cost of computing and inverting the Hessian — prohibitive for large neural networks but foundational for understanding why curvature matters in optimization.
Integrals and Accumulation
Integration is the inverse of differentiation and the mathematics of accumulation. The definite integral ∫ₐᵇ f(x) dx accumulates the infinitely many infinitesimal contributions of f over the interval [a, b]; geometrically, it is the signed area under the curve.
Integrals appear in CS in two primary contexts. The first is probability: a probability density function p(x) must satisfy ∫p(x) dx = 1; expected values are E[f(X)] = ∫f(x)p(x) dx; marginal distributions are obtained by integrating out variables. The normalizing constants that make distributions sum to one — the partition function in energy-based models, the marginal likelihood in Bayesian inference — are integrals that are usually intractable in closed form, which is why approximate inference methods exist. Diffusion models are defined through stochastic differential equations, and the forward and reverse processes they implement are described and analyzed using integral calculus.
The second context is physics simulation, primarily in graphics and robotics. The rendering equation — the fundamental equation of physically based rendering, derived by Kajiya in 1986 — describes the light emitted in a direction as an integral over all incoming light weighted by the surface’s reflectance function. Solving this integral by Monte Carlo sampling (choosing random incoming directions and averaging) is path tracing; understanding the variance and convergence of the estimator requires probability and integral calculus. In robotics and control, the state of a system evolves according to differential equations, and predicting where a system will be in the future requires integrating those equations forward in time.
Differential Equations and the Dynamics of Continuous Systems
A differential equation relates a function to its derivatives. The simplest, dy/dt = ky, says the rate of change of y is proportional to y — the equation of exponential growth and decay, describing everything from bacterial populations to radioactive decay to the charging of a capacitor. Newton’s second law, F = ma, is a differential equation: force equals mass times the second derivative of position with respect to time. Systems governed by differential equations are everywhere in the physical world, and simulating them is one of the primary computational tasks in graphics, robotics, and scientific computing.
Ordinary differential equations (ODEs) involve a single independent variable, usually time. Their analysis reveals whether solutions grow, decay, or oscillate, and whether perturbations are amplified or suppressed — the concept of stability that determines whether a control system converges to its setpoint or diverges. Numerical ODE solvers — Euler’s method, Runge-Kutta methods, adaptive-step methods — approximate the solution by taking small time steps, computing the derivative at the current state, and advancing accordingly. The accuracy and stability of these approximations depends on the ODE’s structure, and choosing the right solver for a given problem is an applied analysis problem.
Partial differential equations (PDEs) involve multiple independent variables — functions of both space and time, or multiple spatial dimensions. The heat equation, the wave equation, the Schrödinger equation, and the Navier-Stokes equations for fluid flow are all PDEs. Scientific computing at scale is largely numerical PDE solution: discretizing the equation in space and time, solving the resulting linear system, and advancing the solution. This is both a numerical analysis problem (stability and accuracy of the discretization) and a linear algebra problem (solving potentially enormous sparse linear systems efficiently).
Neural ODEs, introduced by Chen et al. in 2018, brought differential equations directly into deep learning: the hidden state of the network evolves according to an ODE parameterized by a neural network, with the output computed by an ODE solver. This reinterprets the depth of a neural network as a continuous-time parameter rather than a discrete count of layers, and it connects deep learning to the classical theory of dynamical systems. Diffusion models, which define a stochastic forward process that gradually corrupts data and a learned reverse process that denoises it, are built on stochastic differential equations — the same mathematical structure that models Brownian motion and financial derivatives.
Analysis: When the Intuition Breaks Down
Real analysis is the rigorous foundation of calculus, and it matters for CS in the places where intuitive reasoning about limits and convergence goes wrong.
The convergence of optimization algorithms is an analysis problem. Proving that stochastic gradient descent converges, characterizing the rate at which it does so, and understanding how the learning rate affects convergence requires precise reasoning about sequences of random variables and their limits. The standard convergence results for gradient descent — O(1/t) convergence for convex functions with Lipschitz-continuous gradients, faster for strongly convex functions — are proved using analysis. Understanding these results, rather than just using them, requires the epsilon-delta machinery.
Generalization theory is an analysis problem. The VC dimension, Rademacher complexity, and PAC-Bayes bounds that characterize when learning algorithms generalize from training data to new data are stated and proved using measure theory and functional analysis — the extension of analysis to infinite-dimensional function spaces. The double-descent phenomenon, where overparameterized models continue to improve beyond the interpolation threshold, is an active analysis research area.
Functional analysis, the study of infinite-dimensional vector spaces (function spaces), underlies several CS applications. Kernel methods — SVMs, Gaussian processes — operate in infinite-dimensional reproducing kernel Hilbert spaces; the representer theorem that makes them computationally tractable is a functional analysis result. Quantum computing is linear algebra over a Hilbert space; the foundations are functional analysis. The mathematics of continuous normalizing flows and score-based generative models involves differential geometry on function spaces.
What Studying This Changes
Calculus and analysis change what a practitioner can understand, not just what they can compute.
The most direct change is access to gradient-based learning. Gradient descent, Adam, learning rate schedules, gradient clipping, batch normalization’s effect on optimization — these are all behaviors of an optimizer moving through a loss landscape defined by calculus. A practitioner who has internalized gradients, the chain rule, and the second-order structure of the Hessian can ask: is this optimization problem well-conditioned? Are the gradients likely to vanish in early layers? What is the effective curvature this optimizer is navigating? These questions have answers rooted in calculus, and a practitioner without the foundation cannot ask them.
The second change is access to probabilistic models. Expected values, marginal distributions, KL divergence, entropy — the mathematical quantities that define what probabilistic models are doing — are all defined through integration. The ELBO in variational autoencoders, the score function estimator in policy gradient methods, the forward and reverse KL divergence as different training objectives — all require comfort with integrals to understand. A practitioner who has only computed gradients, not integrals, encounters a wall when probabilistic models are the subject.
The third change is the ability to understand continuous simulation. Neural ODEs, diffusion models, physics engines in graphics and robotics — these are all built on differential equations. A practitioner who knows what a differential equation is, what makes a numerical solver stable, and what the phase portrait of a dynamical system reveals can engage with these systems at the level of the mathematics, not just the API.
The fourth change is calibration about approximation. Analysis trains the habit of asking: how good is this approximation, and when does it break down? Taylor’s theorem says the linear approximation is good to first order; the gradient descent step size must be small enough that this is accurate. Numerical integration is accurate when the step size is small relative to the function’s variation; the right step size depends on the integrand’s smoothness. This calibration — quantifying approximation error and knowing when it matters — is acquired from analysis and transfers to every domain where approximation is used.
Resources
Books and Texts
For geometric intuition before formal study, 3Blue1Brown’s Essence of Calculus (YouTube, free, 11 episodes) builds understanding of derivatives, integrals, and the fundamental theorem geometrically and visually. It does not teach computation but provides the right conceptual framing to make computation meaningful rather than mechanical. Most learners should watch this series before or alongside a formal calculus course.
For the standard foundational treatment, Stewart’s Calculus: Early Transcendentals (9th ed., 2020) is the comprehensive reference that most university calculus courses use. It is thorough, example-rich, and reliable. Most learners do not need to work through all of it: for CS purposes, the critical content is derivatives, the chain rule, partial derivatives, gradients, and enough integration to understand probability densities and expected values. The heavy coverage of integration techniques (trigonometric substitution, partial fractions) is not a CS priority. MIT OpenCourseWare 18.01 and 18.02 provide free alternatives to Stewart for single-variable and multivariable calculus respectively, with complete problem sets and exams.
For multivariable calculus with mathematical care, Hubbard and Hubbard’s Vector Calculus, Linear Algebra, and Differential Forms (5th ed.) develops both subjects simultaneously in a unified geometric framework. It is more demanding than Stewart but produces deeper understanding of the relationships between linear algebra and calculus — particularly relevant for understanding the Jacobian and how gradients transform. Marsden and Tromba’s Vector Calculus is a more conventional alternative at similar depth.
For the ML-specific calculus content — matrix calculus, Jacobians, the relationship between the chain rule and backpropagation — Deisenroth, Faisal, and Ong’s Mathematics for Machine Learning Chapter 5 (free at mml-book.com) is the most direct treatment available. Most calculus courses do not develop matrix calculus; this chapter does. It is the right companion to Strang’s linear algebra for ML practitioners.
For rigorous foundations, Spivak’s Calculus (4th ed.) develops single-variable calculus from the real number axioms through the Riemann integral with unusual clarity and mathematical seriousness. It is demanding — closer to a first analysis course than a standard calculus text — but it is the right choice for learners who want to understand why the theorems hold, not just how to use them. Abbott’s Understanding Analysis (2nd ed.) is more accessible and covers similar analysis content with more pedagogical care; it is the recommended starting point for learners approaching real analysis for the first time. Tao’s Analysis I and Analysis II (free online) develop analysis from first principles with exceptional precision.
For differential equations, Strogatz’s Nonlinear Dynamics and Chaos (2nd ed., 2018) is the most accessible rigorous treatment of ODEs from a dynamical systems perspective — it emphasizes geometric thinking (phase portraits, stability, bifurcations) over solution techniques, which is what CS practitioners most need. Specialized ODE and PDE references belong later, in scientific computing, simulation, graphics, robotics, or domain-specific study rather than in this foundation section.
For automatic differentiation specifically, Karpathy’s micrograd (GitHub, free, ~150 lines of Python) implements scalar reverse-mode autodiff from scratch. Working through it makes the chain rule concrete in code. Baydin et al.’s “Automatic Differentiation in Machine Learning: a Survey” (2018, free) covers the full theory including forward mode, reverse mode, and higher-order differentiation.
For numerical methods — the computational implementation of calculus — Trefethen and Bau’s Numerical Linear Algebra (§2.3 reference) covers the linear algebra side; the numerical ODE and PDE content is treated in §6.5 Scientific Computing and Simulation.
| Book | Role | Type |
|---|---|---|
| Stewart, Calculus: Early Transcendentals (9th ed.) | Standard comprehensive reference | Entry |
| MIT OCW 18.01 + 18.02 (free) | Free alternative with complete problem sets | Entry |
| Hubbard & Hubbard, Vector Calculus, Linear Algebra, and Differential Forms | Unified multivariable treatment | Depth |
| Marsden & Tromba, Vector Calculus | Multivariable calculus at depth | Depth |
| Deisenroth, Faisal & Ong, Mathematics for Machine Learning Ch. 5 (free) | Matrix calculus; chain rule for ML | Entry |
| Spivak, Calculus (4th ed.) | Rigorous single-variable with proofs | Depth |
| Abbott, Understanding Analysis (2nd ed.) | Accessible real analysis entry | Entry |
| Tao, Analysis I and Analysis II (free online) | Real analysis from first principles | Depth |
| Strogatz, Nonlinear Dynamics and Chaos (2nd ed.) | ODEs; dynamical systems; geometric view | Depth |
| Rudin, Principles of Mathematical Analysis | Classical real analysis reference | Auxiliary |
Courses and Lectures
MIT 18.01 (Single Variable Calculus, free on MIT OCW) and 18.02 (Multivariable Calculus) are the standard free courses with complete problem sets, exams, and lecture videos. Several instructors have taught 18.02 on OCW; Denis Auroux’s version is particularly well-regarded.
3Blue1Brown’s Essence of Calculus (YouTube, free) — already mentioned in Books — functions as both a reference and a companion. Return to specific episodes when the geometric intuition behind a concept is unclear.
MIT 18.06 (Strang’s Linear Algebra, §2.3) should be studied alongside or before 18.02 for the connections between multivariable calculus and linear algebra — gradients are linear maps, the Hessian is a symmetric bilinear form — to be visible.
Karpathy’s Neural Networks: Zero to Hero (YouTube, free) builds a complete language model from scratch, implementing automatic differentiation, neural network layers, and training loops step by step. This is not a calculus course, but it makes the chain rule and backpropagation concrete in a way that abstract treatment cannot.
| Course | Platform | Type |
|---|---|---|
| MIT 18.01 Single Variable Calculus (free) | MIT OCW | Entry |
| MIT 18.02 Multivariable Calculus (free) | MIT OCW | Entry |
| Karpathy, Neural Networks: Zero to Hero (free) | YouTube / GitHub | Practice |
Practice, Tools, and Code
3Blue1Brown (mentioned above) is the primary visualization resource. The episodes on derivatives, the chain rule, integrals, Taylor series, and the essence of calculus are the best geometric introductions to each concept.
Paul’s Online Math Notes (free) is a reliable procedural reference for Calculus I-III and differential equations. It is not CS-specific and not especially conceptual, but it is useful when a learner needs a clear worked example, a formula refresher, or extra practice on a technique before returning to the CS-facing material.
Implementing automatic differentiation from scratch is the most important calculus-related coding project for ML practitioners. Karpathy’s micrograd (~150 lines) implements scalar autodiff; extending it to support tensors (as in tinygrad) reveals how production frameworks work. After implementing autodiff, implement the backward pass of a two-layer neural network manually in NumPy, comparing with PyTorch’s computed gradient to verify correctness.
SymPy (Python) and Wolfram Alpha (web) compute symbolic derivatives and integrals, which is useful for checking manual calculations. More importantly, SymPy can be used to explore how the chain rule decomposes complex expressions — running it on neural network loss functions and inspecting the symbolic gradient teaches matrix calculus more concretely than reading about it.
| Resource | Platform | Type |
|---|---|---|
| 3Blue1Brown, Essence of Calculus (free) | YouTube | Practice |
| Paul’s Online Math Notes (free) | tutorial.math.lamar.edu | Reference |
| Karpathy, micrograd (free) | GitHub | Practice |
| SymPy (free) | Python / sympy.org | Practice |
| Wolfram Alpha (free tier) | wolframalpha.com | Practice |
Traps
| Trap | Why it misleads | Better response |
|---|---|---|
| Stopping at single-variable calculus | Most introductory calculus courses cover single-variable derivatives and integrals thoroughly and multivariable calculus briefly or not at all. For CS, the single-variable content is the foundation but multivariable is the destination: partial derivatives, gradients, Jacobians, and the chain rule for vector-valued functions are what backpropagation and probabilistic models require. A learner who considers single-variable calculus “done” and moves on is missing the machinery that ML actually uses. | Treat single-variable calculus as preparation, not completion. After derivatives and basic integration, move explicitly to partial derivatives, gradients, and the multivariable chain rule. Deisenroth et al. Chapter 5 covers the ML-relevant content; MIT 18.02 covers the mathematical foundation. |
| Prioritizing integration techniques over gradient reasoning | Standard calculus courses spend substantial time on integration by parts, trigonometric substitution, and partial fractions — techniques for computing antiderivatives in closed form. For CS, these are rarely useful. What is needed is comfort with gradients, the chain rule, and the connection between differentiation and optimization. The effort allocation in a typical calculus course does not match the needs of a CS practitioner. | After mastering the definition of the derivative, the chain rule, and basic integration, focus on gradients and the multivariable chain rule before investing heavily in integration techniques. Use MIT 18.02 or Deisenroth et al. rather than working through the integration chapters of Stewart exhaustively. |
| Treating autograd as a substitute for understanding | Automatic differentiation frameworks compute correct gradients without manual calculation, which removes the immediate need to understand calculus. Learners who skip from “how to use PyTorch” to “building models” without understanding what .backward() is computing find themselves unable to diagnose gradient issues: vanishing gradients in early layers, exploding gradients in recurrent networks, incorrect gradients from in-place operations. |
Implement a scalar autograd from scratch before using PyTorch’s. Karpathy’s micrograd is the right project. After implementing it, implement the backward pass of a small network manually in NumPy. These implementations, not reading, build the understanding that makes autograd transparent. |
| Avoiding differential equations as “too applied” | Learners with a theoretical inclination often defer differential equations as engineering material with no bearing on serious mathematics. This cuts them off from neural ODEs, diffusion models, score-based generative models, control theory, and physics simulation — a substantial and growing part of the ML landscape. ODEs and SDEs are now first-class citizens of deep learning research. | Study at minimum the first-order ODE solution methods and the geometric (phase portrait) view of dynamical systems that Strogatz develops. The geometric view — understanding the behavior of solutions without computing them explicitly — is more important for CS than the catalog of solution techniques. |
| Neglecting the numerical perspective | Mathematical calculus is exact; numerical calculus is not. Softmax overflows naively. Gradients can vanish to zero or explode to infinity in deep networks. ODE solvers become unstable with step sizes that are too large. These are not edge cases; they are routine engineering problems. A practitioner who has not thought about numerical stability will be surprised by them repeatedly. | Read Goodfellow et al. Deep Learning Chapter 4 (Numerical Computation) early. Implement softmax naively and with the log-sum-exp trick and observe the difference. Study gradient clipping and why it exists. The numerical perspective on calculus is a distinct subject that requires explicit attention. |
| Skipping real analysis entirely | For most applied CS work, informal reasoning about limits and continuity is adequate. But for ML theory — convergence proofs, generalization bounds, the analysis of optimization — formal analysis is the language in which results are stated and proved. A practitioner who has not studied analysis can read the conclusions of ML theory papers but cannot evaluate the proofs. | For learners planning to do ML research, Abbott’s Understanding Analysis is the accessible entry to real analysis. The investment of one semester pays dividends in the ability to read and evaluate theoretical results, not just cite them. For practitioners not planning ML research, informal calculus is adequate, and this trap can be deferred. |
中文
微积分是研究连续变化的数学。代数处理的是量之间的静态关系，而微积分处理的是量如何变化——一个表面冷却得有多快，一个人口增长得有多快，一个物理系统如何随时间演化。它的两个奠基操作是导数和积分：导数度量函数的瞬时变化率，积分则把无限多个无限小的量累积成一个有限总和。微积分基本定理说，这二者互为逆运算：要知道某个东西累积了多少，就找出一个变化率等于累积率的函数，并在端点处求值。这一个洞见，加上系统计算导数和积分的规则，成为物理学、工程学和经济学三百年来的工作语言。
对于计算机科学来说，微积分中真正相关的部分，比这段历史暗示的要更有选择性。系统程序员或数据库工程师可能多年都不需要用到它。但在需要它的地方，这种依赖非常深。机器学习的核心操作——通过梯度下降进行训练——就是多变量微积分的应用。每一次参数更新，都需要计算损失如何随着每个参数变化而变化，并通过链式法则把这个计算沿着可能由数百个函数组成的复合结构传播回去。计算机图形学通过积分来模拟光传输，通过微分方程来模拟物理动力学。科学计算的主要任务，就是求解那些没有闭式解的微分方程。机器人学使用控制理论，而控制理论就是应用于动力系统的微积分。微积分并不是每个子领域都统一要求的东西；但对于那些需要它的子领域来说，它不可替代。
数学分析是被严格化的微积分。Newton 和 Leibniz 发展出的导数与积分规则是正确的，但人们花了两个世纪才把它们放在逻辑上可靠的基础之上——精确说明极限是什么，函数什么时候可微，哪些函数可以积分。实分析就是这个基础；它在 CS 中的重要性，体现在那些关于近似的非形式推理会失效的地方：机器学习理论、数值分析，以及支撑量子计算的泛函分析。
前置知识：线性代数（§2.3）是独立的。概率论（§2.5）也是独立的。微积分是第五章（人工智能）以及若干应用章节的前置基础。
从无穷小量到自动微分
微积分解决的问题非常古老。约公元前 250 年，Archimedes 用穷竭法计算抛物线下的面积：把区域切成薄片，求和，并证明当薄片变得无限薄时，这个和会趋近精确面积。他已经有了积分的思想，但还没有一般工具。Kepler 在 1609 年为验证第二定律而计算行星在椭圆轨道上扫过的面积时，也使用了类似近似。但二者都没有一种适用于任意函数的系统方法。
系统方法几乎同时独立出现了两次。Isaac Newton 在 1665 至 1666 年间发展出他称为“流数法”（method of fluxions）的方法；当时剑桥因瘟疫关闭，他独自在 Lincolnshire 工作——这是数学史上最富成果的两年。他用这种方法从引力定律推出 Kepler 定律，表明描述苹果下落的数学，也能描述行星轨道。Gottfried Wilhelm Leibniz 在 1673 至 1676 年间独立发展了微分学，并于 1684 年发表， introducing 了数学家至今仍在使用的记号——dy/dx 表示导数，∫ 表示积分。随后发生的激烈优先权争论，使欧洲大陆数学和英国数学分裂了一代人；英国数学家忠诚地使用 Newton 较差的记号，而大陆数学家则沿着 Leibniz 的记号继续推进。
两个人都无法严谨解释的是：“无穷小量”到底是什么。他们的计算会把无限小的量 dx 在除以它时当作非零，在它变得不方便时又当作零——这个过程在实践中极其有效，却在逻辑上说不通。哲学家 George Berkeley 在 1734 年嘲讽说：“那么这些流数是什么？是消逝增量的速度。那么这些消逝的增量又是什么？它们既不是有限量，也不是无限小量，也不是无。我们难道不能称它们为已逝数量的幽灵吗？”Berkeley 的批评在数学上是正确的。规则有效；但正当性并不存在。
严谨基础在一个世纪后才到来。Augustin-Louis Cauchy 在 1820 年代发展了极限概念，使“趋近”某个值获得精确含义，而不再依赖有问题的无穷小量。Karl Weierstrass 在 1850 和 1860 年代用 epsilon-delta 定义完成了这一计划——这就是今天分析课程中教授的版本——它把极限、连续性和收敛变成具有明确真值条件的精确数学概念。Bernhard Riemann 在 1854 年把积分定义为矩形和的极限，明确规定哪些函数可以被积分。Henri Lebesgue 在 1902 年进一步扩展出更强大的积分，能够处理 Riemann 定义无法处理的函数。这些发展把微积分从一种极其强大的启发式工具，转化成了严格意义上的数学。
严格化带来了一个悖论式后果：它揭示出有多少事情可能出错。直觉上似乎连续的函数，可能不可微；直觉上似乎收敛的级数，可能发散；直觉上似乎可以交换顺序的极限，可能不能交换。数学分析就是理解这些微妙之处，并谨慎证明相关定理的学科。对多数工程应用来说，这些微妙处不会出现，非形式推理已经足够。但对机器学习理论来说——尤其是优化算法的收敛分析、过参数化模型的泛化理论——这些微妙之处很重要，也需要分析的严谨性才能正确推理。
微积分与计算机科学的交叉，真正开始于 1940 年代。当时第一批计算机被用来求解弹道学、流体动力学和核物理中的微分方程。有限差分方法是主要工具：用差商近似导数，通过小步求和近似积分。数值分析这一领域由此发展出来，用来理解这些近似什么时候准确，什么时候会灾难性发散。关键概念仍然是条件性：有些微分方程具有刚性，意味着微小扰动会指数级增长，使得数值方法如果不经仔细设计就会不稳定。理解刚性需要理解微分方程的结构，而这又需要分析。
自动微分来自另一种需求。到 1960 年代，程序员需要手工计算复杂函数的导数，并把它们显式编码进去——这个过程繁琐且容易出错。Seppo Linnainmaa 1970 年的硕士论文表明，链式法则可以被机械地应用于任何由基本操作组合而成的计算，在不做符号操作、也不做有限差分近似的情况下，把精确导数作为前向计算的副产品计算出来。这就是自动微分。这个洞见此前已经被发现过，也后来被多次重新发现，但 Linnainmaa 的工作清楚确立了它。自动微分的反向模式——在一次反向传播中计算一个标量值函数相对于所有输入的梯度——正是反向传播所实现的东西。当 Rumelhart、Hinton 和 Williams 在 1986 年的 Nature 论文中描述反向传播时，他们是在把反向模式自动微分应用到神经网络上。当你在 PyTorch 中调用 .backward() 时，你运行的就是 Linnainmaa 的算法。
2010 年代深度学习的发展，使微积分成为比以往任何应用都影响更广的专业工具。链式法则被应用于数百层结构，穿过包含数百万元素的张量，这正是训练大型神经网络成为可能的数学操作。梯度下降算法经过几十年优化理论的分析和改造，正是神经网络能够学到东西的原因。要理解反向传播在计算什么，梯度什么时候会消失或爆炸，以及为什么某些架构更容易训练，就需要微积分——不是手工计算不定积分的能力，而是对导数、链式法则，以及它们在复合时如何表现的概念理解。
导数、积分与由此出现的结构
面向 CS 实践者的微积分，围绕三组思想组织自身；每一组都会出现在多个后续语境中。
导数与复合函数的微积分
函数 f 在点 x 处的导数，度量的是瞬时变化率：在这个特定点上，x 每变化一个单位，f(x) 会变化多少。几何上，它是切线的斜率。计算上，它是差商 (f(x+h) - f(x))/h 在 h 趋近于零时的极限。导数本身也是一个函数，它为每个 x 分配该点的斜率——而这个函数还可以再次求导，得到二阶导数，用来度量变化率本身如何变化。
链式法则计算复合函数的导数。如果 z = f(y)，且 y = g(x)，那么 dz/dx = (dz/dy)(dy/dx)。正是这一条规则，使微积分能够应用于由简单部分构成的复杂系统：任意复合函数的导数，都是其各部分导数在适当点处的乘积。在一个有 L 层的神经网络中，损失相对于第一层权重的梯度，是 L 个 Jacobian 矩阵的乘积——每层一个——再乘以损失相对于最终层输出的梯度。反向传播就是从输出开始向后工作，高效计算这个乘积，从而避免冗余计算。
对于多变量函数——而每一个神经网络都是一个拥有数百万变量的函数——导数会变成一个向量：梯度 ∇f = (∂f/∂x₁, …, ∂f/∂xₙ)，其中每个偏导数 ∂f/∂xᵢ 度量在保持其他变量不变时，函数相对于 xᵢ 的变化率。梯度指向最陡上升方向；梯度下降则沿其反方向移动。对于向量值函数，导数是一个矩阵——Jacobian 矩阵 J，其中 J_ij = ∂f_i/∂x_j。反向传播计算的是向量—Jacobian 乘积，而不是完整 Jacobian；这也是它能够扩展到拥有数百万参数的网络的原因。
Taylor 定理说，任何光滑函数在某个点附近都可以被多项式近似。一阶近似是 f(x + δ) ≈ f(x) + ∇f(x)ᵀδ——一个线性函数。梯度下降隐含地做了这种近似，并迈出一个最小化它的步子；步长必须足够小，才能使近似有效。二阶近似包含 Hessian 矩阵 H = ∂²f/∂xᵢ∂xⱼ，即二阶导数组成的矩阵，用来描述 f 的曲率。Newton 方法使用二阶近似，并直接走向最小化该近似的步子；它在高曲率区域比梯度下降收敛快得多，但代价是计算并求逆 Hessian 矩阵——这对大型神经网络来说过于昂贵，但它是理解为什么曲率在优化中重要的基础。
积分与累积
积分是微分的逆运算，也是累积的数学。定积分 ∫ₐᵇ f(x) dx 会在区间 [a, b] 上累积 f 的无限多个无限小贡献；几何上，它是曲线下方的带符号面积。
积分在 CS 中主要出现在两个语境中。第一个是概率：概率密度函数 p(x) 必须满足 ∫p(x) dx = 1；期望值是 E[f(X)] = ∫f(x)p(x) dx；边缘分布通过把变量积分掉得到。让分布总和为一的归一化常数——能量模型中的配分函数、贝叶斯推断中的边缘似然——都是通常没有闭式解的积分，这也是近似推断方法存在的原因。扩散模型通过随机微分方程定义，其实现的前向和反向过程，都使用积分微积分来描述和分析。
第二个语境是物理仿真，主要在图形学和机器人学中。渲染方程——物理真实感渲染的基本方程，由 Kajiya 在 1986 年推导——把某个方向发出的光描述为对所有入射光的积分，并用表面的反射函数加权。通过蒙特卡洛采样求解这个积分（随机选择入射方向并取平均）就是路径追踪；理解这个估计器的方差和收敛性，需要概率论和积分微积分。在机器人学和控制中，系统状态按照微分方程演化；预测系统未来所在位置，需要把这些方程沿时间向前积分。
微分方程与连续系统的动力学
微分方程把一个函数与它的导数联系起来。最简单的 dy/dt = ky 表示 y 的变化率与 y 成正比——这是指数增长与衰减方程，描述从细菌种群到放射性衰变，再到电容器充电的一切现象。Newton 第二定律 F = ma 也是一个微分方程：力等于质量乘以位置对时间的二阶导数。由微分方程支配的系统在物理世界中无处不在，而模拟它们是图形学、机器人学和科学计算中的主要计算任务之一。
常微分方程（ODE）涉及单个自变量，通常是时间。对它们的分析会揭示解是增长、衰减还是振荡，以及扰动是被放大还是被抑制——这就是稳定性概念，决定了一个控制系统会收敛到设定点，还是偏离它。数值 ODE 求解器——Euler 方法、Runge-Kutta 方法、自适应步长方法——会通过小时间步近似求解：在当前状态计算导数，然后据此推进。近似的准确性和稳定性取决于 ODE 的结构；为给定问题选择正确求解器，是一个应用分析问题。
偏微分方程（PDE）涉及多个自变量——例如同时关于空间和时间的函数，或关于多个空间维度的函数。热方程、波方程、Schrödinger 方程，以及描述流体流动的 Navier-Stokes 方程，都是 PDE。大规模科学计算很大程度上就是数值求解 PDE：在空间和时间上离散化方程，求解由此得到的线性系统，并推进解。这既是一个数值分析问题（离散化的稳定性和准确性），也是一个线性代数问题（如何高效求解可能极其庞大的稀疏线性系统）。
Chen 等人在 2018 年引入的 Neural ODE，把微分方程直接带入深度学习：网络的隐藏状态按照一个由神经网络参数化的 ODE 演化，输出则由 ODE 求解器计算。这把神经网络的深度重新解释为一个连续时间参数，而不是离散层数，并把深度学习与经典动力系统理论连接起来。扩散模型定义了一个逐渐破坏数据的随机前向过程，以及一个学习得到的反向去噪过程，它们建立在随机微分方程之上——而这种数学结构同样用于建模 Brownian motion 和金融衍生品。
分析：当直觉失效时
实分析是微积分的严谨基础，并且在关于极限和收敛的直觉推理出错之处，对 CS 很重要。
优化算法的收敛是一个分析问题。证明随机梯度下降会收敛，刻画它的收敛速度，并理解学习率如何影响收敛，需要对随机变量序列及其极限进行精确推理。梯度下降的标准收敛结果——对具有 Lipschitz 连续梯度的凸函数，收敛速度为 O(1/t)；对强凸函数更快——都是用分析证明的。要理解这些结果，而不只是使用它们，需要 epsilon-delta 这套工具。
泛化理论是一个分析问题。VC 维、Rademacher 复杂度和 PAC-Bayes 界限，用来刻画学习算法何时能够从训练数据泛化到新数据；它们的陈述和证明使用测度论和泛函分析——也就是把分析扩展到无限维函数空间中。双重下降现象，即过参数化模型在超过插值阈值之后仍继续改善，是一个活跃的分析研究领域。
泛函分析，即无限维向量空间（函数空间）的研究，是若干 CS 应用的基础。核方法——SVM、高斯过程——运行在无限维再生核 Hilbert 空间中；使它们在计算上可处理的表示定理，就是一个泛函分析结果。量子计算是在 Hilbert 空间上的线性代数；其基础是泛函分析。连续归一化流和基于 score 的生成模型，其数学涉及函数空间上的微分几何。
学习这一部分会改变什么
微积分和分析改变的是实践者能够理解什么，而不只是能够计算什么。
最直接的变化，是进入基于梯度的学习。梯度下降、Adam、学习率调度、梯度裁剪、批归一化对优化的影响——这些都是优化器在一个由微积分定义的损失景观中移动的行为。一个内化了梯度、链式法则和 Hessian 二阶结构的实践者，可以提出这些问题：这个优化问题条件良好吗？早期层的梯度是否可能消失？这个优化器正在穿越的有效曲率是什么？这些问题的答案扎根于微积分；没有这个基础的实践者无法提出它们。
第二个变化，是进入概率模型。期望值、边缘分布、KL 散度、熵——这些定义概率模型在做什么的数学量，都通过积分来定义。变分自编码器中的 ELBO、策略梯度方法中的 score function estimator、作为不同训练目标的正向和反向 KL 散度，都需要对积分感到熟悉才能理解。一个只计算过梯度、没有接触过积分的实践者，在概率模型面前会撞墙。
第三个变化，是理解连续仿真的能力。Neural ODE、扩散模型、图形学和机器人学中的物理引擎，全部建立在微分方程之上。一个知道微分方程是什么、数值求解器为何稳定、动力系统相图揭示什么的人，可以在数学层面处理这些系统，而不只是停留在 API 层面。
第四个变化，是关于近似的校准能力。分析训练的是这样一种习惯：这个近似有多好？它什么时候会失效？Taylor 定理说线性近似在一阶意义上是好的；梯度下降的步长必须足够小，才能让这个近似准确。数值积分在步长相对于函数变化足够小时才准确；正确步长取决于被积函数的光滑性。这种校准能力——量化近似误差，并知道它什么时候重要——来自分析，并会迁移到所有使用近似的领域。
资源
书籍与文本
对于正式学习前的几何直觉，3Blue1Brown 的 Essence of Calculus（YouTube，免费，共 11 集）会用几何和视觉方式建立对导数、积分和微积分基本定理的理解。它不教授计算技巧，但会提供正确的概念框架，使后续计算变得有意义，而不是机械操作。多数学习者都应该在正式微积分课程之前或同时观看这个系列。
对于标准基础处理，Stewart 的 Calculus: Early Transcendentals（第 9 版，2020）是多数大学微积分课程使用的综合参考书。它内容完整、例子丰富、可靠。多数学习者不需要从头到尾学完；就 CS 目的而言，关键内容是导数、链式法则、偏导数、梯度，以及足够理解概率密度和期望值的积分。它对积分技巧的大量覆盖（三角代换、部分分式等）并不是 CS 优先事项。MIT OpenCourseWare 的 18.01 和 18.02 分别提供单变量和多变量微积分的免费替代方案，并带有完整习题和考试。
对于具有数学谨慎性的多变量微积分，Hubbard 和 Hubbard 的 Vector Calculus, Linear Algebra, and Differential Forms（第 5 版）会在一个统一的几何框架中同时发展向量微积分和线性代数。它比 Stewart 要求更高，但能更深入理解线性代数与微积分之间的关系——尤其适合理解 Jacobian，以及梯度如何变换。Marsden 和 Tromba 的 Vector Calculus 是同等深度下更传统的替代选择。
对于机器学习所需的微积分内容——矩阵微积分、Jacobian、链式法则与反向传播之间的关系——Deisenroth、Faisal 和 Ong 的 Mathematics for Machine Learning 第 5 章（可在 mml-book.com 免费获取）是目前最直接的处理。多数微积分课程并不发展矩阵微积分；这一章会做。对 ML 实践者来说，它是 Strang 线性代数的正确伴读材料。
对于严谨基础，Spivak 的 Calculus（第 4 版）从实数公理一直发展到 Riemann 积分，以异常清晰和严肃的数学态度处理单变量微积分。它要求很高——更接近第一门分析课程，而不是标准微积分教材——但对于想理解定理为什么成立，而不只是如何使用定理的学习者，它是正确选择。Abbott 的 Understanding Analysis（第 2 版）更易进入，并以更强的教学意识覆盖类似的分析内容；对于第一次接触实分析的学习者，它是推荐起点。Tao 的 Analysis I 和 Analysis II（免费在线）则以极高精确性从第一原则发展分析。
对于微分方程，Strogatz 的 Nonlinear Dynamics and Chaos（第 2 版，2018）是从动力系统视角进入 ODE 的最易读且严谨的处理。它强调几何思考（相图、稳定性、分岔），而不是解法技巧，而这正是 CS 实践者最需要的。专门的 ODE 和 PDE 参考书应放到后续科学计算、仿真、图形学、机器人学或领域特定学习中，而不是放在这个基础章节中。
对于自动微分本身，Karpathy 的 micrograd（GitHub，免费，约 150 行 Python）从零实现了标量反向模式自动微分。完整学一遍它，会让链式法则在代码中具体化。Baydin 等人的 “Automatic Differentiation in Machine Learning: a Survey”（2018，免费）覆盖完整理论，包括前向模式、反向模式和高阶微分。
对于数值方法——微积分的计算实现——Trefethen 和 Bau 的 Numerical Linear Algebra（§2.3 参考）覆盖线性代数部分；数值 ODE 和 PDE 内容会在 §6.5 Scientific Computing and Simulation 中处理。
| 书籍 | 作用 | 类型 |
|---|---|---|
| Stewart, Calculus: Early Transcendentals（第 9 版） | 标准综合参考 | 入门 |
| MIT OCW 18.01 + 18.02（免费） | 带完整习题的免费替代方案 | 入门 |
| Hubbard & Hubbard, Vector Calculus, Linear Algebra, and Differential Forms | 统一的多变量处理 | 深入 |
| Marsden & Tromba, Vector Calculus | 深入多变量微积分 | 深入 |
| Deisenroth, Faisal & Ong, Mathematics for Machine Learning Ch. 5（免费） | 矩阵微积分；面向 ML 的链式法则 | 入门 |
| Spivak, Calculus（第 4 版） | 带证明的严谨单变量微积分 | 深入 |
| Abbott, Understanding Analysis（第 2 版） | 易进入的实分析入口 | 入门 |
| Tao, Analysis I and Analysis II（免费在线） | 从第一原则展开实分析 | 深入 |
| Strogatz, Nonlinear Dynamics and Chaos（第 2 版） | ODE；动力系统；几何视角 | 深入 |
| Rudin, Principles of Mathematical Analysis | 经典实分析参考 | 辅助 |
课程与讲座
MIT 18.01（Single Variable Calculus，可在 MIT OCW 免费获取）和 18.02（Multivariable Calculus）是标准免费课程，带完整习题、考试和讲课视频。OCW 上有多位教师讲授 18.02 的版本；Denis Auroux 的版本尤其受到好评。
3Blue1Brown 的 Essence of Calculus（YouTube，免费）——前面在书籍部分已经提到——既可作为参考，也可作为伴随材料。当某个概念背后的几何直觉不清楚时，可以回到具体集数。
MIT 18.06（Strang 的 Linear Algebra，§2.3）应与 18.02 同时或先于 18.02 学习，这样多变量微积分与线性代数之间的连接才会显现出来——梯度是线性映射，Hessian 是对称双线性形式。
Karpathy 的 Neural Networks: Zero to Hero（YouTube，免费）从零构建完整语言模型，逐步实现自动微分、神经网络层和训练循环。它不是微积分课程，但它以抽象处理无法做到的方式，让链式法则和反向传播变得具体。
| 课程 | 平台 | 类型 |
|---|---|---|
| MIT 18.01 Single Variable Calculus（免费） | MIT OCW | 入门 |
| MIT 18.02 Multivariable Calculus（免费） | MIT OCW | 入门 |
| Karpathy, Neural Networks: Zero to Hero（免费） | YouTube / GitHub | 实践 |
实践、工具与代码
3Blue1Brown（上文已提到）是主要的可视化资源。关于导数、链式法则、积分、Taylor 级数和微积分本质的几集，是这些概念最好的几何入门。
Paul’s Online Math Notes（免费）是 Calculus I–III 和微分方程的可靠程序性参考。它并不面向 CS，也不特别强调概念，但当学习者需要一个清楚的完整例题、公式回顾，或在回到面向 CS 的材料之前额外练习某个技巧时，它很有用。
从零实现自动微分，是 ML 实践者最重要的微积分相关编码项目。Karpathy 的 micrograd（约 150 行）实现了标量自动微分；把它扩展到支持张量（类似 tinygrad）会揭示生产级框架如何工作。在实现自动微分之后，再用 NumPy 手动实现一个两层神经网络的 backward pass，并与 PyTorch 计算出的梯度比较，以验证正确性。
SymPy（Python）和 Wolfram Alpha（网页）可以计算符号导数和积分，适合检查手算结果。更重要的是，SymPy 可以用来探索链式法则如何分解复杂表达式——把它应用到神经网络损失函数上，并检查符号梯度，会比单纯阅读更具体地教会矩阵微积分。
| 资源 | 平台 | 类型 |
|---|---|---|
| 3Blue1Brown, Essence of Calculus（免费） | YouTube | 实践 |
| Paul’s Online Math Notes（免费） | tutorial.math.lamar.edu | 参考 |
| Karpathy, micrograd（免费） | GitHub | 实践 |
| SymPy（免费） | Python / sympy.org | 实践 |
| Wolfram Alpha（免费层） | wolframalpha.com | 实践 |
陷阱
| 陷阱 | 为什么会误导 | 更好的回应 |
|---|---|---|
| 停留在单变量微积分 | 多数入门微积分课程会充分讲解单变量导数和积分，而多变量微积分则讲得很简略，甚至完全不讲。对 CS 来说，单变量内容是基础，但多变量才是目的地：偏导数、梯度、Jacobian，以及向量值函数的链式法则，才是反向传播和概率模型所需要的东西。一个把单变量微积分视为“完成”的学习者，会错过 ML 真正使用的机制。 | 把单变量微积分视为准备，而不是完成。在掌握导数和基本积分之后，明确转向偏导数、梯度和多变量链式法则。Deisenroth 等人的第 5 章覆盖 ML 相关内容；MIT 18.02 覆盖数学基础。 |
| 把积分技巧置于梯度推理之上 | 标准微积分课程会花大量时间讲分部积分、三角代换和部分分式——这些是为了计算闭式原函数的技巧。对 CS 来说，它们很少有用。真正需要的是对梯度、链式法则，以及微分与优化之间关系的熟悉。典型微积分课程中的精力分配，与 CS 实践者的需求并不匹配。 | 掌握导数定义、链式法则和基本积分之后，先集中学习梯度和多变量链式法则，再大量投入积分技巧。使用 MIT 18.02 或 Deisenroth 等人的材料，而不是穷尽 Stewart 的积分章节。 |
| 把 autograd 当成理解的替代品 | 自动微分框架可以在不手算的情况下给出正确梯度，这会移除立即理解微积分的需求。学习者如果从“如何使用 PyTorch”直接跳到“构建模型”，而不理解 .backward() 在计算什么，就会无法诊断梯度问题：早期层梯度消失、循环网络梯度爆炸、原地操作导致错误梯度。 |
在使用 PyTorch 的自动微分之前，先从零实现一个标量 autograd。Karpathy 的 micrograd 是正确项目。完成之后，再用 NumPy 手动实现一个小网络的 backward pass。这些实现，而不是阅读，会建立使 autograd 透明化的理解。 |
| 把微分方程视为“太应用”而回避 | 有理论倾向的学习者常常把微分方程推迟，认为它只是工程材料，与严肃数学无关。这会让他们与 Neural ODE、扩散模型、基于 score 的生成模型、控制理论和物理仿真脱节，而这些已经是 ML 版图中相当大且持续增长的一部分。ODE 和 SDE 现在已经是深度学习研究中的一等对象。 | 至少学习一阶 ODE 解法，以及 Strogatz 所发展的动力系统几何视角（相图）。对 CS 来说，几何视角——在不显式求解的情况下理解解的行为——比解法技巧目录更重要。 |
| 忽视数值视角 | 数学微积分是精确的；数值微积分不是。朴素 softmax 会溢出。深层网络中的梯度可能消失为零，也可能爆炸到无穷。ODE 求解器会因为步长太大而变得不稳定。这些都不是边缘情况，而是常规工程问题。没有思考过数值稳定性的实践者，会反复被它们惊到。 | 早期阅读 Goodfellow 等人 Deep Learning 第 4 章（Numerical Computation）。分别实现朴素 softmax 和使用 log-sum-exp 技巧的 softmax，并观察差异。学习梯度裁剪及其存在原因。微积分的数值视角是一门独立主题，需要明确关注。 |
| 完全跳过实分析 | 对多数应用 CS 工作来说，关于极限和连续性的非形式推理已经足够。但对于 ML 理论——收敛证明、泛化界、优化分析——形式分析是陈述和证明结果的语言。没有学过分析的实践者可以读懂 ML 理论论文的结论，但无法评价其证明。 | 对计划从事 ML 研究的学习者，Abbott 的 Understanding Analysis 是进入实分析的可访问入口。投入一个学期，会在阅读和评价理论结果方面持续产生回报，而不只是让人能够引用它们。对不计划做 ML 研究的实践者来说，非形式微积分通常足够，这个陷阱可以暂缓处理。 |