English
Scientific computing is the discipline of solving mathematical problems that arise in science and engineering through numerical methods and computation. A differential equation describes how heat flows through a material; a numerical method provides an approximation whose error can be bounded; a code implements the method on a computer; a simulation produces data that would be impossible or impractical to obtain by experiment. The chain from physical problem to numerical solution requires mathematical depth, algorithmic sophistication, and engineering discipline that no single element provides alone.
What distinguishes scientific computing from general programming is the primacy of correctness in the numerical sense. A web application that returns a wrong result crashes or produces obviously wrong output; a scientific computation that returns a wrong result may produce plausible-looking numbers that are systematically incorrect in ways the user cannot detect. Floating-point arithmetic introduces rounding errors that accumulate across millions of operations; ill-conditioned problems amplify small errors into large ones; unstable algorithms produce results that bear no relationship to the true solution. The mathematical discipline of numerical analysis exists to characterize these failure modes and design algorithms that avoid them. Scientific computing without numerical analysis is computation that may or may not be producing correct answers, with no way to know.
The connections to other parts of this guide are dense. Linear algebra (§2.3) is the mathematical substrate of almost all numerical computation. Calculus (§2.4), particularly differential equations, defines the problems that much scientific computing solves. High-performance computing (§4.11) provides the computational scale that large simulations require. Machine learning (§5.2–5.3) is entering scientific computing rapidly: neural surrogates for expensive simulations, physics-informed neural networks that incorporate physical constraints into the loss function, and foundation models for molecular biology. The intersection is live and growing.
Prerequisites: Linear algebra (§2.3) at implementation depth — eigenvalue algorithms, iterative solvers. Calculus (§2.4) — differential equations are the primary problem class. Probability and statistics (§2.5) — Monte Carlo methods. Algorithms (§2.6) — algorithmic analysis of numerical methods.
From Ballistic Tables to AlphaFold
The computational demands of science preceded electronic computers. The first large-scale numerical computation was the New Mathematical Tables project, a British government enterprise in the 1930s that employed human “computers” — mathematicians working with mechanical calculators — to produce accurate tables of logarithms, trigonometric functions, and other values that engineers and navigators needed. The organization of this work — task decomposition, error checking, systematic verification — invented the methodology of large-scale numerical computation before any machine could execute it.
The ENIAC, operational in 1945, was designed specifically for scientific computation: calculating ballistic tables for artillery fire, which required solving the differential equations of projectile motion for thousands of combinations of angle, muzzle velocity, and atmospheric conditions. The speed advantage over human computers was roughly a thousand-fold: computations that took a human computer weeks were done by ENIAC in hours. The project demonstrated that numerical simulation could replace physical experiment for some purposes — a claim that has grown only more dramatic as computers have grown more powerful.
The numerical analysis of the early electronic computer era was organized around a fundamental problem: floating-point arithmetic was new, its error properties were not well understood, and the behavior of algorithms in finite-precision arithmetic could diverge dramatically from their behavior in exact arithmetic. John von Neumann and Herman Goldstine’s 1947 analysis of Gaussian elimination was among the first careful studies: they showed that the condition number of a matrix — roughly, the ratio of its largest to smallest singular value — governed the accuracy of linear system solutions and that partial pivoting during Gaussian elimination controlled error amplification. This line of work gave rise to the condition number as a central concept in numerical analysis, and to the understanding that algorithmic stability — whether small perturbations in input produce small perturbations in output — was a property that could be analyzed and that many obvious algorithms lacked.
The most consequential developments of the 1960s and 1970s were the numerical algorithms for linear algebra that remain in use today. The QR algorithm for computing eigenvalues (Francis and Kublanovskaya, independently 1961) replaced the power method and deflation with a provably stable iteration; the algorithm in its modern form is still the standard for dense eigenvalue computation. The QR decomposition and its role in least-squares problems was analyzed and the numerically stable implementation via Householder reflections was established. Cleve Moler’s MATLAB (1984), built initially as a wrapper around the LINPACK and EISPACK linear algebra libraries, made these algorithms accessible to engineers and scientists who were not numerical analysts. The subsequent NumPy ecosystem in Python, developed in the 2000s through contributions by Jim Hugunin, Travis Oliphant, and others, made the same algorithms available in a free, open-source framework that became the standard scientific computing platform.
The finite element method — for solving partial differential equations by discretizing the domain into small elements and approximating the solution as a piecewise polynomial — was developed in the 1950s and 1960s primarily in structural engineering. Strang and Fix’s 1973 An Analysis of the Finite Element Method provided the mathematical analysis that connected the engineering practice to approximation theory. The finite element method is now the dominant approach for structural mechanics, fluid mechanics, and electromagnetics simulation, embodied in commercial codes (ANSYS, COMSOL, Abaqus) and open-source frameworks (FEniCS, deal.II, OpenFOAM). The codes that model aircraft wing deformation, simulate blood flow through a reconstructed cardiac artery, and compute electromagnetic fields in a microchip all use the mathematical framework that Strang and Fix analyzed.
Molecular dynamics simulation — integrating Newton’s equations of motion for systems of atoms and molecules — emerged from the 1950s Metropolis algorithm and became a tool for understanding protein folding, drug-receptor interactions, and material properties. The computational challenge is formidable: simulating a protein of a few thousand atoms for a microsecond requires on the order of 10¹² integration steps, each requiring evaluation of interatomic forces over all pairs. Specialized hardware (Anton machines at D.E. Shaw Research) and algorithmic optimizations (fast multipole methods, particle-mesh Ewald) have extended accessible timescales, but much of biochemically interesting biology occurs on timescales still inaccessible to direct simulation.
Climate modeling exemplifies the ambition and the difficulty of large-scale simulation. The first general circulation models of the atmosphere were developed in the 1960s at GFDL and NCAR; current Earth system models resolve the atmosphere, ocean, land surface, ice, and carbon cycle, running on the world’s largest supercomputers. They have successfully reproduced twentieth-century climate observations and provide the scientific basis for climate projections. They also illustrate the central methodological challenge: a climate model is not the Earth; it is an approximation whose quality depends on the grid resolution, the parameterizations of subgrid processes (clouds, convection, sea ice), and the accuracy of the numerical methods. Quantifying how much the approximations matter — uncertainty quantification — is an active and difficult research area.
The transformation brought by machine learning is the most significant since the development of the finite element method. AlphaFold2 (DeepMind, 2020) predicted many protein three-dimensional structures from amino acid sequences with accuracy approaching experimental methods, dramatically advancing a central problem that structural biologists had worked on for fifty years. This was not a replacement for physical simulation; it was a learned model trained on the Protein Data Bank — over 100,000 experimentally determined structures — that generalized to novel sequences. The lesson was not that machine learning was better than simulation but that the relationship between the two was more complex than either paradigm’s advocates had expected: where abundant training data existed, learned models could achieve what simulation could not; where first-principles physical understanding was essential, simulation remained the appropriate tool.
Physics-informed neural networks (PINNs), introduced by Raissi, Perdikaris, and Karniadakis in 2019, represent a different kind of integration: incorporate the differential equation governing a system into the neural network’s loss function, so that the trained network satisfies the physics by construction (approximately). This approach is powerful for inverse problems — inferring model parameters from observations — where classical methods require expensive repeated forward simulations. Neural surrogate models learn to approximate expensive simulations at a fraction of the cost, enabling uncertainty quantification and parameter optimization that would be computationally infeasible with the original simulation. The field of scientific machine learning is rapidly developing its own methodology, theory, and tooling.
Numerical Analysis, Differential Equations, and Scientific Machine Learning
Floating-Point Arithmetic and Algorithmic Stability
Every number in a scientific computation is represented as a floating-point value: a mantissa and an exponent in binary. The IEEE 754 standard specifies the representation: a 64-bit double-precision float has 52 bits of mantissa, giving about 15-16 significant decimal digits, and the result of every arithmetic operation is rounded to the nearest representable value. This rounding is small — roughly 10⁻¹⁶ in relative terms — but it accumulates.
Cancellation is the most dangerous source of error: when two nearly equal quantities are subtracted, the relative error in the result can be arbitrarily large. Computing x - sin(x) for small x is problematic because both terms are nearly equal; expanding sin(x) ≈ x - x³/6 + … and using the polynomial gives much more accurate results. The condition number of a matrix A — κ(A) = σₘₐₓ / σₘᵢₙ, the ratio of the largest to the smallest singular value — measures how much errors in the right-hand side of a linear system Ax = b are amplified in the solution x. A matrix with condition number 10⁶ can lose 6 significant digits in the solution even with perfectly computed intermediate results.
Backward stability is the appropriate standard for numerical algorithms. An algorithm is backward stable if the computed result is the exact result for a nearby input — as if the input had been perturbed by a small relative amount before solving. Backward stable algorithms accumulate errors no worse than the problem’s condition number warrants. Gaussian elimination with partial pivoting is backward stable for almost all matrices; without partial pivoting, it is not. The QR decomposition via Householder reflections is backward stable; the classical Gram-Schmidt process is not (modified Gram-Schmidt is).
Numerical Methods for Differential Equations
Differential equations — ordinary (one independent variable) and partial (multiple independent variables) — are the mathematical models of continuous physical processes. Numerical methods approximate their solutions by discretizing in time and/or space.
For ODEs, explicit methods like Runge-Kutta evaluate the right-hand side at several points within each time step to construct a higher-order approximation. The classic fourth-order Runge-Kutta method requires four function evaluations per step and achieves fourth-order accuracy — the error decreases as h⁴ where h is the step size. Stiff ODEs — systems where some components vary much faster than others — require implicit methods that solve a nonlinear system at each step. The backward Euler method is first-order but unconditionally stable; more sophisticated implicit methods (BDF methods, Radau methods) provide higher order for stiff systems.
For PDEs, the choice of discretization method depends on the equation type and geometry. Finite difference methods replace derivatives with difference quotients on a structured grid, producing sparse linear systems. The Laplacian ∇²u is approximated as (u_{i+1,j} - 2u_{i,j} + u_{i-1,j})/h² + (u_{i,j+1} - 2u_{i,j} + u_{i,j-1})/h²; the truncation error is O(h²) in each direction. Finite element methods work on unstructured meshes, solving the weak form of the PDE by seeking an approximate solution in a space of piecewise polynomials. They handle complex geometry and allow local mesh refinement near features of interest. Spectral methods expand the solution in global basis functions (Fourier modes, Chebyshev polynomials) and achieve spectral convergence — exponential decay of error in the number of modes — for smooth solutions.
The sparse linear systems arising from PDE discretizations are solved by iterative methods: the conjugate gradient method (for symmetric positive definite systems), GMRES (for general nonsymmetric systems), and their preconditioned variants. Preconditioning — multiplying the system by an approximate inverse — is essential for scalability: the raw condition number of a discretized Laplacian is O(h⁻²), making unpreconditioned iteration impractically slow for fine grids. Multigrid methods achieve O(N) complexity for elliptic PDEs by using a hierarchy of coarser grids to accelerate convergence of the fine-grid iteration; they are among the algorithmically most sophisticated methods in numerical analysis and among the practically most important.
Monte Carlo and Uncertainty Quantification
Monte Carlo methods estimate expectations by random sampling. The basic estimator — draw N independent samples x₁, …, x_N from distribution p, compute (1/N) Σ f(xᵢ) as an estimate of 𝔼[f(X)] — converges at rate O(1/√N) regardless of dimension, making it attractive for high-dimensional integration where deterministic quadrature requires exponentially many function evaluations. Variance reduction techniques — importance sampling (sampling from a distribution closer to f·p), control variates (subtracting a correlated quantity whose expectation is known), quasi-Monte Carlo (using low-discrepancy sequences instead of random samples) — reduce the effective variance by factors of 10–1000 for well-designed problems.
Markov Chain Monte Carlo (MCMC) samples from distributions specified up to a normalizing constant, which is essential for Bayesian inference in scientific models. Metropolis-Hastings proposes moves from a proposal distribution and accepts or rejects based on the ratio of target densities; the chain converges to the target distribution under mild conditions. Hamiltonian Monte Carlo exploits gradient information to propose moves that traverse the high-probability region efficiently. Diagnostics for MCMC convergence — Gelman-Rubin R̂ statistic, effective sample size, trace plots — are part of the standard methodology.
Uncertainty quantification (UQ) propagates uncertainty in model inputs through the computational model to quantify uncertainty in outputs. Forward UQ (given uncertain inputs, what is the distribution of outputs?) can be addressed by Monte Carlo sampling of the input distribution or by polynomial chaos expansion, which approximates the input-output map as a polynomial in the uncertain parameters. Inverse UQ — calibrating model parameters from observations — is a Bayesian inference problem that posterior sampling via MCMC can address. UQ is increasingly required as scientific computing moves from “produce a number” to “produce a number with a certificate of reliability.”
What Studying This Changes
Scientific computing changes how practitioners understand computational results and their reliability.
The first change is numerical vigilance: the automatic habit of asking what the condition number is, whether the algorithm is stable, and whether the error is acceptable before trusting a result. This changes how code is written — with verification against known analytical solutions, with monitoring of residuals and convergence rates, with awareness that large intermediate values can produce catastrophic cancellation in the final result. The practitioner who has never debugged a numerical instability will produce them without recognizing them; the practitioner who has internalized numerical analysis will see the warning signs.
The second change is the ability to select and apply appropriate methods. Scipy and Julia’s DifferentialEquations.jl provide dozens of ODE solvers; choosing the right one requires knowing whether the system is stiff, what accuracy is required, and what the computational budget is. A stiff system solved with an explicit Runge-Kutta method will require step sizes orders of magnitude smaller than necessary or will produce unstable garbage. The practitioner who understands stiffness and numerical stability can make this choice correctly; the practitioner who uses the default solver may spend orders of magnitude more computation than necessary or get wrong answers.
The third change is scientific methodology: treating computational results as evidence with quantified uncertainty rather than as facts. Scientific computing produces numbers; the scientific value of those numbers depends on understanding what errors they contain, how sensitive they are to model assumptions, and what validation has been performed. The practitioner who publishes computational results without uncertainty quantification and validation against experimental data or analytical solutions is publishing numbers without the evidence of their quality. The discipline of reproducible, validated, uncertainty-quantified computation is the computational analog of rigorous experimental practice.
The fourth change is the ability to engage productively with scientific machine learning. The field is developing rapidly and the claims made for it range from genuinely important to overstated. The practitioner who understands classical numerical methods can assess which problems are genuinely better addressed by learned models (high-dimensional, data-rich, smooth), which are better addressed by classical methods (constrained, sparse data, sharp geometric features, physical interpretability required), and which benefit from hybrid approaches.
Resources
Books and Texts
Trefethen and Bau’s Numerical Linear Algebra (SIAM, 1997) is the essential text on the subject and one of the most beautifully written technical books in mathematics. It develops the algorithms for solving linear systems, least squares, and eigenvalue problems through the lens of backward stability and condition numbers, provides the geometric intuition behind each algorithm, and connects everything to the underlying mathematics with unusual clarity. Despite being nearly thirty years old, the content is timeless — the algorithms it describes are still the algorithms in use.
Higham’s Accuracy and Stability of Numerical Algorithms (2nd ed., SIAM, 2002) is the comprehensive reference for the numerical analyst who wants rigorous error analysis. Where Trefethen and Bau provides conceptual development, Higham provides complete proofs and detailed error bounds for every major algorithm. It is demanding but authoritative; use it as a reference when you need to know whether an algorithm is backward stable and what the error constants are.
Heath’s Scientific Computing: An Introductory Survey (3rd ed., 2018, classroom edition free) provides the broadest accessible introduction: linear algebra, interpolation, integration, ODE and PDE methods, optimization, and Monte Carlo. It is less deep than Trefethen-Bau on any individual topic but provides orientation across the full scope of the field.
Hairer, Nørsett, and Wanner’s Solving Ordinary Differential Equations (two volumes, Springer) is the definitive reference for ODE numerical methods — every method in common use is developed, analyzed, and compared with insight about when to apply it. Volume I covers nonstiff methods; Volume II covers stiff methods and differential-algebraic equations.
LeVeque’s Finite Difference Methods for Ordinary and Partial Differential Equations (SIAM, 2007) is the accessible and rigorous entry to finite difference methods, with careful treatment of stability and convergence. His Numerical Methods for Conservation Laws covers hyperbolic PDE methods for fluid dynamics.
Nocedal and Wright’s Numerical Optimization (2nd ed., Springer, 2006) is the standard reference for optimization methods — gradient descent, Newton’s method, quasi-Newton methods, constrained optimization — with the theoretical foundation and practical guidance that the subject requires. Boyd and Vandenberghe’s Convex Optimization (Cambridge, 2004, free at stanford.edu/~boyd/cvxbook) covers the special case of convex optimization, where global optimality can be guaranteed.
For scientific machine learning, Brunton and Kutz’s Data-Driven Science and Engineering (Cambridge, 2022) covers the intersection of machine learning with physical systems — dimensionality reduction, neural networks, dynamic mode decomposition, sparse identification of nonlinear dynamics — with strong connections to classical numerical methods.
| Book | Role | Type |
|---|---|---|
| Trefethen & Bau, Numerical Linear Algebra | Numerical LA canonical text | Depth |
| Higham, Accuracy and Stability of Numerical Algorithms (2nd ed.) | Error analysis reference | Reference |
| Heath, Scientific Computing: An Introductory Survey (3rd ed., classroom ed. free) | Accessible broad introduction | Entry |
| Hairer, Nørsett & Wanner, Solving Ordinary Differential Equations (2 vols.) | ODE methods canonical reference | Reference |
| LeVeque, Finite Difference Methods for ODEs and PDEs | PDE finite difference | Depth |
| Strang & Fix, An Analysis of the Finite Element Method (2nd ed.) | FEM mathematical foundation | Depth |
| Nocedal & Wright, Numerical Optimization (2nd ed.) | Optimization canonical reference | Reference |
| Boyd & Vandenberghe, Convex Optimization (free) | Convex optimization | Depth |
| Robert & Casella, Monte Carlo Statistical Methods (2nd ed.) | Monte Carlo foundation | Depth |
| Brunton & Kutz, Data-Driven Science and Engineering | Scientific machine learning | Depth |
| Golub & Van Loan, Matrix Computations (4th ed.) | Matrix computation reference | Reference |
Courses and Lectures
MIT 18.335 (Introduction to Numerical Methods, Steven Johnson, free materials including Jupyter notebooks) covers numerical linear algebra, ODE and PDE methods, and optimization with the mathematical rigor and practical orientation that scientific computing requires.
Trefethen’s Oxford numerical analysis lectures (available via SIAM and partly free online) are an unusually engaging presentation of the material in Numerical Linear Algebra.
Software Carpentry and Data Carpentry (softwarecarpentry.org, free) provide the computational workflow foundations — reproducible research, version control for scientific code, testing of numerical software — that domain scientists often lack. The material on Python scientific computing and on reproducible workflows is particularly valuable.
MIT 18.085/18.086 (Mathematical Methods for Engineers, Gilbert Strang, free on MIT OCW) covers differential equations, Fourier analysis, finite elements, and optimization with applied focus and exceptional clarity.
| Course | Platform | Type |
|---|---|---|
| MIT 18.335 Introduction to Numerical Methods (free) | MIT / GitHub | Entry |
| MIT 18.085/18.086 Mathematical Methods for Engineers (free) | MIT OCW | Entry |
| Software Carpentry (free) | softwarecarpentry.org | Entry |
| Scientific Python Lectures (free) | lectures.scientific-python.org | Entry |
Practice, Tools, and Current Sources
NumPy and SciPy (free) are the foundational Python scientific computing libraries. Working through the SciPy documentation for linear algebra (scipy.linalg), ODE integration (scipy.integrate), optimization (scipy.optimize), and sparse matrices (scipy.sparse) while understanding the underlying algorithms — rather than treating them as black boxes — is the practical entry to scientific computing.
Julia (free, julialang.org) is increasingly the language of choice for serious scientific computing: fast as C, with syntax as readable as Python, and with a package ecosystem (DifferentialEquations.jl, LinearAlgebra, Optim.jl) that provides the numerical methods. The DifferentialEquations.jl package is the most comprehensive ODE/DAE/SDE solver library in any language.
FEniCS (free, fenicsproject.org) and deal.II (free) implement the finite element method in Python and C++ respectively. Working through a FEniCS tutorial — solving the Poisson equation, then the heat equation, then a simple structural mechanics problem — provides hands-on experience with PDE simulation at moderate scale.
BLAS and LAPACK (free, netlib.org) are the foundational numerical linear algebra libraries that all scientific computing depends on, either directly or indirectly. Understanding how they are called from NumPy, SciPy, and Julia — and why calling them correctly matters for performance — provides insight into the numerical computing stack.
The Raissi, Perdikaris, Karniadakis papers on PINNs (free, arXiv) are the foundational papers for physics-informed neural networks. Working through a PINN implementation for the 1D Burgers equation — using PyTorch to encode the PDE residual in the loss function — makes the methodology concrete.
| Resource | Platform | Type |
|---|---|---|
| NumPy / SciPy documentation (free) | numpy.org / scipy.org | Practice |
| Julia + DifferentialEquations.jl (free) | julialang.org | Practice |
| SciML documentation (free) | docs.sciml.ai | Reference |
| FEniCS (free) | fenicsproject.org | Practice |
| FEniCSx tutorial (free) | jsdokken.com/dolfinx-tutorial | Practice |
| Raissi, Perdikaris & Karniadakis PINN papers (free) | arXiv | Depth |
| MIT Julia computing course (18.S191, free) | computationalthinking.mit.edu | Entry |
| Software Carpentry Python tutorials (free) | software-carpentry.org | Entry |
Traps
| Trap | Why it misleads | Better response |
|---|---|---|
| Computing without numerical analysis | Libraries like NumPy and SciPy make it possible to run large numerical computations without understanding the algorithms’ accuracy or stability properties. Ill-conditioned linear systems produce wrong answers silently; unstable ODE integrators produce results that look converged but are far from the true solution; Monte Carlo estimators with high variance give confidence intervals that are much wider than reported. | Study numerical analysis alongside scientific computing practice. After running any computation, ask: what is the condition number? Is the algorithm backward stable? Does the result converge with grid refinement or more Monte Carlo samples? Verify implementations against known analytical solutions whenever they exist. |
| Using the default solver without understanding stiffness | Scipy’s solve_ivp with the default RK45 solver works well for non-stiff ODEs. Applied to a stiff ODE — a chemical reaction network, a circuit with fast and slow timescales — it takes steps orders of magnitude smaller than necessary or produces garbage. The failure is not always dramatic; the code may run to completion with results that look plausible but are wrong. |
Learn to recognize stiff systems: eigenvalues of the Jacobian spanning many orders of magnitude, or dynamics with fast transients and slow long-term behavior. Use implicit solvers (Radau, BDF, or Julia’s stiff algorithms) for stiff systems. Verify by comparing with a very small fixed step size. |
| Conflating accuracy with convergence | An algorithm can converge — produce the same answer as the step size decreases — but converge to the wrong answer, because the algorithm is unstable or the problem is so ill-conditioned that small floating-point errors dominate the solution. Convergence testing is necessary but not sufficient for accuracy. | Always verify against independent reference solutions (analytical where available, or a much more expensive high-accuracy computation). Compute condition numbers for linear solves. Estimate error bounds, not just residuals. The residual of a solution to Ax = b can be small even when the error ‖x - x*‖ is large if A is ill-conditioned. |
| Skipping reproducibility as a methodological concern | Scientific computations depend on random seeds (for Monte Carlo methods), library versions (numerical behavior can differ across versions), compiler flags (floating-point optimization settings), and hardware. A result that cannot be reproduced is not a scientific result; it may be an artifact of any of these factors. | Use version control for all scientific code. Pin library versions with conda environments or requirements files. Document random seeds. Archive the exact software environment using containers (Docker or Singularity) for any result that will be published or shared. The Software Carpentry practices on reproducible research are worth adopting systematically. |
| Applying machine learning where classical methods work better | The excitement about physics-informed neural networks and neural surrogates has led some practitioners to apply machine learning to problems where classical numerical methods are demonstrably better: smooth, well-characterized problems with sparse data, where the training data for a neural surrogate is more expensive to generate than the classical simulation. | Assess whether a learning-based approach is justified by the problem structure. Classical FEM or spectral methods with well-established error bounds are often superior for smooth problems with accurate physics models. Learning is most valuable when: the physics model is uncertain or expensive, the dimension is too high for classical grids, or abundant observational data can substitute for physics. |
| Publishing results without uncertainty quantification | A computational result is a number with error. Reporting only the number — without error bounds, without sensitivity analysis, without validation against independent data — obscures how much the result should be trusted. This is increasingly recognized as a scientific integrity concern in computational science, not just a methodological nicety. | Include uncertainty quantification in any scientifically significant computation: error estimates from convergence analysis, sensitivity analysis over uncertain parameters, validation comparisons against experimental data or independent computational results. Report error bars alongside numbers, and make clear whether they represent numerical discretization error, modeling uncertainty, or statistical sampling error. |
中文
科学计算是一门通过数值方法和计算来求解科学与工程中数学问题的学科。一个微分方程描述热量如何在材料中流动；一种数值方法提供一个误差可以被界定的近似；一段代码在计算机上实现这种方法；一次仿真产生那些通过实验不可能或不现实获得的数据。从物理问题到数值解的链条，需要数学深度、算法成熟度和工程纪律，任何单一环节都无法独立提供全部能力。
科学计算区别于一般编程之处，在于数值意义上的正确性具有首要地位。一个 Web 应用返回错误结果，通常会崩溃或产生明显错误输出；一次科学计算返回错误结果，却可能产生看似合理、但以用户无法察觉的方式系统性错误的数字。浮点运算会引入舍入误差，这些误差会在数百万次操作中累积；病态问题会把小误差放大成大误差；不稳定算法会产生与真实解毫无关系的结果。数值分析这门数学纪律，就是为了刻画这些失败模式，并设计避免它们的算法。没有数值分析的科学计算，只是在进行一种“可能正确、也可能不正确，而且无从知道”的计算。
它与本指南其他部分有密集连接。线性代数（§2.3）是几乎所有数值计算的数学底层。微积分（§2.4），尤其是微分方程，定义了大量科学计算要解决的问题。高性能计算（§4.11）提供大型仿真所需要的计算规模。机器学习（§5.2–5.3）正在快速进入科学计算：昂贵仿真的神经 surrogate model，把物理约束纳入损失函数的 physics-informed neural networks，以及用于分子生物学的基础模型。这个交叉领域正在发生，并且还在增长。
前置知识：线性代数（§2.3）——达到实现深度，包括特征值算法和迭代求解器。微积分（§2.4）——微分方程是主要问题类别。概率与统计（§2.5）——蒙特卡洛方法。算法（§2.6）——数值方法的算法分析。
从弹道表到 AlphaFold
科学的计算需求早于电子计算机。最早的大规模数值计算是 New Mathematical Tables project，这是 1930 年代英国政府的一个项目，雇用人类“计算员”——使用机械计算器工作的数学家——来生成精确的对数表、三角函数表和其他工程师、导航员需要的数值表。这项工作的组织方式——任务分解、错误检查、系统验证——在任何机器能够执行这些工作之前，就发明了大规模数值计算的方法论。
ENIAC 于 1945 年投入运行，它专门为科学计算设计：计算火炮射击的弹道表，这需要针对数千种发射角、炮口速度和大气条件组合，求解弹丸运动的微分方程。它相对于人类计算员的速度优势大约是一千倍：人类计算员需要数周完成的计算，ENIAC 数小时就能完成。这个项目证明，数值仿真可以在某些用途上取代物理实验；随着计算机变得更强大，这一主张只变得更加有力。
早期电子计算机时代的数值分析围绕一个根本问题展开：浮点运算是新的，它的误差性质尚未被充分理解，而算法在有限精度运算中的行为，可能与它们在精确算术中的行为发生巨大偏离。John von Neumann 和 Herman Goldstine 1947 年对高斯消元法的分析，是最早的细致研究之一：他们表明，矩阵的条件数——大致是最大奇异值与最小奇异值之比——支配了线性系统解的准确性，而高斯消元中的部分主元选择可以控制误差放大。这一工作脉络使条件数成为数值分析中的核心概念，也使人们理解到算法稳定性——输入中的小扰动是否只会产生输出中的小扰动——是一种可以被分析的性质，而且许多看似自然的算法并不具备它。
1960 年代和 1970 年代最重要的发展，是至今仍在使用的线性代数数值算法。用于计算特征值的 QR 算法（Francis 和 Kublanovskaya 于 1961 年独立提出）用一种可证明稳定的迭代取代了幂法和 deflation；该算法的现代形式至今仍是稠密特征值计算的标准。QR 分解及其在线性最小二乘问题中的作用被分析清楚，并且通过 Householder 反射实现的数值稳定版本也被建立起来。Cleve Moler 的 MATLAB（1984）最初作为 LINPACK 和 EISPACK 线性代数库的包装器构建，使不是数值分析家的工程师和科学家也能使用这些算法。随后，Python 中的 NumPy 生态系统在 2000 年代由 Jim Hugunin、Travis Oliphant 等人贡献发展出来，把同样的算法带入一个免费、开源的框架，并成为标准科学计算平台。
有限元方法——通过把区域离散为小单元，并把解近似为分片多项式来求解偏微分方程——主要在 1950 年代和 1960 年代的结构工程中发展出来。Strang 和 Fix 1973 年的 An Analysis of the Finite Element Method 提供了数学分析，把工程实践与逼近理论连接起来。有限元方法如今是结构力学、流体力学和电磁仿真的主导方法，体现在商业代码（ANSYS、COMSOL、Abaqus）和开源框架（FEniCS、deal.II、OpenFOAM）中。那些模拟飞机机翼变形、模拟重建心脏动脉中的血流，以及计算微芯片中电磁场的代码，都使用了 Strang 和 Fix 分析过的数学框架。
分子动力学仿真——对原子和分子系统积分 Newton 运动方程——源自 1950 年代的 Metropolis 算法，并成为理解蛋白质折叠、药物—受体相互作用和材料性质的工具。计算挑战非常巨大：模拟一个包含几千个原子的蛋白质达到一微秒尺度，需要大约 10¹² 个积分步，而每一步都需要评估所有原子对之间的相互作用力。专用硬件（D.E. Shaw Research 的 Anton 机器）和算法优化（快速多极子方法、particle-mesh Ewald）已经扩展了可达到的时间尺度，但许多生物化学上有趣的生物过程，仍然发生在直接仿真无法到达的时间尺度上。
气候建模体现了大规模仿真的雄心和困难。第一批大气环流模型在 1960 年代由 GFDL 和 NCAR 开发；当前的地球系统模型会解析大气、海洋、陆地表面、冰和碳循环，并运行在世界上最大的超级计算机上。它们已经成功复现了二十世纪的气候观测，并为气候预测提供科学基础。它们也说明了核心方法论挑战：气候模型不是地球；它是一个近似，其质量取决于网格分辨率、亚网格过程参数化（云、对流、海冰），以及数值方法的准确性。量化这些近似有多重要——也就是不确定性量化——是一个活跃且困难的研究领域。
机器学习带来的转变，是有限元方法发展以来最重要的变化。AlphaFold2（DeepMind，2020）可以根据氨基酸序列预测许多蛋白质三维结构，准确性接近实验方法，极大推进了结构生物学家研究了五十年的中心问题。这并不是对物理仿真的替代；它是一个在 Protein Data Bank 上训练出的学习模型——该数据库包含超过 100,000 个通过实验确定的结构——并能泛化到新的序列。这里的教训不是机器学习优于仿真，而是二者之间的关系比任一范式的拥护者预期得更复杂：在训练数据充足的地方，学习模型可以做到仿真无法做到的事；在第一性原理物理理解至关重要的地方，仿真仍然是合适工具。
Physics-informed neural networks（PINNs）由 Raissi、Perdikaris 和 Karniadakis 于 2019 年提出，代表了另一种整合方式：把支配系统的微分方程纳入神经网络的损失函数，使训练出的网络在构造上近似满足物理规律。这种方法对逆问题非常有力——也就是从观测中推断模型参数——因为古典方法通常需要昂贵的反复正向仿真。神经 surrogate models 可以以原始仿真一小部分成本近似昂贵仿真，使不确定性量化和参数优化成为计算上可行的事情。科学机器学习领域正在快速发展自己的方法论、理论和工具。
数值分析、微分方程与科学机器学习
浮点运算与算法稳定性
科学计算中的每个数字都被表示为浮点值：二进制中的尾数和指数。IEEE 754 标准规定了这种表示：一个 64 位双精度浮点数有 52 位尾数，提供大约 15–16 位有效十进制数字，并且每次算术运算的结果都会被舍入到最接近的可表示值。这种舍入很小——相对误差大约是 10⁻¹⁶——但它会累积。
相消是最危险的误差来源：当两个几乎相等的量相减时，结果的相对误差可能任意大。对小 x 计算 x - sin(x) 就有问题，因为两项几乎相等；展开 sin(x) ≈ x - x³/6 + … 并使用多项式，会得到准确得多的结果。矩阵 A 的条件数——κ(A) = σₘₐₓ / σₘᵢₙ，即最大奇异值与最小奇异值之比——衡量线性系统 Ax = b 右端项中的误差会在解 x 中被放大多少。一个条件数为 10⁶ 的矩阵，即使中间结果被完美计算，也可能在解中丢失 6 位有效数字。
向后稳定性是数值算法的合适标准。如果一个算法计算出的结果，等于某个邻近输入的精确结果——仿佛输入在求解前受到一个很小相对扰动——那么这个算法就是向后稳定的。向后稳定算法累积的误差，不会超过问题条件数本身所允许的程度。带部分主元选择的高斯消元法，对几乎所有矩阵都是向后稳定的；没有部分主元选择则不是。通过 Householder 反射实现的 QR 分解是向后稳定的；经典 Gram-Schmidt 过程不是（改进 Gram-Schmidt 是）。
微分方程的数值方法
微分方程——常微分方程（一个自变量）和偏微分方程（多个自变量）——是连续物理过程的数学模型。数值方法通过在时间和/或空间上离散化，来近似它们的解。
对于 ODE，Runge-Kutta 这类显式方法会在每个时间步内部多个点上评估右端项，从而构造更高阶近似。经典四阶 Runge-Kutta 方法每步需要四次函数评估，并达到四阶精度——误差会随步长 h 以 h⁴ 下降。刚性 ODE——一些分量变化远快于另一些分量的系统——需要隐式方法，这类方法每一步都要求解一个非线性系统。Backward Euler 方法是一阶的，但无条件稳定；更复杂的隐式方法（BDF 方法、Radau 方法）则为刚性系统提供更高阶精度。
对于 PDE，离散化方法的选择取决于方程类型和几何结构。有限差分方法在结构化网格上用差商替代导数，产生稀疏线性系统。Laplacian ∇²u 被近似为 (u_{i+1,j} - 2u_{i,j} + u_{i-1,j})/h² + (u_{i,j+1} - 2u_{i,j} + u_{i,j-1})/h²；截断误差在每个方向上都是 O(h²)。有限元方法在非结构化网格上工作，通过寻找分片多项式空间中的近似解来求解 PDE 的弱形式。它们可以处理复杂几何，并允许在感兴趣特征附近进行局部网格细化。谱方法则把解展开到全局基函数中（Fourier 模式、Chebyshev 多项式），并且对光滑解实现谱收敛——误差随模式数量指数级衰减。
PDE 离散化产生的稀疏线性系统，会通过迭代方法求解：共轭梯度法（用于对称正定系统）、GMRES（用于一般非对称系统）以及它们的预条件版本。预条件——用一个近似逆来乘系统——对可扩展性至关重要：离散化 Laplacian 的原始条件数是 O(h⁻²)，这会使未预条件的迭代在细网格上慢到不可用。多重网格方法通过使用一系列更粗网格来加速细网格迭代的收敛，对椭圆型 PDE 达到 O(N) 复杂度；它们既是数值分析中算法 sophistication 最高的方法之一，也是实践中最重要的方法之一。
蒙特卡洛与不确定性量化
蒙特卡洛方法通过随机采样估计期望。基本估计器——从分布 p 中抽取 N 个独立样本 x₁, …, x_N，并计算 (1/N) Σ f(xᵢ) 作为 𝔼[f(X)] 的估计——以 O(1/√N) 速率收敛，且与维度无关，因此在高维积分中很有吸引力，因为确定性求积需要指数级数量的函数评估。方差缩减技术——重要性采样（从更接近 f·p 的分布中采样）、控制变量（减去一个与目标相关且期望已知的量）、准蒙特卡洛（使用低差异序列替代随机样本）——在设计良好的问题中，可以把有效方差降低 10–1000 倍。
Markov Chain Monte Carlo（MCMC）可以从只知道归一化常数以内形式的分布中采样，这对科学模型中的贝叶斯推断至关重要。Metropolis-Hastings 会从一个 proposal distribution 中提出移动，并根据目标密度比接受或拒绝；在温和条件下，链会收敛到目标分布。Hamiltonian Monte Carlo 利用梯度信息提出移动，使采样能够高效穿过高概率区域。MCMC 收敛诊断——Gelman-Rubin R̂ 统计量、有效样本量、trace plots——是标准方法论的一部分。
不确定性量化（uncertainty quantification，UQ）会把模型输入中的不确定性通过计算模型传播出去，从而量化输出中的不确定性。正向 UQ（给定不确定输入，输出分布是什么？）可以通过对输入分布进行蒙特卡洛采样来处理，也可以通过 polynomial chaos expansion 处理，后者把输入—输出映射近似为不确定参数上的多项式。逆向 UQ——根据观测校准模型参数——是一个贝叶斯推断问题，可以通过 MCMC 的后验采样来处理。随着科学计算从“产生一个数字”转向“产生一个带可靠性证书的数字”，UQ 变得越来越必要。
学习这一部分会改变什么
科学计算会改变实践者理解计算结果及其可靠性的方式。
第一个变化，是数值警觉性：在信任结果之前，自动询问条件数是多少、算法是否稳定、误差是否可接受。这会改变代码如何被编写——用已知解析解进行验证，监控残差和收敛速率，并意识到巨大的中间值可能在最终结果中造成灾难性相消。从未调试过数值不稳定的实践者，会制造出不稳定而不自知；内化了数值分析的实践者，则能看见警告信号。
第二个变化，是选择和应用合适方法的能力。Scipy 和 Julia 的 DifferentialEquations.jl 提供了几十种 ODE 求解器；选择正确求解器需要知道系统是否刚性、需要什么精度，以及计算预算是多少。用显式 Runge-Kutta 方法求解刚性系统，会需要比必要小许多数量级的步长，或者产生不稳定的垃圾结果。理解刚性和数值稳定性的实践者，可以正确做出选择；使用默认求解器的实践者，可能消耗比必要多出数个数量级的计算，或得到错误答案。
第三个变化，是科学方法论：把计算结果视为带有量化不确定性的证据，而不是事实。科学计算会产生数字；这些数字的科学价值，取决于我们是否理解其中包含什么误差、它们对模型假设有多敏感，以及做过什么验证。不进行不确定性量化，不用实验数据或解析解验证，就发表计算结果的实践者，是在发表没有质量证据的数字。可复现、经过验证、带有不确定性量化的计算纪律，是严格实验实践在计算领域中的对应物。
第四个变化，是能够有成效地接触科学机器学习。这个领域发展很快，对它提出的主张从真正重要到夸大不等。理解古典数值方法的实践者，能够评估哪些问题确实更适合用学习模型处理（高维、数据丰富、平滑），哪些问题更适合古典方法（有约束、数据稀疏、尖锐几何特征、需要物理可解释性），以及哪些问题受益于混合方法。
资源
书籍与文本
Trefethen 和 Bau 的 Numerical Linear Algebra（SIAM，1997）是该主题的必要文本，也是数学中写得最漂亮的技术书之一。它通过向后稳定性和条件数的视角发展求解线性系统、最小二乘和特征值问题的算法，提供每个算法背后的几何直觉，并以罕见的清晰度把所有内容连接到底层数学。尽管已经接近三十年，内容仍然不过时——它描述的算法仍是今天使用的算法。
Higham 的 Accuracy and Stability of Numerical Algorithms（第 2 版，SIAM，2002）是面向想要严格误差分析的数值分析家的综合参考。Trefethen 和 Bau 提供概念发展，Higham 则为每个主要算法提供完整证明和详细误差界。它要求很高，但具有权威性；当你需要知道某个算法是否向后稳定、误差常数是什么时，应把它作为参考。
Heath 的 Scientific Computing: An Introductory Survey（第 3 版，2018，classroom edition 免费）提供最宽泛且易读的入门：线性代数、插值、积分、ODE 和 PDE 方法、优化、蒙特卡洛。它在任何单一主题上都不如 Trefethen-Bau 深，但能够提供整个领域范围内的定位。
Hairer、Nørsett 和 Wanner 的 Solving Ordinary Differential Equations（两卷，Springer）是 ODE 数值方法的决定性参考——常用的每一种方法都会被发展、分析和比较，并提供关于何时应用它的深刻见解。第一卷覆盖非刚性方法；第二卷覆盖刚性方法和微分代数方程。
LeVeque 的 Finite Difference Methods for Ordinary and Partial Differential Equations（SIAM，2007）是进入有限差分方法的易读且严格入口，细致处理稳定性和收敛性。他的 Numerical Methods for Conservation Laws 覆盖流体动力学中的双曲 PDE 方法。
Nocedal 和 Wright 的 Numerical Optimization（第 2 版，Springer，2006）是优化方法的标准参考——梯度下降、Newton 方法、拟 Newton 方法、约束优化——提供该主题所需的理论基础和实践指导。Boyd 和 Vandenberghe 的 Convex Optimization（Cambridge，2004，可在 stanford.edu/~boyd/cvxbook 免费获取）覆盖凸优化这个特殊情形，在这里全局最优性可以得到保证。
对于科学机器学习，Brunton 和 Kutz 的 Data-Driven Science and Engineering（Cambridge，2022）覆盖机器学习与物理系统的交叉——降维、神经网络、dynamic mode decomposition、非线性动力学的稀疏识别——并与古典数值方法保持强连接。
| 书籍 | 作用 | 类型 |
|---|---|---|
| Trefethen & Bau, Numerical Linear Algebra | 数值线性代数经典文本 | 深入 |
| Higham, Accuracy and Stability of Numerical Algorithms（第 2 版） | 误差分析参考 | 参考 |
| Heath, Scientific Computing: An Introductory Survey（第 3 版，classroom ed. 免费） | 易读的广义入门 | 入门 |
| Hairer, Nørsett & Wanner, Solving Ordinary Differential Equations（两卷） | ODE 方法经典参考 | 参考 |
| LeVeque, Finite Difference Methods for ODEs and PDEs | PDE 有限差分 | 深入 |
| Strang & Fix, An Analysis of the Finite Element Method（第 2 版） | FEM 数学基础 | 深入 |
| Nocedal & Wright, Numerical Optimization（第 2 版） | 优化经典参考 | 参考 |
| Boyd & Vandenberghe, Convex Optimization（免费） | 凸优化 | 深入 |
| Robert & Casella, Monte Carlo Statistical Methods（第 2 版） | 蒙特卡洛基础 | 深入 |
| Brunton & Kutz, Data-Driven Science and Engineering | 科学机器学习 | 深入 |
| Golub & Van Loan, Matrix Computations（第 4 版） | 矩阵计算参考 | 参考 |
课程与讲座
MIT 18.335（Introduction to Numerical Methods，Steven Johnson，免费材料包含 Jupyter notebooks）覆盖数值线性代数、ODE 和 PDE 方法，以及优化，兼具科学计算所需的数学严谨性和实践导向。
Trefethen 的 Oxford 数值分析讲座（可通过 SIAM 获取，部分内容可在线免费获取）以异常有吸引力的方式呈现 Numerical Linear Algebra 中的材料。
Software Carpentry 和 Data Carpentry（softwarecarpentry.org，免费）提供计算工作流基础——可复现研究、科学代码的版本控制、数值软件测试——而这些正是领域科学家常常缺失的能力。其中关于 Python 科学计算和可复现工作流的材料尤其有价值。
MIT 18.085/18.086（Mathematical Methods for Engineers，Gilbert Strang，MIT OCW 免费）以应用为中心，清晰地覆盖微分方程、Fourier 分析、有限元和优化。
| 课程 | 平台 | 类型 |
|---|---|---|
| MIT 18.335 Introduction to Numerical Methods（免费） | MIT / GitHub | 入门 |
| MIT 18.085/18.086 Mathematical Methods for Engineers（免费） | MIT OCW | 入门 |
| Software Carpentry（免费） | softwarecarpentry.org | 入门 |
| Scientific Python Lectures（免费） | lectures.scientific-python.org | 入门 |
实践、工具与当前资料
NumPy 和 SciPy（免费）是 Python 科学计算的基础库。学习 SciPy 文档中的线性代数（scipy.linalg）、ODE 积分（scipy.integrate）、优化（scipy.optimize）和稀疏矩阵（scipy.sparse），并理解其底层算法——而不是把它们当成黑箱——是科学计算的实践入口。
Julia（免费，julialang.org）正越来越成为严肃科学计算的首选语言：速度接近 C，语法像 Python 一样可读，并且拥有提供数值方法的包生态系统（DifferentialEquations.jl、LinearAlgebra、Optim.jl）。DifferentialEquations.jl 包是所有语言中最全面的 ODE/DAE/SDE 求解器库。
FEniCS（免费，fenicsproject.org）和 deal.II（免费）分别用 Python 和 C++ 实现有限元方法。完成 FEniCS 教程——先求解 Poisson 方程，再求解热方程，然后解决一个简单结构力学问题——可以获得中等规模 PDE 仿真的动手经验。
BLAS 和 LAPACK（免费，netlib.org）是所有科学计算直接或间接依赖的基础数值线性代数库。理解它们如何被 NumPy、SciPy 和 Julia 调用，以及为什么正确调用它们对性能很重要，可以帮助理解数值计算栈。
Raissi、Perdikaris、Karniadakis 关于 PINNs 的论文（免费，arXiv）是 physics-informed neural networks 的奠基论文。通过一个一维 Burgers 方程的 PINN 实现——使用 PyTorch 把 PDE 残差编码进损失函数——可以把方法论具体化。
| 资源 | 平台 | 类型 |
|---|---|---|
| NumPy / SciPy documentation（免费） | numpy.org / scipy.org | 实践 |
| Julia + DifferentialEquations.jl（免费） | julialang.org | 实践 |
| SciML documentation（免费） | docs.sciml.ai | 参考 |
| FEniCS（免费） | fenicsproject.org | 实践 |
| FEniCSx tutorial（免费） | jsdokken.com/dolfinx-tutorial | 实践 |
| Raissi, Perdikaris & Karniadakis PINN papers（免费） | arXiv | 深入 |
| MIT Julia computing course（18.S191，免费） | computationalthinking.mit.edu | 入门 |
| Software Carpentry Python tutorials（免费） | software-carpentry.org | 入门 |
陷阱
| 陷阱 | 为什么会误导 | 更好的回应 |
|---|---|---|
| 没有数值分析地计算 | NumPy 和 SciPy 这样的库，让人可以在不了解算法准确性或稳定性性质的情况下运行大型数值计算。病态线性系统会悄悄产生错误答案；不稳定 ODE 积分器会产生看似收敛、但远离真实解的结果；高方差蒙特卡洛估计器给出的置信区间可能远宽于报告值。 | 把数值分析与科学计算实践一起学习。运行任何计算之后都要问：条件数是多少？算法是否向后稳定？结果是否随网格细化或更多蒙特卡洛样本而收敛？只要存在已知解析解，就用它们验证实现。 |
| 不理解刚性就使用默认求解器 | Scipy 的 solve_ivp 默认 RK45 求解器适合非刚性 ODE。把它用于刚性 ODE——化学反应网络，或具有快慢时间尺度的电路——会使步长比必要值小许多数量级，或者产生垃圾结果。失败并不总是戏剧性的；代码可能顺利运行完，并产生看似合理但错误的结果。 |
学会识别刚性系统：Jacobian 的特征值跨越许多数量级，或系统有快速瞬态和缓慢长期行为。对刚性系统使用隐式求解器（Radau、BDF，或 Julia 的 stiff algorithms）。通过与非常小固定步长的结果比较来验证。 |
| 混淆准确性与收敛性 | 一个算法可能收敛——随着步长减小产生同一个答案——但收敛到错误答案，因为算法不稳定，或问题病态到小的浮点误差主导了解。收敛性测试是必要的，但不足以保证准确性。 | 始终用独立参考解验证（有解析解时用解析解，否则使用昂贵得多的高精度计算）。为线性求解计算条件数。估计误差界，而不只是看残差。如果 A 病态，Ax = b 的解残差可以很小，但误差 ‖x - x*‖ 仍然很大。 |
| 忽视可复现性作为方法论问题 | 科学计算依赖随机种子（蒙特卡洛方法）、库版本（数值行为可能随版本不同而变化）、编译器 flags（浮点优化设置）和硬件。一个无法复现的结果不是科学结果；它可能是这些因素中任何一个造成的 artifact。 | 对所有科学代码使用版本控制。用 conda environments 或 requirements files 锁定库版本。记录随机种子。对任何将要发表或分享的结果，用容器（Docker 或 Singularity）归档精确软件环境。Software Carpentry 关于可复现研究的实践值得系统采用。 |
| 在古典方法更好时使用机器学习 | 对 physics-informed neural networks 和 neural surrogates 的兴奋，使一些实践者把机器学习应用到那些古典数值方法明显更好的问题上：光滑、特征明确、数据稀疏，而生成神经 surrogate 训练数据的成本比古典仿真更高的问题。 | 根据问题结构评估学习方法是否合理。对于具有准确物理模型的光滑问题，带有成熟误差界的古典 FEM 或谱方法通常更优。学习最有价值的场景是：物理模型不确定或昂贵，维度过高而无法使用古典网格，或有丰富观测数据可以替代物理模型。 |
| 发表没有不确定性量化的结果 | 一个计算结果是带误差的数字。只报告数字——没有误差界，没有敏感性分析，没有与独立数据的验证——会掩盖这个结果应当被信任到什么程度。人们越来越认识到，这在计算科学中是科学完整性问题，而不仅是方法论细节。 | 在任何具有科学意义的计算中加入不确定性量化：来自收敛分析的误差估计，不确定参数上的敏感性分析，与实验数据或独立计算结果的验证比较。和数字一起报告 error bars，并明确它们表示的是数值离散化误差、建模不确定性，还是统计采样误差。 |