English
Debugging is not the activity of fixing bugs; it is the activity of locating, in a system whose behavior does not match expectations, the cause of the mismatch, under conditions of incomplete information. The distinction matters. Fixing is the outcome; locating is the process; and nearly all the difficulty of debugging is in the locating — once the true cause is found, the fix is often a few lines. The gap between novice and expert debuggers is not who is better at editing code, but who can more quickly and systematically narrow “something is wrong here” down to “this line, for this reason.” That narrowing is a trainable methodology, not a byproduct of talent or accumulated years.
The narrowing is a search, and treating it as a search is what separates methodical debugging from flailing. The cause of a bug could be in any line of code, any data state, any external interaction, any timing window. An experienced but methodologically chaotic engineer debugs by intuition and luck — staring at code, changing things at random, adding print statements everywhere, hoping a run comes out right. This works for simple bugs and fails completely for hard ones (concurrency, memory, cross-system, intermittent), because it does not systematically shrink the search space; it gropes blindly within an enormous space of possibilities. The engineer with methodology has a reliable procedure for any unfamiliar bug: make it reproducible, bisect the search space, form hypotheses and design experiments that can falsify them, distinguish symptom from root cause. This procedure works on the hardest bugs, which is exactly its value.
Debugging methodology is worth training as a distinct skill because its return is extraordinarily asymmetric. On any team, the time the strongest and weakest debuggers take to resolve the same difficult bug can differ by an order of magnitude. This gap does not close automatically with years of writing code — a person can write code for ten years and still debug by staring and random changes. It closes only through deliberate methodological training.
Prerequisites: Programming (§2.1) — you must read and write code to debug it. Operating systems (§4.2) — many of the hardest bugs live at the boundaries of processes, memory, and concurrency. Observability and reliability (§4.10) — debugging in production depends on the information the system exposes about itself.
The Shape of the Skill: From Staring to Systematic Search
Viewing debugging as a reasoning activity makes its internal structure clear: debugging is a search through a large space of possibilities. The bug’s cause could be in any line, any state, any interaction, any timing window. The novice’s failure modes almost all reduce to not treating this space as a search problem.
The evolution of debugging tools is, fundamentally, the progressive widening of how much of the search space is visible. The most primitive debugging is the print statement — insert output at suspected locations, observe where the program reaches and what the values are. Printf debugging remains widely used because it is simple, requires no setup, works in any environment, and is sufficient for the common question “did execution reach here, and what was the value?” Its limitation is that it requires you to guess correctly in advance where to print — and when you do not know where the bug is, that premise fails exactly when you most need it.
Interactive debuggers (gdb, lldb, IDE debuggers for each language) remove this limitation: they let you pause a running program, inspect complete state at any location, step through execution, and set conditional breakpoints. The debugger turns “I must guess the right print location in advance” into “I can explore at runtime.” Mastering the debugger — breakpoints, watchpoints (which pause when a variable is modified), call-stack inspection, conditional breakpoints — is the single highest-leverage skill between novice and intermediate, and a large number of engineers never truly learn it, remaining stuck on print statements their entire careers.
More modern tools extend visibility further. Time-travel debuggers (rr, various record-replay tools) record a complete execution and let you execute backward through the recording — revolutionary for intermittent bugs, because you only need to capture the bug once and can then replay and analyze it repeatedly, rather than praying it recurs. Observability infrastructure (§4.10) extends debugging into production: structured logging, distributed tracing, and continuous profiling let you debug a problem that cannot be reproduced locally and that spans multiple services. What these tools share is the steady expansion of how much of the system’s internal state is visible to you — and the difficulty of debugging is fundamentally the degree to which internal state is invisible.
But tools are only amplifiers. An engineer who does not understand bisection, handed a time-travel debugger, will still rummage blindly through the recording. Methodology is the substance; tools are the means of executing it. This ordering — methodology first, tools as its instruments — is the organizing principle of the skill.
The Core Debugging Workflow
Mature debugging follows a procedure. It is not a rigid checklist — experienced debuggers skip, merge, and parallelize its steps — but each step corresponds to a real epistemological function, and understanding those functions is the key to mastering debugging.
Make the Bug Reproducible
The first step in debugging is almost always to establish a reliable reproduction: a definite set of conditions under which the bug appears consistently. Novices often skip this, rushing to fix, but it is the foundation of everything that follows. A bug that cannot be reproduced cannot be verified as fixed — you change something, the bug does not appear, but you cannot tell whether you fixed it or it merely failed to trigger this time.
Reproducibility comes in degrees. Best is deterministic reproduction: every run with this set of steps triggers the bug. Next is high-probability reproduction. Hardest is the intermittent bug: it appears in production but cannot be reproduced in a controlled environment. Converting an intermittent bug into a reproducible one is often the hardest part of debugging, and once accomplished, the rest becomes routine work. Techniques for increasing the reproduction rate include narrowing to a minimal reproducible example (stripping away all irrelevant code until only the minimum that triggers the bug remains — a process that often exposes the cause by itself), increasing trigger frequency (raising concurrency, adding load, injecting delays to widen timing windows), and recording production inputs for offline replay.
The minimal reproducible example deserves emphasis. Stripping a bug in a complex system down to a self-contained program of a few dozen lines is one of the highest-information-density activities in debugging — each part removed while the bug persists eliminates a large region of the search space, and each part removed that makes the bug disappear precisely localizes a relevant factor. It is also the only effective form in which to ask others (colleagues, open-source maintainers) for help: no one can debug your whole codebase, but anyone can look at a minimal reproducible example.
Bisection: The Core of Debugging Methodology
If a debugger could be taught only one thing, it would be bisection. The bug’s cause lives in some space — a stretch of code, a span of commit history, a set of inputs, a sequence of state changes. The core idea of bisection is that every observation should cut this space in half.
Bisecting the execution path: the bug occurs somewhere in the path from input to erroneous output. Inspect the state at the midpoint — if the midpoint state is already wrong, the cause is in the first half; if the midpoint state is still correct, the cause is in the second half. One check eliminates half. For a bug spanning a thousand lines of execution path, bisection means about ten checks to localize, where reading linearly from the top might mean reading hundreds of lines.
Bisecting commit history: this is the function of git bisect, and it is among its most underused. “This worked fine last week and is broken today” — between last week’s good version and today’s broken one are dozens or hundreds of commits. git bisect lets you bisect this history: it checks out the middle commit, you test whether it is good or bad, and it selects the next half accordingly, until it pinpoints the exact commit that introduced the bug. For a history of 200 commits, about 8 tests find the culprit. git bisect run can drive the whole process with an automated test script. Many engineers have never used bisect, reading diffs by eye when faced with a regression — a substantial efficiency loss.
Bisecting the input space: the bug is triggered by some complex input. Halve the input — if the bug remains, the removed half is irrelevant; if the bug disappears, the cause is in the removed half. This is exactly how a minimal reproducible example is constructed, and the principle behind delta-debugging tools that automate it.
Bisection is the core of debugging methodology because it converts the paralyzing state of “I have no idea where the cause is” into a finite, mechanical procedure. You do not need insight; you do not need to guess right; you only need, at each step, to ask a question that cuts the space in half.
The Hypothesis-Verification Loop
Once bisection has localized the rough region, debugging enters a hypothesis-driven phase. You form a specific hypothesis about the cause — “I think this value becomes null here, because that initialization branch did not execute” — and design an observation that can falsify it.
The key is falsification, not confirmation. Novices tend to design observations that confirm their hypothesis, so that seeing a result consistent with their guess makes them believe they have found the cause, when in fact they merely failed to check whether it is also consistent with other explanations. The mature debugger designs observations that discriminate between hypotheses: if my hypothesis is right I should see X; if it is some other cause I should see Y; so I observe whether it is X or Y. Each such observation genuinely advances understanding rather than merely accumulating supporting evidence.
This is isomorphic to the scientific method. Hypothesis, prediction, experiment, revise the hypothesis based on the result — debugging is experimental science conducted on a program. The engineer who treats it as experimental science is systematically faster than the engineer who treats it as staring at code and meditating.
Distinguish Symptom from Root Cause
The final step, and the one most often shortchanged: is what you found the symptom or the root cause? A program crashes at a particular line — but that line is often only where the problem becomes visible, not where it originates. A null pointer is dereferenced at line 500 and crashes, but why is it null? Perhaps because a branch at line 200 did not execute, which did not execute because a configuration was read wrong at line 50. Line 500 is the symptom; line 50 is the root cause.
Fixing at the symptom — adding a null check so it does not crash — makes that particular crash disappear, but the root cause remains and will resurface in another form, now masked by your null check and harder to find. Distinguishing symptom from root cause requires repeatedly asking “why,” until you reach a level where fixing it will prevent the whole class of problem from recurring. This is the per-bug application of the “five whys” of incident analysis from §4.10.
When to stop asking is a matter of judgment. In principle one could ask all the way back to the initial conditions of the universe; in practice, the root cause is the level at which “fixing it reliably prevents this class of problem and lies within the team’s control.” That judgment is itself a product of experience, but the habit of asking is one that can be built from the start.
AI-Assisted Debugging
AI has become a genuinely useful debugging aid, and using it well fits the methodology rather than replacing it. A language model can explain an unfamiliar error message, suggest likely causes for a described symptom, propose hypotheses to test, and explain what a confusing stack trace means. For the early orientation phase of debugging — understanding what an error means and generating candidate hypotheses — this accelerates the work, especially for errors in unfamiliar territory.
The risk is specific. An AI’s suggested cause is a hypothesis, not a diagnosis — it is a plausible guess based on the description you provided, and it is offered with the same fluent confidence whether it is right or wrong. The debugger who treats the AI’s suggestion as the answer, and fixes what the AI says is wrong without verifying, may “fix” something that was not the cause, leaving the real bug in place and possibly introducing a new one. The danger is greatest with the hard bugs — the intermittent, the concurrent, the subtle — where the AI is least likely to be right and the cost of a wrong fix is highest.
The productive use folds AI into the hypothesis-verification loop: treat the AI’s suggestions as additional hypotheses to test, not as conclusions to act on. The AI proposes candidate causes; you verify them with the same falsifying observations you would apply to your own hypotheses. The AI accelerates hypothesis generation, which is genuinely useful, while the verification discipline — confirming the cause before fixing it — remains exactly as it was. Used this way, AI is a fast source of hypotheses; treated as a source of answers, it produces confident wrong fixes.
What Changes With Mastery
Mastery of debugging methodology changes not only the speed of fixing bugs but the engineer’s whole posture toward unknown failures.
The first change is composure in the face of unfamiliar bugs. The engineer without methodology, faced with an incomprehensible bug, is gripped by anxiety — not knowing where to begin, they stare and change things at random and make it worse. The engineer with methodology has a reliable procedure: first make it reproducible, then bisect, then hypothesize and verify. They do not need to understand the bug at the outset; the methodology carries them toward it step by step. This “I have a method” composure is itself a large efficiency gain, because anxiety-driven random changes are the largest time sink in debugging.
The second change is writing code with debuggability in mind. The engineer who has debugged enough production bugs writes code that prepares for future debugging: structured logging at key points, failing fast and explicitly (rather than continuing with corrupted state until a crash far away), keeping functions small and pure so they can be tested in isolation, avoiding designs that make state hard to observe. Debuggability becomes a design constraint rather than an afterthought.
The third change is a healthy skepticism toward “I’m sure I understand this code.” Debugging teaches, over and over, that the bug is almost always in the place you were certain could not be wrong. The judgment “this code is definitely fine” is exactly where bugs hide, because you do not check what you are certain of. The mature debugger learns to treat their own certainty as a hypothesis to verify rather than a conclusion that licenses skipping a check. This epistemological humility is one of the most valuable byproducts of debugging experience.
The fourth change is the ability to ask for help effectively. Whether an engineer can get useful help — from colleagues, from Stack Overflow, from open-source maintainers — depends almost entirely on whether they can reduce the problem to a minimal reproducible example. The engineer who has this skill offers, when asking for help, a small program others can run and see the problem in directly; the engineer who lacks it offers “my code does not work, here is my whole repository,” which is nearly impossible to help with. The ability to reduce a problem is both the core of debugging alone and the prerequisite for drawing on others.
Resources
Debugging is one of the few skills where the marginal return of reading is lower than that of deliberate practice. But a few books and tools are worth specific investment.
Books and Texts
David Agans’s Debugging: The 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems (2nd ed., 2021) is the most practical book on debugging methodology. Its nine rules — understand the system, make it fail, quit thinking and look, divide and conquer, change one thing at a time, keep an audit trail, check the plug, get a fresh view, and if you didn’t fix it it ain’t fixed — are nearly obvious, which is exactly why they are valuable: they are what novices systematically violate. The book is short and high-return, because it makes explicit what most people only half-learn through years of painful experience.
Andreas Zeller’s Why Programs Fail: A Guide to Systematic Debugging (2nd ed., 2009) is the more academic and systematic treatment. It grounds debugging in the scientific method and develops delta debugging (automated input bisection), program slicing, and causal reasoning in detail. For readers who want to understand the principles behind the heuristics, this is the right depth. Zeller’s online The Debugging Book (debuggingbook.org, free) extends it with executable Jupyter notebooks teaching automated debugging techniques.
Kernighan and Pike’s The Practice of Programming (1999), though not solely about debugging, contains chapters on debugging and testing that distill the practical wisdom of two masters of the Unix tradition with concision and authority.
| Book | Role | Type |
|---|---|---|
| Agans, Debugging: The 9 Indispensable Rules (2nd ed.) | The most practical methodology text | Entry |
| Zeller, Why Programs Fail (2nd ed.) | Systematic, scientific debugging methodology | Depth |
| Zeller, The Debugging Book (free) | Executable text on automated debugging | Depth |
| Debugging chapters in Kernighan & Pike, The Practice of Programming | Distilled practical wisdom | Practice |
Practice, Tools, and Projects
The interactive debugger for your primary language is a tool to genuinely master, not dabble in. For C/C++, gdb or lldb; for Python, pdb and the IDE-integrated debugger; for JavaScript, the browser DevTools debugger and Node’s inspector; for Java, the IDE debugger. What to master is not just breakpoints but: conditional breakpoints (pause only when a condition holds), watchpoints (pause when a variable is modified — the tool for “who changed this value?” bugs), call-stack navigation, and runtime expression evaluation. Investing a day or two in systematically learning your language’s debugger returns value for years.
git bisect is the standard tool for locating regression bugs and is underused by most engineers. Introduce a regression in one of your own projects and locate it with git bisect (including the automated git bisect run mode with a test script) to build the muscle memory fastest.
rr (Mozilla’s record-replay debugger, Linux, free) implements time-travel debugging: record one execution, then replay and execute backward repeatedly. For intermittent and concurrency bugs it turns “pray it recurs” into “capture it once.” It has a learning curve but is transformative for the hardest class of bugs.
AddressSanitizer, ThreadSanitizer, and UndefinedBehaviorSanitizer (compile-time instrumentation built into GCC/Clang, free) automatically detect memory errors, data races, and undefined behavior. For C/C++ engineers, enabling sanitizers by default in testing catches a large class of bugs that would otherwise require arduous debugging. Valgrind provides similar memory-error detection without recompilation.
| Resource | Platform | Type |
|---|---|---|
| Interactive debugger (gdb/lldb/pdb/DevTools) | Language-specific | Practice |
| git bisect (including bisect run) | Git | Practice |
| rr time-travel debugger (free) | rr-project.org | Practice |
| AddressSanitizer / ThreadSanitizer (free) | GCC / Clang | Practice |
| Valgrind (free) | valgrind.org | Practice |
| Perfetto (free) | perfetto.dev | Practice |
| Julia Evans debugging and profiling zines (paid; some free articles) | wizardzines.com | Auxiliary |
Traps
| Trap | Why it misleads | Better response |
|---|---|---|
| Starting to fix before reproducing | Without a reliable reproduction, you cannot verify that a fix worked — the bug does not reappear, but you cannot tell whether you fixed it or it merely failed to trigger. On intermittent bugs, this means repeatedly “fixing” the same bug, each time believing it is done, until it returns. | Establish reproducibility before fixing. Converting an intermittent bug into a reproducible one is often the hardest step, but it is the foundation for everything after. Time invested in narrowing reproduction conditions and raising the trigger rate is fully recovered when verifying the fix. |
| Shotgun print debugging | Inserting print statements everywhere and staring at the resulting flood of output is failing to treat debugging as a search. It occasionally works on simple bugs and drowns you in self-generated output on complex ones, and each wrong guess about where to print wastes a compile-run cycle. | Replace the shotgun with bisection. Every observation — print, breakpoint, or state inspection — should be designed to cut the search space in half. Ask “is the cause before or after this point?” rather than “let me print everything suspicious and see.” |
| Never learning the debugger, staying on prints forever | Large numbers of engineers never truly learn an interactive debugger and rely on print statements for life. Prints answer “did execution reach here, what is the value?” but cannot efficiently answer “who changed this value, and when?” — which is the heart of hard bugs. Watchpoints, conditional breakpoints, and runtime state inspection have no print equivalent. | Spend a day or two systematically learning your primary language’s debugger, focusing on conditional breakpoints and watchpoints. “Who changed this variable?” is seconds of work with a watchpoint and possibly hours with prints. This one-time investment returns value for years. |
| Fixing the symptom, not the root cause | Adding a defensive check at the crash site to stop the crash makes that specific symptom disappear, but the root cause remains and will recur in another form — now masked by your check and harder to find. | Keep asking “why” until you reach the level at which fixing it reliably prevents the whole class of problem. The crash is at line 500, but why is the value wrong? Where did the bad value enter? Trace the causal chain back to its source rather than patching where the symptom surfaces. |
| Reading diffs by eye to find regressions | Faced with a “worked last week, broken now” regression, reading through dozens or hundreds of commit diffs by eye does work that git bisect finishes automatically in minutes. |
Use git bisect for regressions. Give it a good version, a bad version, and a test for good-versus-bad, and it bisects the history to pinpoint the introducing commit. With git bisect run it is fully automated. |
| Treating your own certainty as license to skip a check | Bugs almost always hide where you were certain nothing could go wrong, because you do not check what you are sure of. “I’ve looked at this code a hundred times, it’s definitely fine” is precisely where the bug is. | Treat your certainty as a hypothesis to verify, not a conclusion that licenses skipping a check. When bisection points at code you are “sure is fine,” that is exactly the code to inspect most carefully. Follow the evidence to wherever it points, regardless of how certain you are. |
| Treating an AI’s suggested cause as the diagnosis | An AI’s proposed cause is a plausible hypothesis based on your description, offered with the same confidence whether right or wrong. Fixing what the AI says is wrong without verifying may patch something that was not the cause, leaving the real bug in place — and the danger is greatest on the hard bugs where the AI is least likely to be right. | Fold AI into the hypothesis-verification loop: treat its suggestions as additional hypotheses to test, not conclusions to act on. Let it accelerate hypothesis generation, which is genuinely useful, while keeping the verification discipline — confirm the cause with a falsifying observation before fixing it — exactly as it was. |
中文
调试不是修复 bug 的活动；它是在一个行为不符合预期的系统中，在信息不完整的条件下，定位这种不匹配原因的活动。这个区别很重要。修复是结果；定位是过程；而调试几乎所有困难都在定位上——一旦真正原因被找到，修复往往只是几行代码。新手调试者与专家调试者之间的差距，并不在于谁更擅长编辑代码，而在于谁能更快、更系统地把“这里有东西不对”缩小到“是这一行，原因是这个”。这种缩小范围的能力是一种可以训练的方法论，不是天赋或年份自然累积出来的副产品。
这种缩小范围的过程是一种搜索；把它当作搜索来处理，正是系统化调试与胡乱试错之间的区别。一个 bug 的原因可能在任何一行代码、任何一种数据状态、任何一次外部交互、任何一个时序窗口中。一个有经验但方法混乱的工程师，可能会凭直觉和运气调试——盯着代码看，随机修改，到处加打印语句，希望某次运行结果正确。这对简单 bug 有时有效，但面对困难 bug 时会彻底失效，例如并发、内存、跨系统、间歇性问题，因为它并没有系统地缩小搜索空间；它只是在巨大的可能性空间中盲目摸索。有方法论的工程师，则对任何陌生 bug 都有可靠流程：让它可复现，二分搜索空间，提出假设并设计能证伪假设的实验，区分症状和根因。这套流程能处理最困难的 bug，而这正是它的价值。
调试方法论值得作为一项独立能力训练，因为它的回报极其不对称。在任何团队中，最强和最弱调试者解决同一个困难 bug 所花的时间，可能相差一个数量级。这个差距不会因为写代码年限增长而自动消失——一个人可以写十年代码，仍然靠盯代码和随机修改来调试。它只会通过刻意的方法论训练缩小。
前置知识：编程（§2.1）——必须能读写代码，才能调试代码。操作系统（§4.2）——许多最困难的 bug 存在于进程、内存和并发的边界上。可观测性与可靠性（§4.10）——生产环境中的调试依赖系统暴露出的关于自身的信息。
这项能力的形状：从盯代码到系统化搜索
把调试看作一种推理活动，会让它的内部结构变得清楚：调试是在一个巨大的可能性空间中搜索。bug 的原因可能位于任何一行、任何状态、任何交互、任何时序窗口。新手的失败模式，几乎都可以归结为没有把这个空间当作搜索问题来处理。
调试工具的演进，本质上是不断扩大搜索空间中“可见部分”的过程。最原始的调试是打印语句——在怀疑的位置插入输出，观察程序是否到达那里，以及变量值是什么。printf 式调试至今仍然广泛使用，因为它简单、不需要配置、适用于任何环境，并且足以回答常见问题：“执行到这里了吗？值是什么？”它的限制在于，你必须预先正确猜出应该在哪里打印——而当你不知道 bug 在哪里时，这个前提恰恰会在你最需要帮助时失效。
交互式调试器（gdb、lldb、各语言的 IDE 调试器）移除了这个限制：它们让你能够暂停正在运行的程序，检查任意位置的完整状态，逐步执行，并设置条件断点。调试器把“我必须预先猜对打印位置”变成“我可以在运行时探索”。掌握调试器——断点、观察点（当某个变量被修改时暂停）、调用栈检查、条件断点——是从新手到中级之间最高杠杆的单项能力；大量工程师一生都没有真正学会它，始终停留在打印语句阶段。
更现代的工具进一步扩展了可见性。时间旅行调试器（rr 以及各种 record-replay 工具）会记录一次完整执行，并允许你在记录中向后执行——这对间歇性 bug 是革命性的，因为你只需要捕捉一次 bug，之后就可以反复回放和分析，而不必祈祷它再次出现。可观测性基础设施（§4.10）则把调试扩展到生产环境中：结构化日志、分布式追踪和持续性能分析，使你能够调试那些无法在本地复现、并且跨越多个服务的问题。这些工具的共同点，是持续扩大你能看见的系统内部状态；而调试的困难，本质上取决于内部状态在多大程度上不可见。
但工具只是放大器。一个不理解二分法的工程师，即使拿到时间旅行调试器，也仍然会在记录中盲目翻找。方法论才是实质；工具只是执行方法论的手段。这个顺序——方法论优先，工具作为工具——是这项能力的组织原则。
核心调试工作流
成熟调试遵循一套流程。它不是僵硬清单——有经验的调试者会跳过、合并或并行化其中步骤——但每一步都对应真实的认识论功能。理解这些功能，是掌握调试的关键。
让 bug 可复现
调试的第一步几乎总是建立可靠复现：找到一组明确条件，使 bug 能稳定出现。新手常常跳过这一步，急于修复，但它是一切后续工作的基础。一个无法复现的 bug，也就无法验证是否真的被修复——你改了某些东西，bug 没有出现，但你无法判断是你真的修好了，还是它这次只是没有触发。
可复现性有程度之分。最好的是确定性复现：按这组步骤每次运行都会触发 bug。其次是高概率复现。最困难的是间歇性 bug：它出现在生产环境中，却无法在受控环境中复现。把间歇性 bug 转化为可复现 bug，常常是调试中最困难的部分；一旦完成，剩下的工作就会变成常规。提高复现率的技术包括：缩小到最小可复现示例，也就是移除所有无关代码，只保留触发 bug 的最小部分——这个过程本身往往就会暴露原因；提高触发频率，例如增加并发、增加负载、注入延迟以扩大时序窗口；以及记录生产环境输入，用于离线回放。
最小可复现示例值得特别强调。把复杂系统中的一个 bug 剥离成几十行的自包含程序，是调试中信息密度最高的活动之一——每移除一部分而 bug 仍然存在，就排除了搜索空间中的大片区域；每移除一部分而 bug 消失，就精确定位了一个相关因素。它也是向他人——同事、开源维护者——求助时唯一有效的形式：没有人能调试你的整个代码库，但任何人都可以看一个最小可复现示例。
二分：调试方法论的核心
如果只能教调试者一件事，那就应该是二分。bug 的原因存在于某个空间中——一段代码、一段提交历史、一组输入、一串状态变化。二分的核心思想是：每一次观察都应该把这个空间切成两半。
对执行路径二分：bug 发生在从输入到错误输出的某条路径上。检查中点处的状态——如果中点状态已经错误，原因就在前半段；如果中点状态仍然正确，原因就在后半段。一次检查就排除一半。对于跨越一千行执行路径的 bug，二分意味着大约十次检查就能定位；如果从头线性阅读，可能要读几百行。
对提交历史二分：这就是 git bisect 的作用，而且它极其被低估。“上周还正常，今天坏了”——在上周的正常版本和今天的坏版本之间，可能有几十或几百个 commit。git bisect 让你能够二分这段历史：它 checkout 中间的 commit，你测试它是好还是坏，然后它据此选择下一半，直到定位引入 bug 的确切 commit。对于 200 个 commit 的历史，大约 8 次测试就能找到罪魁祸首。git bisect run 还可以用自动化测试脚本驱动整个过程。许多工程师从未使用过 bisect，面对回归问题时仍然靠肉眼阅读 diff——这是相当大的效率损失。
对输入空间二分：bug 由某个复杂输入触发。把输入切成两半——如果 bug 仍然存在，被移除的那一半就是无关的；如果 bug 消失，原因就在被移除的那一半中。这正是构造最小可复现示例的方法，也是自动化 delta debugging 工具背后的原则。
二分之所以是调试方法论的核心，是因为它把“我完全不知道原因在哪里”这种瘫痪状态，转化为一个有限的、机械的过程。你不需要灵感；不需要猜对；只需要在每一步提出一个能把空间切半的问题。
假设—验证循环
一旦二分把问题定位到大致区域，调试就进入假设驱动阶段。你提出一个关于原因的具体假设——“我认为这个值在这里变成了 null，因为那个初始化分支没有执行”——然后设计一个能够证伪它的观察。
关键是证伪，而不是确认。新手倾向于设计能确认自己假设的观察；看到一个结果与猜测一致，就以为找到了原因。实际上，他们只是没有检查这个结果是否也与其他解释一致。成熟调试者设计的是能够区分不同假设的观察：如果我的假设是对的，我应该看到 X；如果原因是别的，我应该看到 Y；于是我观察到底是 X 还是 Y。每一次这样的观察，都会真正推进理解，而不是只是积累支持性证据。
这与科学方法同构。提出假设，做出预测，进行实验，根据结果修正假设——调试就是在程序上进行的实验科学。把调试看作实验科学的工程师，会系统性地快于把调试看作盯着代码沉思的工程师。
区分症状和根因
最后一步，也最常被敷衍的一步：你找到的是症状，还是根因？程序在某一行崩溃，但那一行往往只是问题变得可见的地方，而不是问题起源的地方。一个 null 指针在第 500 行被解引用并导致崩溃，但它为什么是 null？也许是因为第 200 行的某个分支没有执行；而这个分支没有执行，是因为第 50 行读取配置出了错。第 500 行是症状；第 50 行才是根因。
在症状处修复——加一个 null 检查让程序不崩溃——会让这次特定崩溃消失，但根因仍然存在，并且会以另一种形式重新出现，如今还被你的 null 检查掩盖，变得更难找到。区分症状和根因，要求不断追问“为什么”，直到到达这样一个层级：修复它能防止这一整类问题再次出现。这就是 §4.10 事件分析中 “five whys” 在单个 bug 上的应用。
什么时候停止追问，是判断问题。原则上，你可以一路追问到宇宙初始条件；实践中，根因是这样一个层级：修复它能够可靠防止这一类问题，并且位于团队控制范围内。这种判断本身来自经验，但追问的习惯从一开始就可以建立。
AI 辅助调试
AI 已经成为真正有用的调试辅助工具；良好使用它，是把它纳入方法论，而不是让它替代方法论。语言模型可以解释陌生错误信息，可以为某个症状提出可能原因，可以提出待测试的假设，也可以解释令人困惑的栈追踪意味着什么。对于调试的早期定向阶段——理解错误含义、生成候选假设——它会加速工作，尤其是在不熟悉领域中的错误上。
风险也很具体。AI 提出的原因是假设，不是诊断——它是基于你提供的描述给出的貌似合理猜测；无论正确还是错误，它都会以同样流畅的自信表达出来。如果调试者把 AI 的建议当作答案，不验证就修复 AI 所说的问题，可能会“修复”一个并非原因的东西，真正的 bug 仍然存在，甚至还可能引入新的 bug。危险最大之处，正是困难 bug：间歇性问题、并发问题、细微问题。在这些地方，AI 最不可能正确，而错误修复的代价最高。
有效用法，是把 AI 纳入假设—验证循环：把 AI 的建议当作额外待测试假设，而不是可以直接行动的结论。AI 提出候选原因；你用对待自己假设时同样的证伪观察去验证它们。AI 加速假设生成，这确实有用；但验证纪律——先确认原因，再修复——完全不变。这样使用时，AI 是快速假设来源；把它当作答案来源时，它会产生自信的错误修复。
熟练之后会发生什么变化
掌握调试方法论，改变的不只是修复 bug 的速度，也会改变工程师面对未知故障时的整体姿态。
第一种变化，是面对陌生 bug 时保持镇定。没有方法论的工程师，在面对难以理解的 bug 时会被焦虑抓住——不知道从何开始，于是盯着代码看、随机修改，并把情况变得更糟。有方法论的工程师则有可靠流程：先让它可复现，然后二分，然后假设并验证。他们不需要一开始就理解 bug；方法论会一步一步把他们带向理解。这种“我有方法”的镇定，本身就是巨大的效率收益，因为由焦虑驱动的随机修改，是调试中最大的时间黑洞。
第二种变化，是写代码时考虑可调试性。调试过足够多生产 bug 的工程师，会写出为未来调试做好准备的代码：在关键点使用结构化日志，快速且明确地失败，而不是带着损坏状态继续运行直到远处崩溃；保持函数小而纯，使其可以被隔离测试；避免让状态难以观察的设计。可调试性变成设计约束，而不是事后才想起的东西。
第三种变化，是对“我确定我理解这段代码”保持健康怀疑。调试一再教人：bug 几乎总在你确信不可能出错的地方。判断“这段代码绝对没问题”，恰恰就是 bug 藏身之处，因为你不会检查自己确信的东西。成熟调试者会把自己的确信当作待验证的假设，而不是作为跳过检查的结论。这种认识论谦逊，是调试经验最有价值的副产品之一。
第四种变化，是能够有效求助。工程师能否从同事、Stack Overflow、开源维护者那里获得有用帮助，几乎完全取决于他能否把问题缩小到最小可复现示例。具备这项能力的工程师，在求助时提供的是一个小程序，别人可以运行并直接看到问题；缺少这项能力的工程师提供的是“我的代码不工作，这是我的整个仓库”，这几乎无法帮助。缩小问题的能力，既是独立调试的核心，也是借助他人的前提。
资源
调试是少数阅读边际收益低于刻意练习边际收益的能力之一。不过，有几本书和一些工具值得专门投入。
书籍与文本
David Agans 的 Debugging: The 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems（第 2 版，2021）是最实践化的调试方法论书。它的九条规则——理解系统，让它失败，停止空想并观察，分而治之，一次只改一件事，保留审计轨迹，检查插头，换一个新视角，如果你没修好那就不是修好了——几乎显而易见，而这正是它们有价值的原因：这些就是新手会系统性违反的东西。这本书很短，回报很高，因为它把多数人只在多年痛苦经验中半学半懂的东西明确说出来。
Andreas Zeller 的 Why Programs Fail: A Guide to Systematic Debugging（第 2 版，2009）是更学术也更系统的处理。它把调试建立在科学方法上，并详细发展 delta debugging（自动化输入二分）、程序切片和因果推理。对于想理解启发式方法背后原则的读者，这是合适深度。Zeller 的在线 The Debugging Book（debuggingbook.org，免费）进一步用可执行 Jupyter notebooks 教授自动化调试技术。
Kernighan 和 Pike 的 The Practice of Programming（1999）虽然并不只讨论调试，但其中关于调试和测试的章节，以 Unix 传统两位大师的简洁和权威，提炼了实践智慧。
| 书籍 | 作用 | 类型 |
|---|---|---|
| Agans，Debugging: The 9 Indispensable Rules（第 2 版） | 最实践的方法论文本 | 入门 |
| Zeller，Why Programs Fail（第 2 版） | 系统化、科学化的调试方法论 | 深入 |
| Zeller，The Debugging Book（免费） | 关于自动化调试的可执行文本 | 深入 |
| Kernighan & Pike，The Practice of Programming 中的调试章节 | 提炼过的实践智慧 | 实践 |
实践、工具与项目
你主要语言的 交互式调试器 是应该真正掌握、而不是浅尝辄止的工具。C/C++ 对应 gdb 或 lldb；Python 对应 pdb 和 IDE 集成调试器；JavaScript 对应浏览器 DevTools 调试器和 Node inspector；Java 对应 IDE 调试器。需要掌握的不只是断点，还包括：条件断点（仅在条件成立时暂停）、观察点（变量被修改时暂停——这是处理“谁改了这个值？”类 bug 的工具）、调用栈导航，以及运行时表达式求值。花一两天系统学习自己语言的调试器，会在多年中持续回报。
git bisect 是定位回归 bug 的标准工具，但多数工程师使用不足。可以在自己的项目中有意引入一个回归，并用 git bisect 定位它，包括使用带测试脚本的自动化 git bisect run 模式，这是最快建立肌肉记忆的方法。
rr（Mozilla 的 record-replay 调试器，Linux，免费）实现时间旅行调试：记录一次执行，然后反复回放并向后执行。对于间歇性和并发 bug，它把“祈祷它再次出现”变成“只要捕捉一次”。它有学习曲线，但对于最困难的一类 bug 具有变革性。
AddressSanitizer、ThreadSanitizer 和 UndefinedBehaviorSanitizer（GCC/Clang 内置的编译时插桩工具，免费）会自动检测内存错误、数据竞争和未定义行为。对于 C/C++ 工程师，在测试中默认启用 sanitizer，可以捕捉大量否则需要艰难调试的 bug。Valgrind 则在不重新编译的情况下提供类似的内存错误检测。
| 资源 | 平台 | 类型 |
|---|---|---|
| 交互式调试器（gdb/lldb/pdb/DevTools） | 各语言专用 | 实践 |
git bisect（包括 bisect run） |
Git | 实践 |
| rr 时间旅行调试器（免费） | rr-project.org | 实践 |
| AddressSanitizer / ThreadSanitizer（免费） | GCC / Clang | 实践 |
| Valgrind（免费） | valgrind.org | 实践 |
| Perfetto（免费） | perfetto.dev | 实践 |
| Julia Evans 调试和性能分析 zine（付费；部分免费文章） | wizardzines.com | 辅助 |
陷阱
| 陷阱 | 为什么会误导 | 更好的回应 |
|---|---|---|
| 在复现之前就开始修复 | 如果没有可靠复现，就无法验证修复是否有效——bug 没有再次出现，但你无法判断是你修好了它，还是它这次只是没有触发。对于间歇性 bug，这意味着你会反复“修复”同一个 bug，每次都以为完成了，直到它再次回来。 | 在修复之前先建立可复现性。把间歇性 bug 转化为可复现 bug，常常是最难的一步，但它是一切后续工作的基础。投入在缩小复现条件和提高触发率上的时间，会在验证修复时全部收回。 |
| 霰弹式打印调试 | 到处插入打印语句，然后盯着大量输出看，这是没有把调试当作搜索来处理。它偶尔能解决简单 bug，但在复杂问题上会让你淹没在自己制造的输出中；每一次错误猜测打印位置，都会浪费一次编译—运行循环。 | 用二分替代霰弹。每一次观察——打印、断点、状态检查——都应该被设计成把搜索空间切成两半。问“原因在这个点之前还是之后？”，而不是“我把所有可疑地方都打印出来看看”。 |
| 永远不学调试器，一辈子停留在打印语句 | 大量工程师从未真正学会交互式调试器，一生依赖打印语句。打印可以回答“执行到这里了吗？值是什么？”，但不能高效回答“谁改了这个值，什么时候改的？”——而这正是困难 bug 的核心。观察点、条件断点和运行时状态检查，没有打印语句的等价替代。 | 花一两天系统学习主要语言的调试器，重点学习条件断点和观察点。“谁改了这个变量？”用观察点可能是几秒钟的事，用打印可能是几小时的事。这项一次性投入会在多年中持续回报。 |
| 修复症状，而不是根因 | 在崩溃位置加防御性检查来阻止崩溃，会让特定症状消失，但根因仍然存在，并会以其他形式复发——如今还被你的检查掩盖，更难找到。 | 持续追问“为什么”，直到到达这样一个层级：修复它能可靠防止整个类别的问题。崩溃发生在第 500 行，但这个值为什么错？坏值从哪里进入？沿着因果链追溯到源头，而不是在症状出现的位置打补丁。 |
| 靠肉眼读 diff 找回归 | 面对“上周能用，现在坏了”的回归问题，逐个肉眼阅读几十或几百个 commit diff，是在做 git bisect 几分钟内就能自动完成的工作。 |
对回归问题使用 git bisect。给它一个好版本、一个坏版本，以及一个判断好坏的测试，它会二分历史并定位引入问题的 commit。使用 git bisect run 时，这个过程可以完全自动化。 |
| 把自己的确信当作跳过检查的许可证 | bug 几乎总是藏在你确信不会出错的地方，因为你不会检查自己确信的东西。“我看过这段代码一百次，它绝对没问题”恰恰就是 bug 所在之处。 | 把自己的确信当作待验证假设，而不是可以跳过检查的结论。当二分把你带到一段你“确信没问题”的代码时，那正是最需要仔细检查的代码。跟随证据，无论它指向哪里，即使它指向你最确信的地方。 |
| 把 AI 建议的原因当作诊断 | AI 提出的原因，是基于你描述生成的貌似合理假设；无论正确还是错误，它都会以同样的自信表达出来。不验证就修复 AI 所说的问题，可能只是在修补一个并非原因的东西，让真正的 bug 继续存在——而危险最大之处，正是 AI 最不可能正确的困难 bug。 | 把 AI 纳入假设—验证循环：把它的建议当作额外待测试假设，而不是可以直接行动的结论。让它加速假设生成，这确实有用；但保持验证纪律不变——在修复之前，用能证伪的观察确认原因。 |