English
DevOps concerns the cooperation and technical practices needed to change and operate software reliably. Development, operations, and other contributors need shared information about what is being released, what acceptable service means, and how to respond when a change fails. Responsibilities can remain specialized without making each handoff somebody else’s problem.
Operability is a property of the system being built: can people deploy it, determine whether it is healthy, diagnose a failure, and restore useful service? A program that runs correctly on one developer’s machine may still be difficult to configure, observe, or recover in production. These concerns belong in design and testing as well as in an operations manual.
Continuous delivery keeps a change ready for production release when the organization chooses. Continuous deployment automatically deploys changes that pass the required checks. Deployment puts a version into an environment; release makes a behavior available to users. Feature flags can separate those actions, but checks, permissions, and a response to failure are needed in either approach.
Prerequisites: Testing (§7.2), software process (§7.3), and basic operating-system and network concepts. Reliability engineering (§4.10) provides additional depth. A simple application deployed to one test machine is enough to learn the release process; container orchestration is not a prerequisite.
From Operational Handoffs to Shared Delivery Systems
The separation of programming and operations predates the web. Centralized computing installations needed people to schedule jobs, manage equipment, and recover from failures. These were real responsibilities. Problems arose when the people changing software and those operating it lacked shared knowledge, or were rewarded for incompatible local outcomes without a shared service goal.
Agile development and continuous integration shortened some development cycles, but a working build could still wait for a difficult operational handoff. Patrick Debois organized the first DevOpsDays in Ghent in 2009. The resulting movement gave a name and meeting place to efforts to improve this relationship. It did not invent automation or require every developer to become an expert in every operational specialty.
Humble and Farley’s Continuous Delivery (2010) described a deployment pipeline that accumulates evidence about a release candidate. The Phoenix Project and The DevOps Handbook communicated related ideas about work flow, feedback, and improvement. Their examples are useful for examining a delivery system, not evidence that one organizational arrangement fits every company.
Configuration-management and provisioning tools made intended infrastructure changes reviewable and repeatable. Containers made it easier to package an application with its user-space dependencies. These developments reduced some environmental differences; they did not eliminate host-kernel, processor, configuration, data, or external-service differences. A container image is an identifiable artifact, not a guarantee of identical behavior everywhere.
Site Reliability Engineering connected operation with explicit service objectives, automation, and learning from incidents. Platform engineering addresses repeated infrastructure work by providing shared capabilities that other teams can use. The practical question is whether such a platform reduces the work and waiting required to deliver a service. A portal or a large collection of tools is not enough if teams still need manual intervention for routine tasks.
The Deployment Pipeline, Infrastructure as Code, and Observability
The Deployment Pipeline and Safe Change
A deployment pipeline links a change to an identifiable artifact, checks that artifact, deploys it with controlled configuration, and observes the result. Put cheap, informative checks early so that common failures are found promptly. Longer integration, performance, or recovery tests may follow. The appropriate feedback time depends on what is being checked and how soon developers need the result.
Build once and promote the same artifact through the relevant environments. Record its source revision and dependency inputs, and identify it by a stable version or digest rather than a mutable label alone. Rebuilding for production can introduce differences from what was tested. Configuration and data still vary between environments and require separate validation. A passing staging test is evidence about the tested conditions, not a guarantee about production.
During a rolling deployment, old and new versions can run together. Database changes must allow that overlap. An expand-and-contract migration can first add a compatible field, then introduce code that can work with both representations, backfill and verify existing data, switch readers and writers, and only later remove the old field. Concurrent writes during backfill need an explicit synchronization strategy. Rolling back application code cannot restore a column that has already been removed.
Feature flags separate code deployment from behavior exposure. Keep ownership, defaults, evaluation rules, and removal plans explicit. Test important flag states, especially while old and new code coexist. A disabled user-facing flag does not necessarily stop a background job or reverse a schema migration. Multiple flags can interact, so adding flags also adds combinations to reason about.
Blue-green deployment maintains two application environments and changes traffic routing; a canary sends a limited share of traffic to a candidate version before broader rollout. Both require compatibility with shared state and a workable recovery path. Switching traffic back does not undo payments or data already written. A canary also needs representative traffic and enough observations to detect relevant problems; a quiet cohort can pass while an untested customer group fails.
Define the success and stop conditions before deployment. Compare candidate and baseline behavior, record the release version in diagnostic data, and choose whether failure should trigger a rollback, a forward fix, or temporary feature disablement. Small batches can simplify diagnosis, but release frequency alone does not make changes safe.
Infrastructure as Code and Configuration Management
Infrastructure as code records intended resources and configuration in files that can be versioned, reviewed, and applied through tools. Declarative provisioning describes the desired state; the tool compares it with observed resources and its recorded state to plan changes. Repeatability depends on controlled inputs, providers, permissions, and external conditions. An unchanged file can produce a different result if an unpinned dependency changes.
A plan is a proposal based on a particular observation, not a guarantee that applying it later will succeed unchanged. Review destructive replacements and resource ownership, coordinate concurrent changes, and preserve the state needed by the tool. Configuration files and provisioning state may contain sensitive values, so ordinary source history is not an appropriate place for every secret. A configuration backup also does not replace a database backup or a tested data restore.
Configuration management can update existing machines; immutable infrastructure instead replaces instances with a new version. Both approaches can be appropriate. Replacement reduces accumulated manual differences but still requires managing persistent data, connection draining, capacity, and compatibility while old and new instances overlap.
GitOps uses versioned desired state and automated reconciliation to bring a system toward it. Controllers such as Argo CD and Flux implement this pattern for Kubernetes. The repository records intent; a controller’s status and the running service reveal whether reconciliation succeeded. Reverting a commit changes desired configuration, but cannot automatically reverse an external side effect or recover deleted data. Detecting and explaining divergence is part of the workflow.
Measuring and Monitoring Production
Measure the behavior users rely on. A process being alive is different from a request succeeding, and a successful HTTP response is different from a correct reservation. Combine appropriate service indicators with logs and traces that help investigate failures. Include deployment version and request context where useful, while avoiding unnecessary collection of personal data or secrets.
An SLO specifies a target for a defined indicator and period. If the target is 99.9% successful eligible requests and there are 1,000,000 such requests in the period, the error budget is 1,000 unsuccessful requests. A time-based 99.9% availability target over 30 days instead permits 43.2 minutes of unavailability under continuous measurement. These are different denominators; traffic variation means they are not interchangeable.
Alerts should identify conditions requiring action. A high rate of budget consumption can justify rapid investigation, while a single diagnostic event may belong in a log rather than paging a person. Recovery plans need an owner, access to relevant information, and practice. An incident review should reconstruct the conditions and decisions, then choose specific improvements with follow-up.
Delivery measures (§7.3) describe how changes move; service indicators describe what users experience. Read them together. More deployments accompanied by worse reservation correctness are not success. A decline in reported incidents may also reflect weaker detection rather than better service, so examine how the measurements are collected.
What Studying This Changes
You learn to treat a release as a change to a running system with existing data and users. A successful build is only one piece of evidence. Compatibility during rollout, observation after release, and restoration after failure become part of the work item rather than tasks left implicit.
For a small project, build one versioned artifact and deploy it to a disposable test environment. Record how to configure it, check a representative request, and return to the previous version. Then add a compatible schema change and explain what would make rollback unsafe. Keep the exercise local and simple enough to understand every step.
A useful operational document states how to recognize a particular failure and what response is available. Test that response in the disposable environment. This reveals missing assumptions, such as needing an old artifact, a data backup, or a credential that the recovery procedure did not mention.
Resources
Books and Texts
Humble and Farley’s Continuous Delivery explains the pipeline as evidence about a release. The DevOps Handbook examines technical and organizational practices; The Phoenix Project provides a fictional setting for discussing work queues and handoffs. Morris’s Infrastructure as Code addresses managing infrastructure changes. Read examples in the context of their platform and scale rather than copying the entire tool stack.
| Book | Role | Type |
|---|---|---|
| Kim, Humble, Debois & Willis, The DevOps Handbook (2nd ed.) | Comprehensive practical implementation | Practice |
| Humble & Farley, Continuous Delivery | Deployment pipeline foundational text | Depth |
| Kim, Behr & Spafford, The Phoenix Project | Fictional introduction to delivery coordination | Entry |
| Forsgren, Humble & Kim, Accelerate | Empirical foundation | Entry |
| Morris, Infrastructure as Code (2nd ed.) | IaC principles and practices | Practice |
Courses, Papers, and Current Sources
The SRE Workbook chapter on canarying explains what a limited rollout can reveal. OpenGitOps describes desired state and reconciliation. Use project documentation for deployment tools and distinguish a configuration operation from restoration of application data.
| Resource | Platform | Type |
|---|---|---|
| State of DevOps reports (free) | dora.dev | Reference |
| Google SRE book and workbook (free) | sre.google | Depth |
Practice, Tools, and Current Sources
Build a small pipeline in a disposable environment before adopting an orchestrator. Learn one CI system, artifact identification, configuration handling, and a tested recovery path. OpenTofu documentation is one entry to declarative provisioning. Feature-flag, telemetry, and fault-injection tools are useful when you can state the behavior to observe; begin experiments on a local test system. Hosted services and cloud resources have separate usage terms and costs.
| Resource | Platform | Type |
|---|---|---|
| GitHub Actions / GitLab CI documentation | GitHub / GitLab | Practice |
| Terraform / OpenTofu documentation | developer.hashicorp.com / opentofu.org | Practice |
| Argo CD (free) | argoproj.io | Practice |
| Argo Rollouts (free) | argoproj.github.io | Practice |
| OpenTelemetry documentation (free) | opentelemetry.io | Practice |
| Backstage Software Catalog (free) | backstage.io | Practice |
| OpenFeature (free) | openfeature.dev | Practice |
| Flagsmith (open source) | flagsmith.com | Practice |
| Principles of Chaos Engineering (free) | principlesofchaos.org | Reference |
| Chaos Mesh / LitmusChaos (free) | CNCF / project docs | Practice |
Traps
| Trap | Why it misleads | Better response |
|---|---|---|
| Equating automation with correctness | A script can consistently deploy a wrong configuration. | Review inputs and observe the resulting service behavior. |
| Assuming containers erase environment differences | Kernels, configuration, data, and external dependencies still vary. | Identify artifacts and test relevant environmental assumptions. |
| Treating rollback as undo | Returning to old code does not undo writes or restore a removed schema. | Design compatibility and recovery before rollout. |
| Using deployment count as the goal | Counts can improve while user outcomes and reliability worsen. | Read delivery measures together with service behavior and useful outcomes. |
| Building a platform before understanding repeated needs | A large tool stack can create another support queue. | Start with a recurring task and measure whether teams can complete it more reliably. |
中文
DevOps 关注可靠地修改和运行软件所需的协作与技术实践。开发、运维及其他参与者需要共同了解发布内容、可接受的服务水平，以及变更失败后如何处理。职责可以保持专业分工，同时避免每次交接都把问题推给下一方。
可运维性是系统本身的性质：人们能否部署它、判断它是否健康、诊断故障并恢复有用服务？一个程序在开发者的机器上运行正确，到了生产环境仍可能难以配置、观察或恢复。这些问题应进入设计和测试，而不只是出现在运维手册中。
持续交付使修改在组织决定时能够发布到生产环境；持续部署则自动部署通过规定检查的修改。部署是把某个版本放入环境，发布是向用户开放某个行为。功能开关可以把两者分开，但两种方式都需要检查、权限控制和失败后的处理办法。
前置知识：测试（§7.2）、软件过程（§7.3），以及基本操作系统和网络概念。可靠性工程（§4.10）提供进一步知识。把简单应用部署到一台测试机器，就可以学习发布过程；容器编排不是前置要求。
从运维交接到共同维护交付系统
编程与运维的分工早于 Web。集中式计算设施需要有人安排作业、管理设备并从故障中恢复，这些都是真实职责。当修改软件的人与运行软件的人缺乏共同信息，或者只按相互冲突的局部指标得到奖励、没有共同服务目标时，问题才会出现。
敏捷开发和持续集成缩短了部分开发周期，但一个能够工作的构建结果，仍可能等待困难的运维交接。Patrick Debois 于 2009 年在根特组织了首届 DevOpsDays。由此发展的运动为改善这种关系提供了名称与交流场所；它没有发明自动化，也不要求每个开发者精通所有运维专业。
Humble 和 Farley 在 2010 年的《Continuous Delivery》中，描述了通过部署流水线不断积累发布候选版本证据的方法。《The Phoenix Project》和《The DevOps Handbook》传播了关于工作流动、反馈和改进的相关思想。这些例子有助于分析交付系统，却不能证明某一种组织安排适合所有公司。
配置管理和资源配置工具，使预期的基础设施变更可以审查和重复执行。容器使应用及其用户空间依赖更容易打包。这些发展减少了部分环境差异，却没有消除宿主机内核、处理器、配置、数据或外部服务的差异。容器镜像是可标识的制品，不是到处表现相同的保证。
站点可靠性工程把运行工作与明确的服务目标、自动化和事故学习联系起来。平台工程则通过提供其他团队可直接使用的共享能力，处理重复的基础设施工作。实际问题是：平台是否减少了交付服务所需的工作与等待？如果日常任务仍需人工介入，单有门户或一大批工具并不够。
Deployment Pipeline、Infrastructure as Code 与 Observability
部署流水线与变更控制
部署流水线把修改与可标识的制品关联起来，检查制品，使用受控配置部署，再观察结果。把成本低、信息量足够的检查放在前面，可以尽早发现常见失败；较长的集成、性能或恢复测试随后运行。合理的反馈时间取决于检查对象，以及开发者何时需要结果。
构建一次，再把同一个制品逐步推广到相关环境。记录源码版本和依赖输入，用稳定版本或内容摘要标识制品，而不只使用可变标签。生产部署前重新构建，可能引入与已测试版本不同的内容。配置和数据仍会随环境变化，需要单独检查。预发布环境测试通过，只提供关于被测条件的证据，不保证生产环境正确。
滚动部署期间，新旧版本可能同时运行，因此数据库变更必须允许这种重叠。扩展再收缩的迁移可以先增加兼容字段，再引入同时支持两种表示的代码，回填并核对已有数据，切换读写方，最后才删除旧字段。回填期间的并发写入需要明确同步策略。回滚应用代码，不能恢复已经删除的列。
功能开关把代码部署与行为开放分开。应明确负责人、默认值、求值规则和删除计划，检查重要的开关状态，尤其是新旧代码共存时。关闭面向用户的开关，并不一定停止后台任务或撤销数据结构迁移。多个开关之间还可能相互影响，因此增加开关也增加了需要推理的组合。
蓝绿部署保留两个应用环境并切换流量路由；金丝雀发布先把有限比例的流量送到候选版本，再决定是否扩大范围。两者都需要兼容共享状态，并有可行的恢复路径。把流量切回旧版，不会撤销已经发生的支付或数据写入。金丝雀还需要有代表性的流量，以及足以发现相关问题的观察量；安静的小群体可以通过，而尚未测试的客户群体仍可能失败。
部署之前应确定成功条件和停止条件。比较候选版与基线行为，在诊断数据中记录发布版本，并决定失败时回滚、向前修复，还是暂时关闭功能。小批量有助于简化诊断，但发布频率本身不会使修改安全。
Infrastructure as Code 与 Configuration Management
基础设施即代码把预期资源与配置写成可版本管理、审查和由工具应用的文件。声明式资源配置描述期望状态，工具把它与观察到的资源及自身记录的状态比较，生成变更计划。可重复性依赖于输入、提供方、权限和外部条件受到控制。如果未固定的依赖变化，即使文件没变，结果也可能不同。
计划是基于某次观察提出的方案，并不保证稍后应用时能够原样成功。需要审查破坏性替换和资源归属、协调并发变更，并保存工具所需状态。配置文件和资源状态可能含有敏感值，因此并非所有秘密都适合放进普通源码历史。配置备份也不能代替数据库备份和经过验证的数据恢复。
配置管理可以更新已有机器；不可变基础设施则用新版本替换实例。两种方式都可能适用。替换减少了长期人工修改造成的差异，但仍需要处理持久化数据、连接排空、容量，以及新旧实例重叠时的兼容性。
GitOps 使用有版本记录的期望状态和自动协调机制，使系统逐渐达到该状态。Argo CD、Flux 等控制器在 Kubernetes 中实现这种模式。仓库记录意图，控制器状态和实际服务则说明协调是否成功。撤销一次提交会改变期望配置，却不能自动逆转外部副作用或找回已删除数据。发现并解释偏差，也是流程的一部分。
测量与监控 Production
应测量用户依赖的行为。进程存活不同于请求成功，HTTP 响应成功也不同于预约结果正确。把合适的服务指标与有助于调查的日志、追踪结合起来；在有用时记录部署版本和请求上下文，同时避免不必要地收集个人数据或秘密。
服务水平目标（SLO）为明确的指标和时间范围规定目标。如果目标是在一个周期内，符合统计口径的请求有 99.9% 成功，且总共有 1,000,000 个这样的请求，那么错误预算就是 1,000 个失败请求。若采用基于时间的 99.9% 可用性目标，按连续测量的 30 天计算，则允许 43.2 分钟不可用。两者分母不同；流量变化意味着它们不能互换。
告警应指出需要采取行动的状况。错误预算消耗过快可能需要立即调查，单个诊断事件则可能适合记录日志，而不必叫醒值班人员。恢复计划需要负责人、相关信息的访问能力和实际演练。事故复盘应重建当时的条件与决策，再选择具体改进并跟踪落实。
交付指标（§7.3）描述修改如何流动，服务指标描述用户实际经历什么，两者应结合观察。如果部署增多却伴随预约正确性下降，就不是成功。报告的事故减少，也可能源于检测变弱，而不是服务变好，因此还要检查指标怎样收集。
学习这一部分会改变什么
你会把发布理解为对一个已有数据和用户的运行系统作出变更。构建成功只是其中一项证据；推出期间的兼容性、发布后的观察，以及失败后的恢复，都应成为工作项的一部分，而不是隐含地留给别人。
对于小型项目，可以构建一个带版本的制品，并部署到可丢弃的测试环境。记录怎样配置、怎样检查一个有代表性的请求，以及怎样恢复到上一版本。然后增加一次兼容的数据结构变更，解释什么情况会使回滚不再安全。保持练习局部而简单，确保每一步都能理解。
有用的运维文档应说明如何识别某种故障，以及可以怎样处理。在可丢弃环境中测试该处理办法，可以发现遗漏的假设，例如恢复需要旧制品、数据备份，或流程没有提到的凭据。
资源
书籍与文本
Humble 和 Farley 的《Continuous Delivery》把流水线解释为积累发布证据的过程。《The DevOps Handbook》讨论技术与组织实践；《The Phoenix Project》提供讨论队列与交接的虚构情境。Morris 的《Infrastructure as Code》处理基础设施变更。阅读示例时应考虑其平台与规模，而不照搬整套工具。
| 书籍 | 作用 | 类型 |
|---|---|---|
| Kim, Humble, Debois & Willis, The DevOps Handbook（第 2 版） | 综合实践实施 | 实践 |
| Humble & Farley, Continuous Delivery | Deployment pipeline 奠基文本 | 深入 |
| Kim、Behr、Spafford，The Phoenix Project | 用虚构案例介绍交付协调 | 入门 |
| Forsgren, Humble & Kim, Accelerate | 经验基础 | 入门 |
| Morris, Infrastructure as Code（第 2 版） | IaC 原则与实践 | 实践 |
课程、论文与当前资料
SRE 工作手册的金丝雀发布章节 解释有限范围推出可以发现什么。OpenGitOps 描述期望状态与协调过程。使用部署工具的项目文档，并区分配置操作与应用数据恢复。
| 资源 | 平台 | 类型 |
|---|---|---|
| State of DevOps reports（免费） | dora.dev | 参考 |
| Google SRE book and workbook（免费） | sre.google | 深入 |
实践、工具与当前资料
采用编排平台之前，先在可丢弃环境里搭建小型流水线。学习一种 CI 系统、制品标识、配置处理，以及经过测试的恢复路径。OpenTofu 文档 是学习声明式资源配置的一个入口。能够说明要观察什么行为时，再使用功能开关、遥测和故障注入工具；可以从本地测试系统开始试验。托管服务与云资源有各自的使用条件和费用。
| 资源 | 平台 | 类型 |
|---|---|---|
| GitHub Actions / GitLab CI 文档 | GitHub / GitLab | 实践 |
| Terraform / OpenTofu 文档 | developer.hashicorp.com / opentofu.org | 实践 |
| Argo CD（免费） | argoproj.io | 实践 |
| Argo Rollouts（免费） | argoproj.github.io | 实践 |
| OpenTelemetry documentation（免费） | opentelemetry.io | 实践 |
| Backstage Software Catalog（免费） | backstage.io | 实践 |
| OpenFeature（免费） | openfeature.dev | 实践 |
| Flagsmith（开源） | flagsmith.com | 实践 |
| Principles of Chaos Engineering（免费） | principlesofchaos.org | 参考 |
| Chaos Mesh / LitmusChaos（免费） | CNCF / project docs | 实践 |
陷阱
| 误区 | 为什么会误导 | 更好的做法 |
|---|---|---|
| 把自动化等同于正确 | 脚本可以稳定部署错误配置。 | 审查输入，并观察实际服务行为。 |
| 认为容器消除了环境差异 | 内核、配置、数据和外部依赖仍然可能不同。 | 标识制品，检查相关环境假设。 |
| 把回滚当成撤销全部影响 | 回到旧代码不会撤销写入，也不会恢复已删除的数据结构。 | 推出前设计兼容性与恢复办法。 |
| 把部署数量当成目标 | 计数可以改善，而用户结果和可靠性却下降。 | 把交付指标与服务行为、有用结果一起观察。 |
| 未了解重复需求就建设平台 | 庞大工具栈可能制造另一个支持队列。 | 从反复出现的任务开始，检查团队能否更可靠地完成它。 |