English
A robot is a machine that perceives its environment, decides what to do, and acts physically upon the world. The three-part structure — sense, plan, act — sounds simple and is extraordinarily difficult. Sensors are noisy and incomplete; planning under uncertainty over continuous spaces is computationally demanding; and acting on physical systems produces contact forces, dynamic responses, and failure modes that simulation cannot fully predict. Most importantly, the three components interact: perception errors propagate into planning, planning assumptions shape what sensing is relevant, and actions change the environment that must next be perceived. A robot is not a computer that happens to move; it is a feedback system whose correctness depends on the whole loop.
The mathematical foundations are distinct from general programming. Kinematics and dynamics describe how robot bodies move — transformations in 3D space using Lie group theory, equations of motion using Lagrangian or Newton-Euler mechanics. Control theory describes how to drive a physical system to a desired state despite model uncertainty and disturbances. Probabilistic state estimation provides the tools — Kalman filters, particle filters, SLAM — for inferring where the robot is and what its environment contains from uncertain sensor data. Motion planning provides the algorithms for finding feasible trajectories from current state to goal state in high-dimensional configuration spaces. Each of these is a substantive mathematical subject, not a technicality to be implemented via library calls.
The connection to other chapters in this guide is substantial. Computer vision (§5.3 includes relevant deep learning) provides the perception backbone for modern systems. Reinforcement learning (§5.4) has become a primary training method for complex manipulation and locomotion policies. Computer organization (§4.1) and operating systems (§4.2) provide the computational substrate on which robotic systems run. The connection to AI safety (§5.7) is growing: as robots gain autonomy, the question of what they are optimizing for and how to specify the right objectives becomes practically urgent.
Prerequisites: Linear algebra (§2.3) — rigid body transformations use Lie groups built on linear algebra. Probability and statistics (§2.5) — probabilistic estimation is central. Calculus (§2.4) — dynamics and control depend on differential equations. Algorithms (§2.6) — motion planning is algorithmic at its core.
From Programmable Machines to Learning Bodies
The history of robotics is a series of hard-won lessons about the gap between what machines can be specified to do and what the physical world requires.
The first industrial robot, Unimate, was installed at a General Motors plant in 1961. It performed spot welding on automobile bodies, repeating the same programmed trajectory thousands of times per day. Unimate was not autonomous — it had no sensors and no feedback; it simply replayed a fixed sequence of motions. This was enough for factory automation where the environment was carefully controlled and the task was simple and repetitive. For the next three decades, industrial robotics developed along this controlled-environment axis: precisely engineered cells where every component arrived in a known position, allowing programmed trajectories to work reliably.
The question of what an autonomous robot in an uncontrolled environment would look like was addressed by SRI’s Shakey, developed between 1966 and 1972. Shakey could navigate a room with obstacles using a camera and LIDAR-like sensors, plan routes to goals, and execute plans while monitoring for unexpected conditions. The planners who built Shakey were optimistic: the AI winter’s first phase was still ahead, and it seemed plausible that general intelligence was just a few years of refinement away. Shakey’s successor environment was the STRIPS planner, which introduced the logical planning representation that classical AI (§5.1) would develop for decades. What Shakey demonstrated was that perception and planning in even simplified environments was far harder than it appeared, and that combining them reliably required engineering effort that far exceeded the algorithmic insight alone.
The probabilistic turn in robotics, formalized in Thrun, Burgard, and Fox’s Probabilistic Robotics (2005), arose from a recognition that classical planners assumed the world was known and commands were executed perfectly — both assumptions badly wrong in practice. A robot navigating a real corridor encountered doors that were sometimes open and sometimes closed, people who moved unpredictably, and small orientation errors that compounded over distance until it was hopelessly lost. Treating position and environment state as probability distributions, updated by Bayesian filtering as sensor data arrived, produced dramatically more robust navigation. Simultaneous Localization and Mapping (SLAM) — building a map of the environment while simultaneously localizing within that map — became the enabling technology for mobile robots operating in unknown environments. The SLAM problem, which is simultaneously a perception problem, an estimation problem, and a computational problem, drove substantial theoretical and algorithmic work through the 2000s.
The DARPA Grand Challenge in 2004 and 2005 demonstrated what sufficiently engineered systems could do. In 2004, no vehicle completed the 240-kilometer desert course; in 2005, five vehicles finished it. Sebastian Thrun’s Stanford team with Stanley, and the Carnegie Mellon team with Sandstorm and H1ghlander, had built systems that combined GPS, LIDAR, stereo vision, and learned terrain classification into functional autonomous driving in structured off-road environments. The 2007 Urban Challenge pushed further into city driving with traffic rules. These projects established that autonomous driving was engineering-feasible in bounded environments, and they launched a decade of investment that eventually produced Waymo, Cruise, and a dozen other autonomous vehicle programs.
The outcomes of that decade were more complicated than the founders anticipated. Waymo’s technology has proven capable of autonomous operation in Phoenix, Arizona — a relatively flat, well-mapped city with favorable weather — but has shown limited ability to generalize to more complex environments without extensive additional engineering. The “long tail” of edge cases — the unusual situations that appear rarely during development but are encountered with some frequency over millions of miles of driving — has proven resistant to systematic solutions. Every edge case addressed reveals new edge cases. This pattern — solid performance in familiar conditions, brittleness at the distribution boundary — characterizes robotic systems generally and has been a recurring theme across decades.
Boston Dynamics’ BigDog (2005) and its successors introduced a different problem: dynamic locomotion. Wheeled robots could be controlled with classical techniques; legged robots, whose stability required continuously active control at frequencies of hundreds of hertz, demanded different approaches. Marc Raibert’s work at MIT in the 1980s had established the theoretical foundations for dynamic legged locomotion using simplified spring-mass models; Boston Dynamics translated this into practical hardware through hydraulic actuation, fast computation, and extensive engineering. Spot, the commercialized successor, can navigate rough terrain, open doors, and recover from being kicked — capabilities that require the control loop to operate correctly across the full range of physical interactions.
The deep learning revolution reached robotics around 2016, with Sergey Levine and colleagues demonstrating that end-to-end policies — convolutional networks that mapped directly from image pixels to motor torques — could learn robot manipulation tasks from thousands of real trials without hand-engineering the intermediate representations. This was striking because the received wisdom had been that end-to-end learning for robotics required too much real-world data and produced policies too brittle for practical use. The demonstration that these policies worked, for specific tasks, in laboratory conditions, launched a large research program.
The sim-to-real transfer problem immediately became central. Training neural network policies in simulation was much faster and cheaper than training on real robots, but policies trained in simulation often failed on real hardware because the simulation was imperfect — sensor characteristics, physical contacts, and control dynamics were all approximated. Domain randomization — training in environments with randomized physical parameters, so that the real world appeared as a sample from the training distribution — proved partially effective. OpenAI’s Dactyl system (2019) demonstrated robotic dexterous manipulation of a Rubik’s cube using domain-randomized training, achieved after remarkable engineering effort. The degree to which sim-to-real transfer works remains a key research question.
The current frontier is the convergence of large-scale pre-training with robotics. Google DeepMind’s RT-1 (2022) demonstrated that a robot policy trained on a large and diverse dataset of robot demonstrations could generalize substantially more broadly than policies trained on small datasets for specific tasks. RT-2 (2023) extended this by jointly training a vision-language model with robot action data, producing policies that could respond to natural-language instructions and reason about novel task categories. The approach draws directly on the scaling laws and foundation model methodology of §5.5; whether the same pattern of emergent capabilities will appear for robot manipulation as it appeared for language is an active and open empirical question.
Geometry, Estimation, and the Control Loop
Kinematics and Dynamics: The Mathematical Body
Before a robot can be controlled, its motion must be described mathematically. Kinematics maps joint angles to positions and orientations of the robot’s links in space, without regard to the forces that cause the motion. Forward kinematics — given joint angles, find end-effector pose — is always solvable by composing transformation matrices. Inverse kinematics — given desired end-effector pose, find joint angles — is generally not unique and often has no closed-form solution, requiring iterative numerical methods or analytical solutions for specific robot geometries.
Rigid body transformations are represented as elements of the Special Euclidean group SE(3): a rotation matrix R ∈ SO(3) and a translation vector t ∈ ℝ³, combined as a 4×4 homogeneous transformation matrix. The Lie group structure of SO(3) and SE(3) — with their corresponding Lie algebras so(3) and se(3) as tangent spaces at the identity — provides the mathematical framework for composition of transformations, computing velocities and their integration, and formulating optimization problems over the space of rigid body configurations. Lynch and Park’s Modern Robotics develops this Lie group framework systematically and is the most readable introduction to modern kinematic formulations.
Dynamics extends kinematics to include forces and torques. The Lagrangian formulation — constructing the kinetic and potential energy functions and applying the Euler-Lagrange equations — produces equations of motion in terms of joint angles, velocities, and accelerations. The Newton-Euler formulation provides an equivalent recursive algorithm suitable for efficient computation. The mass matrix, Coriolis terms, and gravity vector that appear in the equations of motion are the physical quantities that control design must account for.
Estimation: Knowing Where You Are
State estimation is the problem of inferring the robot’s state — position, orientation, velocity, and the state of relevant environment features — from sensor measurements that are noisy, incomplete, and asynchronous.
The Kalman filter provides the optimal linear estimator: given a linear system with Gaussian noise, it produces the minimum-variance estimate of the state. The prediction step propagates the state estimate and uncertainty through the dynamics model; the update step incorporates sensor measurements by weighting the prior estimate and the measurement according to their relative uncertainties. The Extended Kalman Filter (EKF) handles nonlinear dynamics by linearizing around the current estimate; the Unscented Kalman Filter (UKF) and particle filters provide alternatives for highly nonlinear systems or non-Gaussian distributions.
SLAM is the simultaneous estimation of the robot’s trajectory and a map of the environment. Classical formulations — EKF-SLAM, particle-filter SLAM — suffered from computational cost that scaled quadratically with the number of map features. Graph-based SLAM reformulates the problem as a nonlinear least-squares optimization over a pose graph, with efficient sparse solvers (g2o, GTSAM) exploiting the sparsity structure of the problem. Visual-inertial odometry combines camera measurements with IMU (inertial measurement unit) data to estimate both trajectory and a sparse visual map in real time, enabling localization without a pre-built map. Modern systems like ORB-SLAM3 achieve robust localization and mapping in real time on modest hardware.
Control: Closing the Loop
Control theory provides the tools for driving a physical system to a desired state despite model uncertainty and external disturbances.
Proportional-Integral-Derivative (PID) control is the workhorse of industrial control: the control signal is a weighted sum of the tracking error, its integral, and its derivative. PID is simple, robust, and effective for many single-joint control problems; tuning the three gains for satisfactory performance is standard industrial practice. For multi-joint robots, the joint interactions produce cross-coupling terms that PID cannot handle without additional feedforward terms.
Model Predictive Control (MPC) optimizes a sequence of control inputs over a finite horizon by solving an optimization problem at each time step, applying the first input, and resolving the problem at the next step. MPC explicitly handles constraints on states and inputs, accounts for the coupled multi-body dynamics, and can incorporate rolling horizon plans. The computational cost scales with the horizon length and the complexity of the dynamics model; the development of fast embedded optimization solvers (ACADO, OSQP, CasADi) has made MPC practical for real-time robotic control.
Reinforcement learning for control learns a policy that maps states to actions by optimizing expected cumulative reward through interaction with the environment. For contact-rich manipulation and dynamic locomotion, RL has produced policies that outperform hand-designed controllers, particularly for tasks where the dynamics are complex and difficult to model analytically. The challenge is sample efficiency — learning a stable bipedal walking policy in simulation typically requires millions of simulation steps — and sim-to-real transfer. Hybrid approaches that combine RL with classical control structure (adding a learned residual to a model-based baseline) often achieve better performance than either alone.
What Studying This Changes
Robotics changes how practitioners understand computation’s relationship to the physical world.
The first change is the instinct to reason about uncertainty and feedback. Software systems that run without feedback from the physical environment can be designed as open-loop programs. Robotic systems cannot; they must close the loop. The practitioner trained in probabilistic state estimation asks, for any system that must act on the world: what is the state, how uncertain is the estimate, and how does each action change both the state and the uncertainty? This framing applies beyond robotics to any feedback-controlled system.
The second change is calibrated understanding of the sim-to-real gap. Simulation is fast, cheap, and predictable; the real world is slow, expensive, and full of surprises. The practitioner who has grappled with sim-to-real transfer treats simulation results as evidence, not as truth, and designs experiments to characterize the distribution shift rather than assuming it away. This discipline transfers to any application where a model trained in one distribution is deployed in another.
The third change is appreciation for integration as the hard problem. Perception, estimation, planning, and control each work in isolation in well-designed components. Making them work together, with errors in one component propagating through the stack, with timing and communication constraints, and under adversarial physical conditions — this is the engineering challenge that consumes most of the effort in real robotic systems. The practitioner trained in robotics treats integration as a first-class concern rather than an implementation detail.
The fourth change is physical intuition. Understanding how forces, contacts, inertia, and feedback behave in real systems — how a robot arm overshoots when gains are too high, how a legged robot’s stability depends on its center of mass position, how sensor noise accumulates into position error — requires direct experience with physical systems. This intuition is not directly transferable from reading, and practitioners who have built and debugged physical robotic systems reason about physical dynamics differently from those who have only simulated them.
Resources
Books and Texts
Lynch and Park’s Modern Robotics: Mechanics, Planning, and Control (2017, free at modernrobotics.org) is the right starting point for most learners. Its Lie group formulation of kinematics and dynamics is the modern mathematical language of the field; its motion planning and control chapters provide a unified treatment; and the freely available textbook with accompanying software makes it accessible. The accompanying Coursera course by the same authors provides structured pacing.
Thrun, Burgard, and Fox’s Probabilistic Robotics (2005) remains the foundational reference for state estimation and SLAM, despite its age. The particle filter, the EKF-SLAM, and the graph-based SLAM formulations it develops are the mathematical substrate of every mobile robot navigation system. Chapter 2 (Gaussian Filters) through Chapter 11 (SLAM) constitute the essential probabilistic robotics curriculum.
Siciliano, Sciavicco, Villani, and Oriolo’s Robotics: Modelling, Planning and Control (2009) is the comprehensive classical reference — deep and mathematically careful, covering kinematics, dynamics, control, and motion planning with rigor. For practitioners who want the foundational treatment at depth, it is the right primary text; for newcomers, Lynch and Park is more accessible.
LaValle’s Planning Algorithms (2006, free at planning.cs.uiuc.edu) is the foundational text on motion planning, covering configuration spaces, sampling-based planning (RRT, PRM), feedback motion planning, and planning under uncertainty. Its scope is broader than robotics — it covers planning problems in general — and the depth is appropriate for serious study.
Murray, Li, and Sastry’s A Mathematical Introduction to Robotic Manipulation (1994, free at murray.cds.caltech.edu) develops the Lie group framework for robot kinematics and manipulation mechanics that Lynch and Park build on. Despite its age, its mathematical development is clear and complete.
| Book | Role | Type |
|---|---|---|
| Lynch & Park, Modern Robotics (free) | Contemporary foundational text | Entry |
| Thrun, Burgard & Fox, Probabilistic Robotics | Probabilistic estimation; SLAM | Depth |
| Siciliano et al., Robotics: Modelling, Planning and Control | Comprehensive classical reference | Reference |
| LaValle, Planning Algorithms (free) | Motion planning foundational text | Depth |
| Murray, Li & Sastry, Mathematical Introduction to Robotic Manipulation (free) | Lie group kinematics and manipulation | Depth |
| Spong, Hutchinson & Vidyasagar, Robot Modeling and Control (2nd ed.) | Robot dynamics and control | Depth |
| Siegwart, Nourbakhsh & Scaramuzza, Introduction to Autonomous Mobile Robots (2nd ed.) | Mobile robotics foundation | Depth |
Courses and Lectures
Lynch and Park’s Coursera Robotics Specialization (free to audit) accompanies the textbook and provides the most structured freely available path through modern robotics.
MIT 6.832 (Underactuated Robotics, Russ Tedrake, free at underactuated.mit.edu) covers the intersection of dynamics, control, and motion planning for underactuated systems — robot legs, manipulation with limited actuation — with lecture videos, problem sets, and code notebooks. The course is intellectually demanding and represents the research frontier.
Stanford CS223A (Introduction to Robotics, free lecture notes and videos) covers classical robotics — kinematics, dynamics, workspace analysis, control — with mathematical depth.
Berkeley CS287 (Advanced Robotics, Pieter Abbeel, materials available) covers probabilistic robotics, planning, and learning-based robotics at graduate depth.
Cyrill Stachniss’s robotics lecture series on YouTube (free) covers mobile robotics, SLAM, and probabilistic state estimation with exceptional clarity. These are the most useful video resources for the probabilistic foundations.
| Course | Platform | Type |
|---|---|---|
| Lynch & Park Coursera Robotics Specialization (free to audit) | Coursera | Entry |
| MIT 6.832 Underactuated Robotics (free) | underactuated.mit.edu | Depth |
| Stanford CS223A Introduction to Robotics (free) | Stanford / YouTube | Entry |
| Cyrill Stachniss SLAM lectures (free) | YouTube | Practice |
Practice, Tools, and Current Sources
ROS 2 (Robot Operating System, free, ros.org) is the standard middleware for robotic software. Its publish-subscribe communication model, hardware abstraction, and extensive library of tools (rviz for visualization, rosbag for data recording, move_it for manipulation planning) make it the practical infrastructure for most non-industrial robotic development. Working through the official ROS 2 tutorials provides the entry to robotic software development.
Gymnasium (formerly OpenAI Gym, free) provides standardized environments for RL research on locomotion and manipulation. The MuJoCo physics engine (now free for research) provides high-quality simulation for contact-rich tasks.
GTSAM (free, Georgia Tech Smoothing and Mapping) implements the factor graph optimization at the core of modern SLAM. Working through GTSAM’s Python API and implementing a simple pose-graph SLAM system makes the optimization structure concrete.
The RT-1 and RT-2 papers (free, robotics-transformer.github.io) are the primary sources for the foundation model approach to robotic manipulation. Reading them alongside the §5.5 material on foundation models provides the bridge between language model scaling and robotics.
Physical hands-on work is irreplaceable. Building a simple mobile robot — even with a Raspberry Pi, a motor controller, and cheap encoders — and implementing PID drive control, then a basic EKF for dead-reckoning, then a simple occupancy-grid mapping system, produces physical intuition that simulation cannot. The failures — the drift that accumulates in dead reckoning, the feedback oscillations from gain mistuning, the sensor noise that corrupts the map — are the education.
| Resource | Platform | Type |
|---|---|---|
| ROS 2 documentation and tutorials (free) | ros.org | Practice |
| Drake (free) | drake.mit.edu | Practice |
| NVIDIA Isaac Sim (free; hardware requirements; NVIDIA ecosystem) | developer.nvidia.com/isaac-sim | Practice |
| Gazebo simulator (free) | gazebosim.org | Practice |
| Foxglove documentation (free tier / paid platform) | docs.foxglove.dev | Practice |
| Gymnasium + MuJoCo (free) | gymnasium.farama.org | Practice |
| GTSAM (free) | gtsam.org | Practice |
| Cyrill Stachniss SLAM Python notebooks (free) | GitHub | Practice |
| Physical robot build project | Local hardware | Practice |
Traps
| Trap | Why it misleads | Better response |
|---|---|---|
| Treating simulation results as ground truth | Simulation provides fast, cheap iteration but cannot capture all the behavior of physical systems — contact dynamics, sensor noise statistics, motor backlash, and cable routing effects all introduce discrepancies. Policies and systems that perform well in simulation frequently fail in real deployment; this pattern has recurred across the field’s history. | Design experiments explicitly to characterize the sim-to-real gap. Profile real robot behavior and compare it to simulation predictions. Use domain randomization to produce policies robust to variation, and evaluate with a held-out real robot test set that does not influence design decisions. |
| Skipping the mathematical foundations | Libraries handle forward kinematics, trajectory optimization, and Kalman filtering for standard use cases, allowing practitioners to deploy robotic systems without understanding the underlying mathematics. When something goes wrong — a controller diverges, an estimator drifts, a planner fails to find a path — the practitioner without the mathematical foundation cannot diagnose the cause. | Work through Lynch and Park’s mathematical development before using the ROS move_it planning library or a Kalman filter implementation from a library. Derive the kinematic equations for a simple two-joint arm by hand. Understanding what the tools are computing makes them diagnosable. |
| Demo-to-deployment overconfidence | Robotic demonstrations are carefully staged: specific objects in specific positions, lighting conditions optimized for the vision system, no people or unexpected objects in the workspace. Real deployment has none of these guarantees. The field has a long history of impressive demonstrations followed by disappointing deployment experiences. | When evaluating a robotic system’s capabilities, ask explicitly what conditions the demonstration required and what fraction of real deployment conditions those are. The DARPA challenge experience is instructive: success in the challenge conditions did not predict success in the long tail of real driving conditions. |
| Framework fluency without conceptual depth | ROS is the robotics infrastructure standard, and learners can develop practical competence in using it — configuring navigation stacks, launching manipulation pipelines, writing subscriber/publisher nodes — while remaining shallow on what the underlying algorithms are doing. When the navigation fails because the localization particle filter has collapsed, or when the manipulator fails because inverse kinematics has multiple solutions and chose the wrong one, the framework-only practitioner cannot diagnose the problem. | Learn what the major ROS packages are doing before using them in production systems. Study the adaptive Monte Carlo localization algorithm before depending on the amcl package. Understand trajectory optimization well enough to interpret the output of MoveIt before relying on it for safety-critical manipulation. |
| AI-for-everything without classical foundations | The excitement about learning-based robotics can create the impression that classical approaches — analytical kinematics, PID control, Kalman filtering, sampling-based planning — are obsolete. In practice, learned policies still fail modes that classical controllers handle reliably: they are brittle to distribution shift, require vast amounts of training data, and produce behavior that is opaque and hard to verify. Most production robotic systems combine classical and learned components. | Study classical control, estimation, and planning before studying learning-based robotics. Understanding when classical methods work well and why they fail in certain regimes (contact-rich manipulation, highly dynamic locomotion, generalization across task categories) provides the motivation for learning-based approaches and the judgment to combine them appropriately. |
| Safety as an afterthought | Robots operate in the physical world and can cause serious harm: industrial robots have caused fatalities, collaborative robots have caused injuries, autonomous vehicles have caused deaths. Unlike software bugs that can be patched, physical harm cannot be undone. Systems that treat safety as a feature to add after core functionality is implemented regularly deploy with safety gaps. | Treat safety requirements as primary constraints that shape system design from the beginning. Study the ISO 10218 and ISO/TS 15066 standards for industrial and collaborative robot safety. Apply fail-safe design principles: systems should fail into safe states, not unsafe ones. Test failure modes explicitly. |
中文
机器人是一种能够感知环境、决定做什么，并对物理世界采取行动的机器。这个三段式结构——感知、规划、行动——听起来简单，实际极其困难。传感器有噪声，也不完整；在连续空间中处理不确定性进行规划，计算上非常困难；而作用于物理系统时，会产生接触力、动态响应和仿真无法完全预测的失败模式。最重要的是，这三个组件会彼此影响：感知错误会传播到规划中，规划假设会塑造哪些感知是相关的，而行动又会改变下一步必须被感知的环境。机器人不是一个刚好会移动的计算机；它是一个反馈系统，其正确性取决于整个闭环。
它的数学基础不同于一般编程。运动学和动力学描述机器人身体如何运动——用建立在线性代数上的李群理论处理三维空间中的变换，用拉格朗日力学或 Newton-Euler 力学处理运动方程。控制理论描述如何在模型不确定性和外部扰动下，把物理系统驱动到期望状态。概率状态估计提供了一组工具——Kalman 滤波、粒子滤波、SLAM——用于从不确定传感器数据中推断机器人在哪里，以及环境中包含什么。运动规划提供算法，用于在高维构型空间中，从当前状态找到通向目标状态的可行轨迹。每一项都是实质性的数学主题，而不是可以通过库调用处理的技术细节。
它与本指南其他章节有大量连接。计算机视觉（§5.3 中包含相关深度学习内容）为现代系统提供感知骨架。强化学习（§5.4）已经成为训练复杂操作和运动策略的主要方法。计算机组成（§4.1）和操作系统（§4.2）提供机器人系统运行的计算基础。它与 AI 安全（§5.7）的连接也在增强：随着机器人获得自主性，它们到底在优化什么，以及如何指定正确目标，会变成具有现实紧迫性的实践问题。
前置知识：线性代数（§2.3）——刚体变换使用建立在线性代数上的李群。概率与统计（§2.5）——概率估计是核心。微积分（§2.4）——动力学和控制依赖微分方程。算法（§2.6）——运动规划在核心上是算法问题。
从可编程机器到学习身体
机器人学的历史，是一系列关于“机器可以被规定去做什么”与“物理世界实际要求什么”之间差距的艰难教训。
第一台工业机器人 Unimate 于 1961 年安装在 General Motors 的工厂中。它在汽车车身上执行点焊，每天重复同一条编程轨迹数千次。Unimate 并不自主——它没有传感器，也没有反馈；它只是重放一串固定动作。对于环境被严格控制、任务简单且重复的工厂自动化来说，这已经足够。在接下来的三十年中，工业机器人沿着这种受控环境轴线发展：精确工程化的工作单元，每个部件都以已知位置到达，使编程轨迹能够可靠工作。
一个不受控环境中的自主机器人会是什么样子，这个问题由 SRI 的 Shakey 回答。Shakey 于 1966 到 1972 年间开发，能够使用摄像头和类似 LIDAR 的传感器在有障碍物的房间中导航，规划通向目标的路线，并在监控意外情况的同时执行计划。构建 Shakey 的规划研究者很乐观：AI 寒冬的第一阶段还没有到来，通用智能似乎只需要几年改进就能实现。Shakey 的后继环境是 STRIPS planner，它引入了古典 AI（§5.1）在之后几十年中持续发展的逻辑规划表示。Shakey 展示的是，即使在简化环境中，感知和规划也远比看起来困难；而要可靠地结合二者，所需工程努力远远超过算法洞见本身。
机器人学中的概率转向，由 Thrun、Burgard 和 Fox 的 Probabilistic Robotics（2005）形式化。它来自这样一种认识：古典规划器假设世界已知、命令会被完美执行——这两个假设在实践中都严重错误。一个在真实走廊中导航的机器人，会遇到有时开、有时关的门，会遇到不可预测移动的人，也会遇到微小方向误差，这些误差会随着距离累积，直到机器人彻底迷失。把位置和环境状态视为概率分布，并在传感器数据到来时用贝叶斯滤波更新这些分布，会产生健壮得多的导航。同步定位与建图（Simultaneous Localization and Mapping，SLAM）——在构建环境地图的同时，也在该地图中定位自己——成为移动机器人在未知环境中运行的使能技术。SLAM 问题同时是感知问题、估计问题和计算问题，它推动了 2000 年代大量理论和算法工作。
DARPA Grand Challenge 在 2004 和 2005 年展示了经过充分工程化的系统可以做到什么。2004 年，没有车辆完成 240 公里的沙漠赛道；2005 年，有五辆车完成。Sebastian Thrun 的 Stanford 团队用 Stanley，以及 Carnegie Mellon 团队用 Sandstorm 和 H1ghlander，构建了结合 GPS、LIDAR、双目视觉和学习式地形分类的系统，在结构化越野环境中实现了功能性自动驾驶。2007 年的 Urban Challenge 则进一步进入带交通规则的城市驾驶。这些项目证明，在有边界环境中，自动驾驶在工程上是可行的，并启动了持续十年的投资，最终产生了 Waymo、Cruise 和十几个其他自动驾驶项目。
那十年的结果比创始者预期的复杂。Waymo 的技术已经证明可以在 Arizona 的 Phoenix 实现自主运行——这是一座相对平坦、地图良好、天气友好的城市——但在没有大量额外工程投入的情况下，它对更复杂环境的泛化能力有限。“长尾”边缘案例——那些在开发过程中很少出现，但在数百万英里驾驶中会以一定频率遇到的异常情况——已经证明难以被系统性解决。每解决一个边缘案例，都会揭示新的边缘案例。这种模式——在熟悉条件下表现稳固，在分布边界上脆弱——总体上刻画了机器人系统，并且是几十年来反复出现的主题。
Boston Dynamics 的 BigDog（2005）及其后继者引入了另一个问题：动态运动。轮式机器人可以用古典技术控制；腿式机器人则不同，其稳定性需要以数百赫兹频率进行连续主动控制，因此要求不同方法。Marc Raibert 在 1980 年代 MIT 的工作，用简化的弹簧—质量模型建立了动态腿式运动的理论基础；Boston Dynamics 通过液压驱动、快速计算和大量工程工作，把它转化为实际硬件。商业化后继者 Spot 可以在崎岖地形中导航、开门，并在被踢后恢复——这些能力要求控制闭环能够在完整物理交互范围内正确运行。
深度学习革命大约在 2016 年进入机器人学。Sergey Levine 及其同事证明，端到端策略——直接把图像像素映射到电机力矩的卷积网络——可以在不手工设计中间表示的情况下，通过数千次真实试验学习机器人操作任务。这一点很惊人，因为此前的共识是，机器人学中的端到端学习需要过多真实世界数据，而且得到的策略过于脆弱，难以实际使用。这些策略在特定任务和实验室条件下有效的展示，启动了一个大型研究项目。
sim-to-real 迁移问题随即成为核心。在仿真中训练神经网络策略，比在真实机器人上训练快得多、便宜得多；但仿真中训练的策略常常在真实硬件上失败，因为仿真并不完美——传感器特性、物理接触和控制动力学都只是近似。领域随机化——在物理参数随机变化的环境中训练，使真实世界看起来像训练分布中的一个样本——被证明部分有效。OpenAI 的 Dactyl 系统（2019）通过领域随机化训练，实现了机器人灵巧操作魔方，但背后需要显著的工程努力。sim-to-real 迁移到底能在多大程度上有效，仍然是关键研究问题。
当前前沿是大规模预训练与机器人学的融合。Google DeepMind 的 RT-1（2022）证明，在一个大而多样的机器人示范数据集上训练的机器人策略，能够比只在特定任务小数据集上训练的策略更广泛地泛化。RT-2（2023）进一步把视觉—语言模型与机器人行动数据联合训练，产生了能够响应自然语言指令，并对新任务类别进行推理的策略。这一路线直接借用了 §5.5 中缩放定律和基础模型的方法论；同样的涌现能力模式是否会像在语言中那样出现在机器人操作中，仍然是一个开放的经验问题。
几何、估计与控制闭环
运动学与动力学：数学化的身体
在机器人能够被控制之前，必须先用数学描述它的运动。运动学把关节角映射为空间中机器人连杆的位置和姿态，而不考虑导致运动的力。正运动学——给定关节角，求末端执行器位姿——总是可以通过组合变换矩阵求解。逆运动学——给定期望末端执行器位姿，求关节角——通常不唯一，并且经常没有闭式解，需要迭代数值方法，或针对特定机器人几何结构使用解析解。
刚体变换被表示为特殊欧几里得群 SE(3) 的元素：一个旋转矩阵 R ∈ SO(3) 和一个平移向量 t ∈ ℝ³，被组合为一个 4×4 齐次变换矩阵。SO(3) 和 SE(3) 的李群结构——以及它们对应的李代数 so(3) 和 se(3)，作为单位元处的切空间——为变换组合、速度计算与积分，以及在刚体构型空间上表述优化问题提供了数学框架。Lynch 和 Park 的 Modern Robotics 系统发展了这一李群框架，是现代运动学表述最易读的入门。
动力学把运动学扩展到力和力矩。拉格朗日表述——构造动能和势能函数，并应用 Euler-Lagrange 方程——会产生关于关节角、速度和加速度的运动方程。Newton-Euler 表述提供了等价的递归算法，适合高效计算。运动方程中出现的质量矩阵、Coriolis 项和重力向量，是控制设计必须考虑的物理量。
估计：知道自己在哪里
状态估计的问题是：从带噪声、不完整且不同步的传感器测量中，推断机器人的状态——位置、姿态、速度，以及相关环境特征的状态。
Kalman 滤波提供了最优线性估计器：给定一个带高斯噪声的线性系统，它会产生状态的最小方差估计。预测步骤会通过动力学模型传播状态估计和不确定性；更新步骤则根据相对不确定性，把先验估计和传感器测量加权结合。扩展 Kalman 滤波（EKF）通过在当前估计附近线性化来处理非线性动力学；无迹 Kalman 滤波（UKF）和粒子滤波则为高度非线性系统或非高斯分布提供替代方法。
SLAM 是对机器人轨迹和环境地图的同步估计。经典表述——EKF-SLAM、粒子滤波 SLAM——存在计算成本问题，其成本会随着地图特征数量平方增长。基于图的 SLAM 把问题重新表述为位姿图上的非线性最小二乘优化，并用高效稀疏求解器（g2o、GTSAM）利用问题的稀疏结构。视觉—惯性里程计结合相机测量和 IMU（惯性测量单元）数据，实时估计轨迹和稀疏视觉地图，使无需预建地图的定位成为可能。ORB-SLAM3 等现代系统可以在普通硬件上实现稳健的实时定位与建图。
控制：闭合回路
控制理论提供工具，用于在模型不确定性和外部扰动下，把物理系统驱动到期望状态。
比例—积分—微分控制（PID）是工业控制的主力：控制信号是跟踪误差、误差积分和误差导数的加权和。PID 简单、稳健，并且对许多单关节控制问题有效；调节三个增益以获得满意性能，是标准工业实践。对于多关节机器人，关节之间的相互作用会产生耦合项，如果没有额外前馈项，PID 无法处理这些耦合。
模型预测控制（MPC）会在每个时间步求解一个优化问题，在有限时域内优化一串控制输入，应用第一个输入，然后在下一个时间步重新求解问题。MPC 显式处理状态和输入约束，考虑耦合多体动力学，并且可以纳入滚动时域计划。计算成本随着时域长度和动力学模型复杂度增长；快速嵌入式优化求解器（ACADO、OSQP、CasADi）的发展，已经使 MPC 能够用于实时机器人控制。
用于控制的强化学习，通过与环境交互优化期望累积奖励，学习一个从状态映射到动作的策略。对于富接触操作和动态运动，RL 已经产生了超过手工控制器的策略，尤其是在那些动力学复杂、难以解析建模的任务中。挑战在于样本效率——在仿真中学习一个稳定双足行走策略通常需要数百万个仿真步——以及 sim-to-real 迁移。把 RL 与古典控制结构结合起来的混合方法（例如在基于模型的基线控制器上叠加一个学习得到的 residual），往往比单独使用任一种方法表现更好。
学习这一部分会改变什么
机器人学会改变实践者理解计算与物理世界关系的方式。
第一个变化，是本能地围绕不确定性和反馈进行推理。那些不需要物理环境反馈就能运行的软件系统，可以被设计成开环程序。机器人系统不能；它们必须闭合回路。接受过概率状态估计训练的实践者，对于任何必须作用于世界的系统，都会问：状态是什么，估计有多不确定，每个行动如何改变状态以及不确定性？这种框架不仅适用于机器人，也适用于任何反馈控制系统。
第二个变化，是对 sim-to-real 差距形成校准过的理解。仿真快速、便宜、可预测；真实世界缓慢、昂贵、充满意外。处理过 sim-to-real 迁移的实践者，会把仿真结果视为证据，而不是真相，并设计实验刻画分布偏移，而不是假装它不存在。这种纪律会迁移到任何“在一个分布中训练、在另一个分布中部署”的应用中。
第三个变化，是意识到集成才是难题。感知、估计、规划和控制，各自在设计良好的组件中可以单独工作。让它们一起工作，在一个组件中的错误会穿过整个栈传播，在时间和通信约束下运行，并面对对抗性的物理条件——这才是真实机器人系统中消耗大部分努力的工程挑战。受过机器人学训练的实践者，会把集成视为一等问题，而不是实现细节。
第四个变化，是物理直觉。理解力、接触、惯性和反馈在真实系统中如何表现——增益过高时机械臂如何过冲，腿式机器人的稳定性如何取决于质心位置，传感器噪声如何累积成位置误差——需要与物理系统的直接经验。这种直觉无法直接从阅读中迁移；真正构建和调试过物理机器人系统的实践者，会以不同方式推理物理动力学。
资源
书籍与文本
Lynch 和 Park 的 Modern Robotics: Mechanics, Planning, and Control（2017，可在 modernrobotics.org 免费获取）是多数学习者的正确起点。它对运动学和动力学的李群表述，是这个领域的现代数学语言；它的运动规划和控制章节提供了统一处理；并且免费教材与配套软件使它易于获取。由同一作者提供的 Coursera 课程可以提供结构化节奏。
Thrun、Burgard 和 Fox 的 Probabilistic Robotics（2005）尽管已有年头，仍然是状态估计和 SLAM 的基础参考。它所发展的粒子滤波、EKF-SLAM 和基于图的 SLAM 表述，是每一个移动机器人导航系统的数学底层。第 2 章（Gaussian Filters）到第 11 章（SLAM）构成了概率机器人学的必要课程。
Siciliano、Sciavicco、Villani 和 Oriolo 的 Robotics: Modelling, Planning and Control（2009）是全面的古典参考书——深刻且数学严谨，覆盖运动学、动力学、控制和运动规划。对于想要深入基础处理的实践者，这是合适的主文本；对于新手，Lynch 和 Park 更易读。
LaValle 的 Planning Algorithms（2006，可在 planning.cs.uiuc.edu 免费获取）是运动规划的基础文本，覆盖构型空间、采样式规划（RRT、PRM）、反馈运动规划和不确定性下规划。它的范围比机器人学更广——它覆盖一般规划问题——深度适合严肃学习。
Murray、Li 和 Sastry 的 A Mathematical Introduction to Robotic Manipulation（1994，可在 murray.cds.caltech.edu 免费获取）发展了机器人运动学和操作力学中的李群框架，而 Lynch 和 Park 正是在此基础上展开。尽管这本书有些年头，但其数学发展清楚而完整。
| 书籍 | 作用 | 类型 |
|---|---|---|
| Lynch & Park, Modern Robotics（免费） | 当代基础文本 | 入门 |
| Thrun, Burgard & Fox, Probabilistic Robotics | 概率估计；SLAM | 深入 |
| Siciliano et al., Robotics: Modelling, Planning and Control | 全面古典参考 | 参考 |
| LaValle, Planning Algorithms（免费） | 运动规划基础文本 | 深入 |
| Murray, Li & Sastry, Mathematical Introduction to Robotic Manipulation（免费） | 李群运动学与操作 | 深入 |
| Spong, Hutchinson & Vidyasagar, Robot Modeling and Control（第 2 版） | 机器人动力学与控制 | 深入 |
| Siegwart, Nourbakhsh & Scaramuzza, Introduction to Autonomous Mobile Robots（第 2 版） | 移动机器人基础 | 深入 |
课程与讲座
Lynch 和 Park 的 Coursera Robotics Specialization（可免费旁听）配套教材，是目前最结构化的免费现代机器人学学习路径。
MIT 6.832（Underactuated Robotics，Russ Tedrake，可在 underactuated.mit.edu 免费获取）覆盖欠驱动系统中动力学、控制和运动规划的交叉——机器人腿、受限驱动下的操作——并提供讲座视频、习题和代码 notebooks。这门课在智识上要求很高，也代表研究前沿。
Stanford CS223A（Introduction to Robotics，免费讲义和视频）以数学深度覆盖古典机器人学——运动学、动力学、工作空间分析和控制。
Berkeley CS287（Advanced Robotics，Pieter Abbeel，材料可获取）以研究生深度覆盖概率机器人学、规划和基于学习的机器人学。
Cyrill Stachniss 的 YouTube 机器人学讲座系列（免费）以极高的清晰度覆盖移动机器人、SLAM 和概率状态估计。对于概率基础来说，这是最有用的视频资源。
| 课程 | 平台 | 类型 |
|---|---|---|
| Lynch & Park Coursera Robotics Specialization（可免费旁听） | Coursera | 入门 |
| MIT 6.832 Underactuated Robotics（免费） | underactuated.mit.edu | 深入 |
| Stanford CS223A Introduction to Robotics（免费） | Stanford / YouTube | 入门 |
| Cyrill Stachniss SLAM lectures（免费） | YouTube | 实践 |
实践、工具与当前资料
ROS 2（Robot Operating System，免费，ros.org）是机器人软件的标准中间件。它的发布—订阅通信模型、硬件抽象，以及大量工具库（rviz 用于可视化，rosbag 用于数据记录，move_it 用于操作规划），使它成为多数非工业机器人开发的实践基础设施。完成官方 ROS 2 教程，是进入机器人软件开发的入口。
Gymnasium（原 OpenAI Gym，免费）为运动和操作中的 RL 研究提供标准化环境。MuJoCo 物理引擎（现在对研究免费）为富接触任务提供高质量仿真。
GTSAM（免费，Georgia Tech Smoothing and Mapping）实现了现代 SLAM 核心中的因子图优化。完成 GTSAM 的 Python API 练习，并实现一个简单位姿图 SLAM 系统，可以让优化结构变得具体。
RT-1 和 RT-2 论文（免费，robotics-transformer.github.io）是机器人操作中基础模型路线的一手来源。把它们与 §5.5 中关于基础模型的材料一起阅读，可以建立语言模型缩放与机器人学之间的桥梁。
物理动手工作不可替代。构建一个简单移动机器人——即使用 Raspberry Pi、电机控制器和廉价编码器——并实现 PID 驱动控制，然后实现基础 EKF dead-reckoning，再实现简单占据栅格地图系统，会产生仿真无法提供的物理直觉。那些失败——dead reckoning 中不断累积的漂移、增益调节错误造成的反馈振荡、破坏地图的传感器噪声——就是教育本身。
| 资源 | 平台 | 类型 |
|---|---|---|
| ROS 2 documentation and tutorials（免费） | ros.org | 实践 |
| Drake（免费） | drake.mit.edu | 实践 |
| NVIDIA Isaac Sim（免费；硬件要求；NVIDIA 生态） | developer.nvidia.com/isaac-sim | 实践 |
| Gazebo simulator（免费） | gazebosim.org | 实践 |
| Foxglove documentation（免费层 / 付费平台） | docs.foxglove.dev | 实践 |
| Gymnasium + MuJoCo（免费） | gymnasium.farama.org | 实践 |
| GTSAM（免费） | gtsam.org | 实践 |
| Cyrill Stachniss SLAM Python notebooks（免费） | GitHub | 实践 |
| Physical robot build project | 本地硬件 | 实践 |
陷阱
| 陷阱 | 为什么会误导 | 更好的回应 |
|---|---|---|
| 把仿真结果当成真实答案 | 仿真提供快速、廉价的迭代，但无法捕捉物理系统的全部行为——接触动力学、传感器噪声统计、电机背隙和线缆布置效应都会引入差异。在仿真中表现良好的策略和系统，经常在真实部署中失败；这一模式在领域历史中反复出现。 | 明确设计实验来刻画 sim-to-real 差距。分析真实机器人行为，并与仿真预测比较。使用领域随机化来产生对变化更稳健的策略，并用不影响设计决策的真实机器人保留测试集进行评估。 |
| 跳过数学基础 | 库可以处理标准用例中的正运动学、轨迹优化和 Kalman 滤波，使实践者在不了解底层数学的情况下也能部署机器人系统。当某些东西出错时——控制器发散、估计器漂移、规划器找不到路径——没有数学基础的实践者无法诊断原因。 | 在使用 ROS move_it 规划库或库中的 Kalman 滤波实现之前，先完整学习 Lynch 和 Park 的数学发展。手动推导一个简单两关节机械臂的运动学方程。理解工具正在计算什么，才使工具变得可诊断。 |
| 从演示到部署的过度自信 | 机器人演示通常被精心布置：特定物体放在特定位置，视觉系统获得优化过的光照条件，工作空间中没有人或意外物体。真实部署没有这些保证。这个领域长期存在令人印象深刻的演示，随后在部署中令人失望的历史。 | 评估一个机器人系统能力时，明确询问演示需要哪些条件，以及这些条件覆盖了真实部署条件的多少比例。DARPA Challenge 的经验很有启发性：在挑战条件下成功，并不预示能在真实驾驶长尾条件下成功。 |
| 只有框架熟练度，没有概念深度 | ROS 是机器人基础设施标准，学习者可以在使用它上形成实践能力——配置导航栈、启动操作流水线、编写 subscriber/publisher nodes——但仍然对底层算法理解很浅。当导航因为定位粒子滤波崩塌而失败，或机械臂因为逆运动学存在多个解并选择了错误解而失败时，只懂框架的实践者无法诊断问题。 | 在生产系统中使用主要 ROS 包之前，先学习它们在做什么。依赖 amcl 包之前，先学习 adaptive Monte Carlo localization 算法。依赖 MoveIt 做安全关键操作之前，先充分理解轨迹优化，能够解释其输出。 |
| 没有古典基础就 AI-for-everything | 基于学习的机器人学令人兴奋，可能给人一种印象：古典方法——解析运动学、PID 控制、Kalman 滤波、采样式规划——已经过时。实践中，学习策略仍然会在古典控制器能可靠处理的模式中失败：它们对分布偏移脆弱，需要大量训练数据，行为不透明且难以验证。多数生产级机器人系统会结合古典组件和学习组件。 | 先学习古典控制、估计和规划，再学习基于学习的机器人学。理解古典方法何时有效、为什么在某些区域失败（富接触操作、高动态运动、跨任务类别泛化），会提供学习方法的动机，也提供正确组合二者的判断力。 |
| 把安全当成事后问题 | 机器人在物理世界中运行，可能造成严重伤害：工业机器人造成过死亡事故，协作机器人造成过伤害，自动驾驶车辆造成过死亡。与可以打补丁的软件 bug 不同，物理伤害无法撤销。那些把安全当成功能实现后再添加的系统，经常带着安全缺口部署。 | 从一开始就把安全要求当成塑造系统设计的主要约束。学习工业与协作机器人安全标准 ISO 10218 和 ISO/TS 15066。应用 fail-safe 设计原则：系统应当故障到安全状态，而不是危险状态。明确测试失败模式。 |