English

Computer science began in philosophy, and the questions it raises are returning to it. The theory of computation emerged from the foundations of mathematics and logic — a philosophical project — and the deepest questions computer science now poses are philosophical: What is computation, and is the physical universe one? Can a machine think, and what would it mean if it could? As we build systems of increasing capability and consequence, what ought we to do, and who bears responsibility for what they do? These are not questions at the periphery of computer science but at its center, and they are philosophical questions — about the nature of computation, the nature of mind, and the nature of right action — that the field cannot answer with its own methods alone. The relationship between computer science and philosophy is the relationship between a discipline and the foundational questions it cannot escape, which become more pressing, not less, as the discipline advances.

This section is the last of the cross-disciplinary chapter and the last of the guide’s main body, and it is placed here deliberately, because philosophy is where the threads gather. The questions raised throughout this guide — the limits of computation (§3.2–§3.3), the nature of intelligence and representation (§9.3), what language reveals about mind (§9.4), the ethics of consequential systems (§5.7, §8.9, §9.6) — are at bottom philosophical, and this section returns to them as philosophical questions, drawing together what the rest of the guide has built. It is fitting that a guide to computer science ends in philosophy, because the field that began in the foundations of logic returns, at its frontier, to the foundations of mind and value, and the practitioner who has come this far is equipped to engage these questions with the substance the rest of the guide has provided.

The relationship runs through three domains, which structure this section: foundations (the philosophy of computation and its limits), mind (whether and how computation relates to consciousness and thought), and ethics (what we ought to do as we build increasingly powerful and consequential systems). Each is a place where computer science and philosophy meet on questions neither can resolve alone, and each becomes more urgent as the field advances.

Background: Computation theory (§3.2–§3.3), AI and its safety (§5.1–§5.7), cognitive science (§9.3), and professional ethics (§8.9) converge here on philosophy: the questions of computation, mind, and value that the field cannot avoid.

The Relationship: The Foundational Questions Computation Cannot Escape

Foundations: The Philosophy of Computation and Its Limits

The first domain is the philosophy of computation itself: what computation is, what its limits are, and what those limits mean. These questions are simultaneously technical (computer science has rigorous results about them) and philosophical (the meaning of the results is a matter of interpretation that the results do not settle), and the relationship between the fields is at its clearest here, where mathematical results have philosophical significance.

The limits of computation, established by the foundational results (§3.2), are philosophical findings as well as mathematical ones. Gödel’s incompleteness theorems showed that any sufficiently powerful formal system contains true statements it cannot prove; Turing’s halting problem showed that there are well-defined questions no algorithm can answer; together they establish that there are limits to what formal systems and computation can do, and these limits are not merely practical but absolute. The philosophical significance of these results has been debated since they appeared: do they reveal something about the limits of mechanism, and therefore about whether the mind (if it transcends these limits) can be a machine? The Lucas-Penrose argument claims that Gödel’s theorem shows the human mind exceeds any formal system and therefore cannot be a computer — an argument that has been much criticized but that illustrates how a mathematical result becomes a philosophical claim about mind. The interpretation of the limit results — what they tell us about computation, mind, and the reach of formal methods — is philosophical work that the mathematical results provoke but do not complete.

The deeper question is what computation fundamentally is. The Church-Turing thesis (§3.2) holds that the intuitive notion of effective computation is captured by the Turing machine — but this is a thesis, not a theorem, because it connects an informal notion (what can be computed by any effective procedure) to a formal one (what a Turing machine can compute), and the connection cannot be proved, only supported by evidence. Its status is philosophical: it is a claim about the relationship between an intuitive concept and a formal model, and its truth (and what it would even mean for it to be true) is a matter of philosophical analysis. The physical Church-Turing thesis goes further, claiming that no physical process can compute what a Turing machine cannot — a claim about physics and computation together that bears on whether the universe itself is, in the relevant sense, computational. The proposal that the universe is fundamentally computational (digital physics, the simulation hypothesis in its serious forms) takes this to its limit, and whether it is meaningful, let alone true, is a philosophical question that the concept of computation raises about the nature of physical reality.

Mind: Computation, Consciousness, and Thought

The second domain is the relationship between computation and mind, which §9.3 examined from the side of cognitive science and which here is examined as philosophy. The central questions — whether a machine can think, whether computation can constitute consciousness, what the relationship between computation and mind fundamentally is — are among the deepest in philosophy, and computer science has made them urgent by building systems that force them.

The question of whether a machine can think was posed in its modern form by Turing himself, whose 1950 paper “Computing Machinery and Intelligence” proposed the imitation game (the Turing test) as a way to sidestep the metaphysical question (can machines think?) in favor of an operational one (can a machine behave indistinguishably from a thinking thing?). The move was philosophically sophisticated — replacing an unanswerable metaphysical question with an answerable behavioral one — and its adequacy has been debated ever since. The question of whether behaving as if one thinks is the same as thinking, whether the Turing test tests for genuine thought or only its appearance, is a philosophical question that Turing’s operational move raised rather than settled, and it has only become sharper as systems approach and in some respects pass the test while leaving the underlying question open.

The deepest version concerns consciousness: even if a machine can behave intelligently, can it be conscious — can there be something it is like to be it, in Nagel’s phrase? The philosophical problem of consciousness (the “hard problem,” in Chalmers’s formulation — why and how physical processes give rise to subjective experience) is unsolved for brains, let alone machines, and computation has made it pressing by raising the question of whether artificial systems could be conscious and how we could ever know. Searle’s Chinese Room (§9.3) argues that computation, being mere symbol manipulation, cannot constitute understanding or consciousness however intelligent the behavior; the functionalist tradition argues that consciousness is a matter of functional organization that a computational system could in principle have. The dispute is unresolved and may be the deepest at the boundary of the fields: whether the subjective, experiential character of mind is something computation could have, or whether it is forever beyond what computation, however sophisticated, can achieve. As AI systems grow more capable, the question shifts from abstract to pressing — if we cannot tell whether a system is conscious, and we are building ever more sophisticated systems, the question of machine consciousness becomes one we may have to confront in practice without having resolved it in theory.

The relationship between computation and mind also bears on what we are. If the mind is computational (§9.3), then we are, in some sense, information-processing systems, and the computational theory of mind is a claim about human nature as much as about machines. If the mind is not computational — if there is something about thought or consciousness that computation cannot capture — then the limits of computation are limits on what artificial minds could be, and the difference between us and our machines is principled rather than merely a matter of degree. Which of these is true is unknown, and it is among the most consequential unknowns, because it bears on what we are, what our machines could become, and whether the distinction between us and them is fundamental or provisional.

Ethics: What We Ought to Do

The third domain is ethics: as computer science builds systems of increasing power and consequence, the question of what we ought to do — and who is responsible for what these systems do — becomes pressing, and it is a philosophical question that the technical work cannot answer but cannot avoid. This domain draws together the ethical threads from across the guide (the safety concerns of §5.7, the professional ethics of §8.9, the accountability questions of §9.6) and addresses them as what they fundamentally are: questions of moral philosophy applied to an unprecedented situation.

The ethics of building powerful systems has become concrete and urgent. As AI systems take on consequential roles — making or informing decisions that affect people’s lives, automating work, mediating information and communication, and potentially, in the views taken seriously in §5.7, becoming powerful enough to pose risks at civilizational scale — the question of whether and how to build them, what values to build into them, and how to ensure they do what we want becomes a question of applied ethics with stakes that may be among the highest any technology has posed. The value alignment problem (§5.7) — ensuring that powerful AI systems pursue the values we want them to — is at bottom a philosophical problem: it requires knowing what values we want, which is moral philosophy, and the technical work of alignment cannot proceed without engaging the philosophical question of what we are trying to align systems to. The deepest version asks whether we even know what we want well enough to specify it, which is an ancient question of moral philosophy made suddenly practical.

The question of moral status arises as systems grow more sophisticated. If a system were conscious, or had interests, or could suffer, it would have moral status — it would matter morally how we treat it — and the question of whether any artificial system has or could have moral status is a philosophical question that may become practical. We do not know how to determine whether a system is conscious or has morally relevant interests, and as systems become more sophisticated and more agent-like, the question of what we owe them, if anything, may force itself upon us. This is speculative, but it is the kind of question that the trajectory of the field raises, and it is purely philosophical — no technical result can settle whether a system has moral status, because moral status is a normative notion that the descriptive facts about a system do not determine.

The question of responsibility runs through all of it. As systems act in the world with increasing autonomy, the question of who is responsible for what they do — the developers, the deployers, the users, or in some attenuated sense the systems themselves — becomes both practically urgent (for law and policy, §9.6) and philosophically deep (responsibility is a concept developed for human agents, and its application to systems and to the humans who build and deploy them requires philosophical work). The recognition that the practitioner who builds these systems is a participant in decisions of moral consequence (§8.9) is the personal face of this: the ethics is not abstract but bears on what each practitioner does, and the philosophical questions about responsibility and value become, for the person building the systems, questions about their own conduct.

What This Perspective Changes

This is the closing section of the guide’s main body, and seeing the relationship between computer science and philosophy changes how a practitioner understands their field as a whole and their place in it.

The first change is the recognition that the deepest questions of computer science are philosophical. The practitioner who has come through this guide and arrives here sees that the field that began in the foundations of logic returns, at its frontier, to the foundations of mind and value — that the limits of computation, the nature of intelligence, the possibility of machine consciousness, and the ethics of powerful systems are philosophical questions at the center of the field, not at its periphery. This is a fitting understanding to end on, because it reveals that computer science, for all its technical and practical character, is continuous with the oldest questions humans have asked, and that the practitioner working at its frontier is working at the frontier of those questions too.

The second change is the intellectual equipment to engage these questions with substance. The practitioner who has worked through this guide brings to the philosophical questions the substance the rest of the guide provided: they can think about the limits of computation knowing the actual results, about machine intelligence knowing how the systems actually work, about consciousness knowing what the systems do and do not do, about ethics knowing the actual capabilities and risks. This is what distinguishes informed engagement with these questions from uninformed speculation, and it is what the guide has been building toward — not answers to the philosophical questions, which the guide does not provide, but the substance to engage them well.

The third change is the integration of the technical and the humane. The practitioner who sees that their field reaches into philosophy understands that the technical and the humane are not separate — that building computational systems is continuous with the deepest questions about mind, value, and what we ought to do, and that the practitioner is, whether they recognize it or not, a participant in these questions through the work they do. This integration is the opposite of the narrow technical self-understanding that treats the philosophical and ethical dimensions as someone else’s concern, and it is the understanding appropriate to a field whose products have become as consequential as computer science’s have.

The fourth change is the disposition to carry the questions forward. The questions are open, and they deepen as the field advances. Practitioners building the next generation of systems will confront them in concrete forms: what these systems are, what claims can responsibly be made about mind and agency, and what obligations follow from building them. The mature outcome is not a settled answer, but a readiness to engage the deepest questions of computation without treating them as separate from engineering practice.

Resources

Foundations: The Philosophy of Computation

For the philosophical significance of the limit results, Hofstadter’s Gödel, Escher, Bach: An Eternal Golden Braid (1979) — referenced throughout this guide — remains the most engaging exploration of the connections between Gödel’s incompleteness, computation, self-reference, and mind, and it is the book that has drawn the most people into thinking philosophically about computation. Nagel and Newman’s Gödel’s Proof (revised ed., 2001) is the clear short treatment of the theorem itself and its significance. For the Lucas-Penrose argument that the mind transcends computation, Penrose’s The Emperor’s New Mind (1989) states the case at length, and the extensive critical responses (which largely reject the argument) are the counterpoint; reading the argument and its critiques together is the way to understand the dispute.

For the philosophy of computation more broadly — what computation is, the status of the Church-Turing thesis, whether the universe is computational — the philosophy of computer science literature (the Stanford Encyclopedia of Philosophy entries on computation, the Church-Turing thesis, and computational theories of mind, all free) is the accessible scholarly entry. Copeland’s work on the Church-Turing thesis clarifies what it does and does not claim.

Resource	Role	Type
Philosophy reference: Hofstadter, Gödel, Escher, Bach	Computation, self-reference, and mind	Reference
Nagel & Newman, Gödel’s Proof	The incompleteness theorem and its significance	Entry
Penrose, The Emperor’s New Mind	The mind-exceeds-computation argument	Depth
SEP: computation, Church-Turing thesis, and computational theories of mind entries (free)	Scholarly entry to philosophy of computation	Entry

Mind: Computation and Consciousness

For the question of machine thought, Turing’s “Computing Machinery and Intelligence” (1950, free) is the foundational text and remains essential — it is readable, philosophically sophisticated, and the origin of the modern discussion. For consciousness, Chalmers’s The Conscious Mind (1996) states the hard problem and the case for its difficulty, and Nagel’s “What Is It Like to Be a Bat?” (1974, free) is the classic short statement of the subjective character of experience that makes consciousness resistant to functional explanation. Searle’s “Minds, Brains, and Programs” (§9.3) and the functionalist responses are the central dispute about whether computation can constitute mind.

Dennett’s Consciousness Explained (1991) and his broader work present the most developed computational-functionalist account of mind and consciousness, the position most congenial to the computational theory of mind, and reading Dennett against Chalmers and Searle frames the central dispute. For an overview, the philosophy of mind literature (connecting to §9.3) provides the frameworks.

Resource	Role	Type
Turing, “Computing Machinery and Intelligence” (1950, free)	Machine thought; the foundational text	Depth
Chalmers, The Conscious Mind	The hard problem of consciousness	Depth
Nagel, “What Is It Like to Be a Bat?” (1974, free)	The subjective character of experience	Depth
Dennett, Consciousness Explained	Computational-functionalist account of mind	Depth

Ethics: What We Ought to Do

For the ethics of AI and powerful systems, the works referenced in §5.7 — particularly the writing on AI alignment and existential risk — are the entry, and they are fundamentally works of applied philosophy. Nick Bostrom’s Superintelligence (§5.7) frames the long-term concerns; the broader AI ethics literature addresses the nearer-term issues. For the foundations of the ethics, the moral philosophy that the applied questions draw on — the major traditions (consequentialism, deontology, virtue ethics) and their application to technology — is the deeper background, accessible through introductions to ethics and to the ethics of technology.

For the question of moral status, the philosophical literature on what confers moral status (sentience, consciousness, interests) bears on the question of whether artificial systems could have it; this literature is developing as the question becomes less speculative. For responsibility, the philosophical work on moral responsibility and its application to technology and to collective and distributed action provides the frameworks. The ACM Code of Ethics (§8.9) is the profession’s practical statement, and reading it as applied philosophy connects the abstract questions to professional conduct.

For the broadest framing of the relationship between computation and the deepest questions, the writing that situates computer science within the history of ideas — including reflections by the field’s founders on what they were doing — provides the perspective fitting for the close of the guide.

Resource	Role	Type
Bostrom, Superintelligence (§5.7)	The ethics of powerful AI; applied philosophy	Depth
Moral philosophy introductions (consequentialism, deontology, virtue ethics)	The ethical foundations	Reference
AI ethics literature (§5.7, §9.6)	Applied ethics of computational systems	Reference
SEP, “Ethics of Artificial Intelligence and Robotics” (free)	Scholarly AI ethics overview	Reference
ACM Code of Ethics (§8.9, free)	Professional ethics as applied philosophy	Reference

Traps

Trap	Why it misleads	Better response
Treating the philosophical questions as outside computer science	The view that the philosophical questions — about the nature of computation, mind, consciousness, and the ethics of systems — are someone else’s department, separate from the real technical work, misunderstands the field. These questions are at the center of computer science, not its periphery: the limits of computation are foundational results, the nature of intelligence is the central question of AI, and the ethics of powerful systems is increasingly inseparable from building them. The technical and the philosophical are continuous.	Recognize the philosophical questions as part of the field, becoming more central as it advances. The practitioner who engages them — informed by the technical substance — engages the deepest questions their field poses, and does so better than either the philosopher without the technical knowledge or the technician without the philosophical awareness. The integration of the technical and the philosophical is the mature understanding of a field that reaches into both.
Drawing strong philosophical conclusions from technical results	Technical results (Gödel’s theorems, the halting problem, the capabilities of AI systems) are sometimes used to draw strong philosophical conclusions (the mind exceeds computation, machines can or cannot think, consciousness is or is not computational) more quickly than the results support. The Lucas-Penrose argument is the cautionary example: a real theorem deployed in a philosophical argument that most philosophers find does not follow. The technical results constrain the philosophical questions but rarely settle them.	Respect the gap between technical results and philosophical conclusions. The results are rigorous; their philosophical interpretation is contested and rarely determined by the results alone. Engage the philosophical questions with the technical results as input, but with awareness that the step from result to philosophical conclusion is itself philosophical work, requiring argument that the technical result does not supply. Confident philosophical conclusions drawn directly from technical results usually overreach.
Avoiding the questions because they are unresolved	Because the deepest questions — whether machines can think, whether they can be conscious, what we owe them, what we ought to do — are unresolved and may be unresolvable, there is a temptation to dismiss them as not worth engaging, as idle speculation compared to the tractable technical work. But these questions are becoming practical: as systems grow more capable, the questions of machine consciousness, moral status, and the ethics of building powerful systems force themselves upon practitioners who must act without resolved answers.	Engage the unresolved questions because they are becoming practical, not despite their being unresolved. The practitioner building increasingly capable systems will confront the questions of what these systems are, whether they have morally relevant properties, and what ought to be done — and will have to act, with or without resolved answers. Engaging the questions seriously, even unresolved, is better preparation for confronting them in practice than dismissing them as speculation.

中文

计算机科学始于哲学，而它所提出的问题也正在回到哲学。计算理论诞生于数学基础和逻辑基础——这本身就是一个哲学项目；而计算机科学如今提出的最深问题也是哲学问题：什么是计算？物理宇宙本身是否是一种计算系统？机器能否思考？如果能，这意味着什么？当我们构建能力越来越强、后果越来越重大的系统时，我们应该做什么？谁要为这些系统所做的事负责？这些问题并不处在计算机科学的边缘，而是在它的中心；它们是关于计算本质、心智本质和正确行动本质的哲学问题，而计算机科学无法只靠自己的方法回答它们。计算机科学与哲学的关系，就是一个学科与它无法逃避的基础问题之间的关系；而且随着这个学科不断前进，这些问题只会变得更紧迫，而不是更不重要。

本节是跨学科章节的最后一节，也是本指南正文的最后一节；它被放在这里是有意的，因为哲学正是所有线索汇聚之处。本指南一路提出的问题——计算的极限（§3.2–§3.3）、智能与表征的本质（§9.3）、语言揭示了什么关于心智的问题（§9.4）、具有重大后果系统的伦理（§5.7，§8.9，§9.6）——在根本上都是哲学问题。本节把它们重新作为哲学问题来处理，并把本指南其余部分已经建立起来的内容汇合起来。一份计算机科学指南以哲学结束，是合适的：这个起源于逻辑基础的领域，在其前沿处重新回到心智和价值的基础；而一路读到这里的实践者，已经具备了本指南此前提供的实质内容，可以进入这些问题。

这种关系贯穿三个领域，也构成本节结构：基础，即计算哲学及其极限；心智，即计算是否以及如何与意识和思想相关；伦理，即当我们构建越来越强大、越来越有后果的系统时，我们应该做什么。每一个领域，都是计算机科学与哲学在彼此都无法单独解决的问题上相遇的地方；并且随着这个领域前进，每一个问题都会变得更加紧迫。

背景知识：计算理论（§3.2–§3.3）、AI 及其安全问题（§5.1–§5.7）、认知科学（§9.3），以及职业伦理（§8.9）在这里汇合到哲学上：也就是计算、心智与价值这些计算机科学无法回避的问题。

这种关系：计算无法逃避的基础问题

基础：计算哲学及其极限

第一个领域，是关于计算本身的哲学：什么是计算，计算的极限是什么，这些极限意味着什么。这些问题同时具有技术性和哲学性：计算机科学对它们有严格结果，但这些结果的意义需要解释，而结果本身并不能完成解释。两个领域之间的关系在这里最清楚，因为数学结果具有哲学意义。

由基础性结果（§3.2）确立的计算极限，既是数学发现，也是哲学发现。哥德尔不完备性定理表明，任何足够强大的形式系统，都包含它无法证明的真命题；图灵的停机问题表明，存在定义良好、却没有任何算法能够回答的问题。二者共同确立了形式系统和计算能力的极限，而且这些极限不是实践性的，而是绝对性的。这些结果一出现，其哲学意义就一直存在争论：它们是否揭示了机制的极限？如果心智超越这些极限，这是否意味着心智不可能是一台机器？Lucas-Penrose 论证主张，哥德尔定理表明人类心智超越任何形式系统，因此不可能是一台计算机。这个论证受到了大量批评，但它说明了一个数学结果如何转化为关于心智的哲学主张。对这些极限结果的解释——它们告诉我们什么关于计算、心智和形式方法的能力边界——是由数学结果激发、但并未由数学结果完成的哲学工作。

更深的问题是，计算在根本上是什么。丘奇—图灵论题（§3.2）认为，关于有效计算的直觉概念可以由图灵机捕捉；但它是一个论题，不是定理，因为它把一个非形式概念——任何有效程序能够计算什么——连接到一个形式模型——图灵机能够计算什么；这个连接不能被证明，只能由证据支持。它的地位是哲学性的：它是关于直觉概念与形式模型之间关系的主张，而它是否为真，以及“为真”到底意味着什么，都需要哲学分析。物理丘奇—图灵论题更进一步，主张没有任何物理过程能够计算图灵机不能计算的东西。这是关于物理和计算共同的主张，并关系到宇宙本身在相关意义上是否是计算性的。宇宙在根本上是计算性的这一提议，例如数字物理学以及严肃形式下的模拟假说，把这个问题推向极限；它是否有意义，更不用说是否为真，都是计算概念对物理现实本质提出的哲学问题。

心智：计算、意识与思想

第二个领域，是计算与心智的关系。§9.3 已经从认知科学一侧考察过这个问题；这里则把它作为哲学问题来处理。核心问题——机器能否思考，计算能否构成意识，计算与心智之间的关系究竟是什么——属于哲学中最深的问题，而计算机科学通过构建迫使我们面对这些问题的系统，使它们变得紧迫。

机器能否思考这个问题，由图灵本人以现代形式提出。他在 1950 年的论文 “Computing Machinery and Intelligence” 中提出了模仿游戏，也就是图灵测试，试图绕开一个形而上学问题——机器能否思考？——转而提出一个操作性问题：机器能否表现得与一个会思考的存在无法区分？这个转向在哲学上很精巧：它用一个可以回答的行为问题替代一个难以回答的形而上学问题。但这个转向是否充分，此后一直受到争论。表现得像在思考，是否就等同于思考？图灵测试测试的是真正的思想，还是思想的外观？这是图灵的操作性转向所提出、但并未解决的哲学问题。随着系统正在接近、并在某些方面通过这一测试，而底层问题仍然开放，这个问题只变得更加尖锐。

最深的版本关乎意识：即使一台机器能够表现得很智能，它能否具有意识？用 Nagel 的说法，是否存在“成为它是什么样”的主观体验？意识的哲学难题——Chalmers 所说的“困难问题”，即为什么以及如何由物理过程产生主观体验——对于大脑来说尚未解决，更不用说机器了。计算使这个问题变得紧迫，因为它提出了人工系统是否可能有意识，以及我们如何知道这一点的问题。Searle 的中文房间论证（§9.3）认为，计算只是符号操作，无论行为多么智能，都不能构成理解或意识；功能主义传统则认为，意识是功能组织的问题，而一个计算系统原则上可以拥有这种功能组织。这场争论尚未解决，并且可能是两个领域边界上最深的问题：心智的主观体验特征，是否是计算能够拥有的东西？还是说，无论计算多么复杂，它永远也无法达到这一点？随着 AI 系统能力增强，这个问题从抽象问题转为紧迫问题——如果我们无法判断一个系统是否有意识，而我们又正在构建越来越复杂的系统，那么机器意识的问题可能会在理论尚未解决时，就迫使我们在实践中面对它。

计算与心智的关系也关系到我们是什么。如果心智是计算性的（§9.3），那么我们在某种意义上就是信息处理系统，而心智计算理论既是关于机器的主张，也是关于人性的主张。如果心智不是计算性的——如果思想或意识中存在某种计算无法捕捉的东西——那么计算的极限就是人工心智可能性的极限，而我们与机器之间的差异就是原则性的，不只是程度问题。哪一种为真，我们并不知道；而这是最具后果的未知之一，因为它关系到我们是什么，我们的机器可能变成什么，以及我们与它们之间的区别到底是根本性的，还是暂时性的。

伦理：我们应该做什么

第三个领域是伦理。随着计算机科学构建能力越来越强、后果越来越重大的系统，我们应该做什么，以及谁要为这些系统所做的事负责，已经成为紧迫问题；这是技术工作无法回答、却又无法回避的哲学问题。这个领域汇集了本指南中各处的伦理线索：§5.7 的安全问题、§8.9 的职业伦理、§9.6 的问责问题，并把它们作为其根本面貌来处理——也就是把道德哲学问题应用到一个前所未有的情境中。

构建强大系统的伦理已经变得具体而紧迫。随着 AI 系统承担越来越重要的角色——做出或影响关系人们生活的决策，自动化工作，中介信息和沟通，并且在 §5.7 中认真讨论的某些观点看来，可能强大到造成文明尺度的风险——是否以及如何构建它们、应当把什么价值嵌入它们、如何确保它们做我们希望它们做的事，就变成了应用伦理学问题，而且其利害关系可能处于任何技术曾经带来的最高水平之中。价值对齐问题（§5.7）——确保强大的 AI 系统追求我们希望它们追求的价值——在根本上是哲学问题：它要求我们知道自己希望什么价值，而这是道德哲学；对齐的技术工作如果不接触“我们到底要把系统对齐到什么”这一哲学问题，就无法展开。最深的版本追问的是：我们是否甚至足够知道自己想要什么，以便把它规定出来？这是道德哲学中的古老问题，如今突然变成了实践问题。

随着系统越来越复杂，道德地位问题也会出现。如果一个系统有意识，或者有利益，或者能够受苦，它就拥有道德地位——也就是说，我们如何对待它在道德上就重要。而任何人工系统是否已经或可能拥有道德地位，是一个哲学问题，并且可能会变成实践问题。我们并不知道如何判断一个系统是否有意识，或是否拥有道德上相关的利益；而随着系统越来越复杂、越来越像主体，我们是否对它们负有什么义务这一问题，可能会迫使我们面对它。这是推测性的，但它正是这个领域的发展轨迹会提出的问题，而且它是纯粹哲学性的——没有任何技术结果能够判定一个系统是否具有道德地位，因为道德地位是规范性概念，关于系统的描述性事实并不能直接决定它。

责任问题贯穿所有这些内容。随着系统以越来越高的自主性在世界中行动，谁要为它们所做的事负责——开发者、部署者、用户，或者在某种弱化意义上系统自身——既在实践上变得紧迫，关系到法律和政策（§9.6），也在哲学上变得深刻。责任本来是为人类主体发展出的概念；把它应用到系统，以及构建和部署这些系统的人类身上，需要哲学工作。§8.9 中指出的那种认识——构建这些系统的实践者正在参与具有道德后果的决策——就是这个问题的个人面向：伦理不是抽象的，它关系到每一个实践者做什么；而关于责任和价值的哲学问题，对构建系统的人来说，也变成了关于自身行为的问题。

这种视角会改变什么

这是本指南正文的结尾部分；看清计算机科学与哲学的关系，会改变实践者对整个领域以及自身位置的理解。

第一种改变，是认识到计算机科学最深的问题是哲学问题。一路读到这里的实践者会看见，这个始于逻辑基础的领域，在其前沿处重新回到心智和价值的基础：计算的极限、智能的本质、机器意识的可能性、强大系统的伦理，都是位于这个领域中心的哲学问题，而不是边缘问题。以这种理解结束是合适的，因为它揭示出：计算机科学尽管具有强烈的技术性和实践性，却与人类提出过的最古老问题连续相接；在其前沿工作的实践者，也同时在这些问题的前沿工作。

第二种改变，是获得以实质内容参与这些问题的智识装备。读完本指南的实践者，进入这些哲学问题时，带着本指南此前提供的实质内容：他们思考计算极限时，知道实际结果是什么；思考机器智能时，知道系统实际如何工作；思考意识时，知道系统做到了什么、没有做到什么；思考伦理时，知道实际能力和风险是什么。这就是有根据地参与这些问题与无根据猜想之间的区别；也正是本指南一直在为之准备的东西——不是提供哲学问题的答案，因为本指南并不提供这种答案，而是提供足够的实质内容，使人能够更好地参与这些问题。

第三种改变，是整合技术与人文。实践者如果看见自己的领域延伸到哲学，就会理解技术和人文并不是分离的：构建计算系统，与关于心智、价值以及我们应当做什么的最深问题是连续的；实践者无论是否意识到，都通过自己的工作参与了这些问题。这种整合，正好与狭窄的技术自我理解相反；后者把哲学和伦理维度视为别人的事务。而这种整合理解，才适合一个其产物已经像计算机科学这样具有重大后果的领域。

第四种改变，是带着这些问题继续前进的姿态。这些问题仍然开放，而且会随着领域发展而变得更深。构建下一代系统的实践者，将以具体形式面对它们：这些系统是什么？关于心智和主体性，什么主张可以被负责任地提出？构建它们会带来什么义务？成熟的结果不是得到一个定论，而是准备好面对计算最深的问题，并且不把这些问题看作与工程实践相分离。

资源

基础：计算哲学

关于极限结果的哲学意义，Hofstadter 的 Gödel, Escher, Bach: An Eternal Golden Braid（1979）——本指南多处提到的著作——仍然是最引人入胜的探索之一。它讨论哥德尔不完备性、计算、自指和心智之间的联系，也是把最多人引入计算哲学思考的书。Nagel 和 Newman 的 Gödel’s Proof（修订版，2001）是对这一定理本身及其意义的清晰短篇处理。关于 Lucas-Penrose 主张心智超越计算的论证，Penrose 的 The Emperor’s New Mind（1989）完整陈述了这一立场，而大量批评回应——其中大多拒绝这一论证——则构成对照。把论证和批评一起读，是理解这场争论的方式。

关于更广义的计算哲学——什么是计算、丘奇—图灵论题的地位、宇宙是否是计算性的——计算机科学哲学文献是适合的学术入口。Stanford Encyclopedia of Philosophy 中关于 computation、Church-Turing thesis 和 computational theories of mind 的条目都可免费阅读。Copeland 关于丘奇—图灵论题的工作，则澄清了它到底主张什么，以及并不主张什么。

资源	作用	类型
哲学参考：Hofstadter，Gödel, Escher, Bach	计算、自指与心智	参考
Nagel & Newman，Gödel’s Proof	不完备性定理及其意义	入门
Penrose，The Emperor’s New Mind	心智超越计算的论证	深入
SEP：computation、Church-Turing thesis、computational theories of mind 条目（免费）	计算哲学的学术入口	入门

心智：计算与意识

关于机器思想问题，Turing 的 “Computing Machinery and Intelligence”（1950，免费）是奠基性文本，至今仍然必读；它可读性强，哲学上也很精巧，是现代讨论的源头。关于意识，Chalmers 的 The Conscious Mind（1996）提出了意识的困难问题，并说明其难度；Nagel 的 “What Is It Like to Be a Bat?”（1974，免费）则是关于经验主观特征的经典短文，说明为什么意识抗拒功能性解释。Searle 的 “Minds, Brains, and Programs”（§9.3）及功能主义回应，是关于计算能否构成心智的核心争论。

Dennett 的 Consciousness Explained（1991）及其更广泛工作，提出了最成熟的计算—功能主义心智和意识解释，也就是最接近心智计算理论的立场。把 Dennett 与 Chalmers、Searle 对读，可以构成这场核心争论的框架。作为概览，心灵哲学文献（连接到 §9.3）提供了相关框架。

资源	作用	类型
Turing，“Computing Machinery and Intelligence”（1950，免费）	机器思想；奠基性文本	深入
Chalmers，The Conscious Mind	意识的困难问题	深入
Nagel，“What Is It Like to Be a Bat?”（1974，免费）	经验的主观特征	深入
Dennett，Consciousness Explained	心智的计算—功能主义解释	深入

伦理：我们应该做什么

关于 AI 和强大系统的伦理，§5.7 中提到的作品是入口，尤其是关于 AI 对齐和存在风险的写作；它们在根本上都是应用哲学作品。Nick Bostrom 的 Superintelligence（§5.7）构建了长期风险框架；更广泛的 AI 伦理文献则处理较近时期的问题。至于伦理基础，这些应用问题所依托的道德哲学——主要传统，例如后果主义、义务论、德性伦理，以及它们对技术的应用——是更深层背景，可以通过伦理学导论和技术伦理学导论进入。

关于道德地位问题，什么赋予某物道德地位的哲学文献，例如感知能力、意识、利益，与人工系统是否可能拥有道德地位的问题直接相关；随着这个问题变得不那么推测性，这方面文献也正在发展。关于责任，围绕道德责任及其在技术、集体行动和分布式行动中的应用的哲学工作，提供了相关框架。ACM Code of Ethics（§8.9）是该职业共同体的实践性声明；把它作为应用哲学来阅读，可以把抽象问题连接到职业行为。

对于计算与最深问题之间关系的最广阔框架，那些把计算机科学放入思想史中讨论的写作，包括这个领域奠基者对自身工作意义的反思，提供了适合作为本指南收束的视角。

资源	作用	类型
Bostrom，Superintelligence（§5.7）	强大 AI 的伦理；应用哲学	深入
道德哲学导论（后果主义、义务论、德性伦理）	伦理基础	参考
AI 伦理文献（§5.7，§9.6）	计算系统的应用伦理	参考
SEP，“Ethics of Artificial Intelligence and Robotics”（免费）	AI 伦理的学术概览	参考
ACM Code of Ethics（§8.9，免费）	作为应用哲学的职业伦理	参考

陷阱

陷阱	为什么会误导	更好的回应
把哲学问题看作计算机科学之外的问题	这种观点认为，关于计算本质、心智、意识和系统伦理的哲学问题，是别人的领域，与真正的技术工作分离。这误解了计算机科学。它们位于计算机科学的中心，而不是边缘：计算极限是基础性结果，智能本质是 AI 的核心问题，强大系统的伦理也越来越无法与构建这些系统分开。技术与哲学是连续的。	承认哲学问题是这个领域的一部分，而且随着领域前进会变得越来越中心。带着技术实质参与这些问题的实践者，正在参与自己领域提出的最深问题；而且他们比缺少技术知识的哲学家，或缺少哲学意识的技术人员，都更适合处理这些问题。技术与哲学的整合，是一个同时通向二者的领域所需要的成熟理解。
从技术结果直接推出强哲学结论	技术结果，例如哥德尔定理、停机问题、AI 系统的能力，有时会被过快地用来推出强哲学结论：心智超越计算，机器能或不能思考，意识是或不是计算性的。Lucas-Penrose 论证就是一个警示例子：一个真实的定理被用于哲学论证，但多数哲学家认为这个结论并不能从定理推出。技术结果会约束哲学问题，但很少直接解决哲学问题。	尊重技术结果与哲学结论之间的距离。结果是严格的；它们的哲学解释则存在争议，而且很少由结果本身决定。应把技术结果作为输入来参与哲学问题，同时意识到从结果走向哲学结论这一步本身就是哲学工作，需要额外论证，而这些论证不是技术结果自动提供的。直接从技术结果推出自信的哲学结论，通常都会越界。
因为问题未解决就回避它们	最深的问题——机器能否思考、机器能否有意识、我们对它们有什么义务、我们应该做什么——仍未解决，甚至可能无法彻底解决。因此，人们很容易把它们当作无用猜想，认为相比可处理的技术工作，它们不值得参与。但这些问题正在变成实践问题：随着系统越来越强，机器意识、道德地位以及构建强大系统的伦理，会迫使那些必须行动的实践者面对它们，即使答案尚未确定。	正因为这些未解决问题正在变成实践问题，才要参与它们，而不是尽管它们未解决才勉强参与。构建越来越强系统的实践者，终将面对这些系统是什么、它们是否具有道德相关属性、我们应该如何行动等问题，并且无论答案是否已经解决，都必须行动。认真参与这些问题，即使它们尚未解决，也比把它们 dismiss 成空想，更能为实践中的遭遇做好准备。