English

Web and application development is the discipline of building software that runs over networks and is used by people — websites, web services, mobile applications, APIs, and cloud-deployed systems. This description encompasses the majority of contemporary software engineering work, which means the discipline’s breadth is not accidental: it is what the job of software engineer actually requires. The challenge is that breadth in isolation is not mastery. A practitioner who knows HTML, a JavaScript framework, a backend framework, a database driver, and a deployment tool has assembled a toolkit without understanding what the tools are doing, why they interact the way they do, or how to reason about novel situations.

What converts toolkit knowledge into mastery is understanding at the level of mechanisms and principles. HTTP is not just the protocol that browsers use; it is the contract that defines what a web request is, what headers can contain, how caching works, how authentication headers flow, how status codes communicate semantics. A relational database is not just something that stores data and responds to SQL; it is a system with specific transaction semantics, index structures that determine query performance, and isolation levels that determine what concurrent transactions can see. A frontend framework is not magic that makes the browser update the screen; it manages a component tree, reconciles a virtual DOM or uses fine-grained reactivity to determine what changed, and issues the minimum DOM operations to bring the browser in sync with application state. Understanding what tools are doing makes failures diagnosable, makes architectural decisions motivated, and makes switching between tools possible because the underlying concepts transfer.

The rest of this guide is prerequisite to this chapter in a direct sense. Databases (§4.4) covers what databases are actually doing. Computer networks (§4.3) covers the protocols web applications are built on. Operating systems (§4.2) covers the environment in which servers run. Security (§4.5) covers the attack surface that web applications present. HCI (§6.2) covers how to design the interfaces that web applications expose. Distributed systems (§4.6) covers what cloud-deployed, multi-service applications are, at scale. Web development as a discipline integrates all of these; its specific contribution is the integration itself and the application-level patterns that have emerged from decades of practice.

Prerequisites: Programming (§2.1) — fluency in at least one language. Networks (§4.3) — HTTP, DNS, TLS. Databases (§4.4) — SQL, transactions. Security (§4.5) — application security concerns overlap directly.

From Static Pages to the AI-Integrated Application

The World Wide Web was invented by Tim Berners-Lee at CERN in 1989-1991 as an information sharing system for physicists. The original design was simple: HTML documents, HTTP as the transfer protocol, URLs as addresses. The first web browser, WorldWideWeb (later renamed Nexus), was also Berners-Lee’s work. The original web was entirely static: servers served pre-written HTML files, browsers displayed them, and there was no mechanism for interactivity or dynamic content. This simplicity enabled explosive adoption. By 1993, CERN had made the web technology free for anyone to use; within two years, there were tens of thousands of websites.

The first step toward dynamic content was the Common Gateway Interface (CGI), standardized in 1993. CGI allowed web servers to execute arbitrary programs and return their output as web content — the program could query a database, compute a response, and return HTML. This was slow (a new process per request) and primitive (string manipulation of HTML) but demonstrated the concept. Perl became the dominant CGI language because its text manipulation capabilities were well-suited to HTML generation. The mid-1990s explosion of CGI scripts, often poorly written and insecure, gave rise to some of the earliest systematic thinking about web application security.

The server-side scripting languages of the late 1990s — PHP (1994), ASP (1996), ColdFusion, and their contemporaries — made dynamic web development more accessible. PHP especially: it embedded code directly in HTML files, with a simple deployment model (drop a .php file on the server, it runs). This accessibility drove adoption; PHP powered the early WordPress, Wikipedia, and Facebook. The architecture of a PHP application in 2000 was monolithic: all code on one server, typically one file per page, database queries inline with HTML generation, authentication and authorization as ad hoc logic. It worked at small scale and fell apart at large scale.

The Ruby on Rails framework (2004) was the intervention that introduced systematic architectural thinking to web development for a generation of practitioners. David Heinemeier Hansson extracted Rails from Basecamp and released it as an opinionated framework embodying Model-View-Controller architecture, convention over configuration, and DRY (Don’t Repeat Yourself) principles. You did not choose where to put files or how to name database tables — Rails had conventions, and following them made a significant class of web application fast to build. ActiveRecord, Rails’ ORM, abstracted SQL into Ruby objects. Scaffolding generated boilerplate. The Rails demo showing a blog in fifteen minutes was the most effective framework marketing in the history of web development and drove enormous adoption.

Ajax — Asynchronous JavaScript and XML, named by Jesse James Garrett in 2005 but using techniques that had been available since Internet Explorer 4 in 1997 — changed the interaction model. Instead of every action requiring a full page reload, JavaScript could make HTTP requests in the background and update parts of the page. Google Maps (2005) and Gmail (2004) demonstrated what this enabled: applications that felt responsive and native rather than page-by-page. The web stopped being a document medium and became an application platform. The frameworks that emerged — jQuery (2006) for DOM manipulation, then Backbone.js (2010) for client-side structure — were the first generation of frontend JavaScript architecture.

Node.js (2009), created by Ryan Dahl, put JavaScript on the server. The motivation was I/O performance: instead of blocking threads on I/O operations (database queries, network requests), Node.js used an event loop where a single thread processed many concurrent I/O operations asynchronously. This made Node.js unusually efficient for high-concurrency I/O-bound workloads and had a secondary benefit: developers could use the same language on both client and server. The Node.js ecosystem (npm) grew explosively; by the mid-2010s, npm was the largest package repository in existence.

React (2013), released by Facebook as an open-source library, introduced the component model and virtual DOM diffing that structured the next decade of frontend development. The key idea: UI is a function of state. Given the current application state, the UI can be rendered deterministically; when state changes, the library computes the minimum set of DOM updates to bring the display in sync. This model — declarative UI as a function of state, unidirectional data flow — made large applications more predictable and easier to reason about than jQuery-era imperative DOM manipulation. React’s adoption was rapid: by 2016, it was the dominant frontend library, and by 2020, React’s ecosystem (Next.js for server rendering, React Native for mobile, a rich component library ecosystem) had become the default stack for much of the industry.

TypeScript (Microsoft, 2012, widely adopted from 2017 onward) added static types to JavaScript. The adoption pattern was unusual: JavaScript is dynamically typed and had been for its entire history, and adding types retroactively to an ecosystem of millions of packages and billions of lines of code required a gradual approach. TypeScript’s opt-in, gradually-typed approach succeeded. By 2023, most major JavaScript projects had migrated to TypeScript, type-checking had become a standard part of the development workflow, and type definitions for thousands of JavaScript packages were maintained by the community. TypeScript did not change what JavaScript could do; it changed what developers could know statically about their code before running it.

The serverless and edge computing era, beginning around 2014 with AWS Lambda and accelerating through the late 2010s and 2020s, changed the deployment model. Instead of provisioning and managing servers, developers deployed functions that ran on managed infrastructure. The platform handled scaling (zero to thousands of instances automatically), operational concerns (no server management), and billing (pay per invocation rather than per server). Edge deployment — running code at content delivery network nodes geographically close to users — extended this to compute that runs closer to users, reducing latency for globally distributed applications. Cloudflare Workers, Vercel Edge Functions, and similar platforms made edge deployment accessible.

The AI integration era began in earnest with the release of the OpenAI API in 2020 and accelerated dramatically with ChatGPT and the subsequent proliferation of capable language model APIs in 2022-2023. For web application development, this introduced a new pattern: application code that calls AI APIs for natural language processing, content generation, code assistance, and reasoning tasks. The RAG (Retrieval-Augmented Generation) pattern — retrieving relevant context and passing it to a language model — became standard for building AI-powered search and question-answering features. The agent pattern — AI systems that can call tools (APIs, databases, code execution) and take sequences of actions — emerged as the next frontier. Framework tooling (LangChain, LlamaIndex, Vercel AI SDK) matured rapidly, and the skill set of a full-stack developer now includes the ability to integrate language model capabilities into production applications.

Architecture, Performance, and the Full Stack

The HTTP Layer: Understanding the Protocol

HTTP is the substrate of web application development. Every web request is an HTTP transaction: a method (GET, POST, PUT, DELETE, PATCH), a URL, headers, and optionally a body. Every response has a status code, headers, and a body. Every caching decision, every authentication scheme, every CORS policy operates through HTTP headers. A web developer who does not understand HTTP is working with a tool whose mechanics are opaque.

HTTP/1.1 (1997) introduced persistent connections (a single TCP connection serves multiple requests) and chunked transfer encoding. HTTP/2 (2015) introduced multiplexing (multiple concurrent requests over a single TCP connection) and header compression, significantly reducing latency for pages that load many resources. HTTP/3 (2022) replaced TCP with QUIC, eliminating head-of-line blocking and improving performance on lossy networks.

REST (Representational State Transfer) is the architectural style that characterizes the majority of web APIs. Its defining constraints: statelessness (each request contains all information needed to process it, no session state on the server), uniform interface (consistent resource-based URLs with standard HTTP verbs), and layered system (intermediaries like load balancers and caches are transparent). REST’s practical benefits: predictable URL structure, exploitability of HTTP caching, and interoperability through standard methods and status codes. REST’s practical weaknesses: over-fetching (the endpoint returns all fields, the client needs only a subset) and under-fetching (multiple requests required to assemble related data). GraphQL (2015, Facebook) addresses these by allowing clients to request exactly the fields they need in a single query.

Authentication and authorization are orthogonal HTTP concerns. Authentication establishes identity: who is making this request? Authorization establishes permission: is this identity allowed to do this thing? Cookie-based sessions (the traditional approach) store session state server-side; the browser presents a session cookie and the server looks up the associated user. Token-based authentication — JWT (JSON Web Tokens) being the most common form — stores session state client-side in a signed token; the server validates the signature and reads the claims. OAuth 2.0 provides the protocol for delegated authorization: allowing a user to grant a third-party application access to their account on another service without sharing credentials.

State Management: The Hardest Problem in Application Development

Application state is the data that determines what the application displays and how it responds to user actions. Managing state correctly — maintaining consistency between displayed data and actual data, handling concurrent updates, recovering from failures, updating efficiently — is where most application bugs live.

On the frontend, state management was initially ad hoc (jQuery era) and became systematized as applications grew more complex. Flux (Facebook, 2014) and Redux (2015) introduced unidirectional data flow: actions describe what happened, reducers derive new state from old state and the action, the store holds the current state, the view is derived from the store. This made state mutations explicit and traceable. The cost was verbosity. React’s Context API and hooks (useState, useReducer, useContext) made local and semi-global state management more natural without a separate library. The landscape fragmented — Zustand, Jotai, Recoil, Valtio, and others — reflecting genuine disagreements about the right model.

Server state and client state are distinct concerns. Client state — UI state like “is this modal open?” — belongs in the component. Server state — data fetched from an API, cached locally, synchronized with the server — has different characteristics: it must be invalidated when the server changes, it may be stale, it may be shared across components. Libraries like React Query and SWR specialize in server state management, providing caching, background refetching, and optimistic updates out of the box.

Real-time state — state that must reflect server-side changes as they occur — requires persistent connection mechanisms. WebSocket provides full-duplex communication over a persistent TCP connection; server events push data when it changes, without polling. Server-Sent Events (SSE) provides one-way server-to-client streaming over HTTP, simpler to implement than WebSocket when bidirectional communication is not needed. Long polling (the client makes a request, the server holds it open until something changes) is the fallback for environments where WebSocket is unavailable.

Performance: Where Measurement Replaces Intuition

Web performance is measured in terms that correlate with user experience: Time to First Byte (TTFB), First Contentful Paint (FCP), Largest Contentful Paint (LCP), Cumulative Layout Shift (CLS), and Interaction to Next Paint (INP). Google’s Core Web Vitals, which became a search ranking factor in 2021, made these metrics economically significant: slow pages lose search visibility. The connection between measurable performance and business outcome is unusually clear in web development, making performance optimization tractable.

Frontend performance improvement follows a hierarchy. Network optimization reduces transfer size (minification, compression, image optimization, HTTP/2 multiplexing, CDN caching) and reduces round trips (resource hints, preloading, service workers for offline and prefetch). JavaScript bundle size directly affects parse and execution time; tree shaking (removing unused code), code splitting (loading only the code needed for the current route), and lazy loading (loading components on demand) are the standard techniques. Critical rendering path optimization ensures that the browser can render above-the-fold content as quickly as possible by minimizing render-blocking resources.

Backend performance bottlenecks are diagnosed with profiling. Most performance problems in web applications are database-related: missing indexes causing full table scans, N+1 query patterns (one query per result in a list), excessive query count from ORM abstraction layers that hide query costs. Caching at multiple levels — database query cache, application-level cache (Redis, Memcached), CDN edge cache — reduces database load and improves response times. Connection pooling ensures that establishing new database connections (expensive) does not dominate request latency.

The database connection pool is a good example of a mechanism that is invisible in tutorial-level development and critical in production. Each web server instance maintains a pool of persistent database connections; incoming requests acquire a connection from the pool, use it, and return it. Without pooling, every request would open and close a TCP connection and perform a handshake — hundreds of milliseconds of overhead for a query that takes milliseconds. The pool size is a critical tuning parameter: too small and requests queue waiting for connections; too large and the database server’s connection limit is exceeded.

What Studying This Changes

Web and application development changes how practitioners build software that is actually used.

The first change is protocol literacy: the ability to read and reason about what is happening at the HTTP layer. A practitioner who can read HTTP headers in browser DevTools, who understands what a 401 vs. 403 response means and why, who can trace a CORS error to its source, who understands the Cache-Control header semantics — that practitioner can diagnose a wide range of production problems that are invisible to practitioners who only work at the framework API level.

The second change is architectural judgment. The choice between a monolith and microservices, between server-side rendering and client-side rendering, between a relational database and a document store, between REST and GraphQL — these are engineering decisions with consequences that play out over years of product development. Practitioners who have studied the trade-offs can make these decisions based on their actual requirements rather than on what is currently fashionable.

The third change is security awareness as a constant disposition. SQL injection, cross-site scripting, cross-site request forgery, broken authentication, excessive data exposure — the OWASP Top 10 vulnerabilities are not exotic corner cases. They appear in production applications built without security consciousness by skilled engineers. The practitioner who has internalized the OWASP vulnerability classes instinctively sanitizes inputs, uses parameterized queries, validates JWT signatures, applies the principle of least privilege to API responses, and implements proper CORS policies without requiring a separate security review phase.

The fourth change is the ability to reason about production behavior. Development environments are controlled and forgiving; production environments are not. The practitioner who has studied production operations — log aggregation, metrics, distributed tracing, error tracking — thinks about observability during development rather than after deployment. They instrument their code, define meaningful metrics, and structure logs for queryability before something goes wrong.

Resources

Books and Texts

Kleppmann’s Designing Data-Intensive Applications (O’Reilly, 2017) — referenced throughout this guide — is the most valuable single text for backend and distributed systems understanding. Its treatment of databases, replication, distributed transactions, stream processing, and consistency is more depth than any web development text provides and is directly relevant to any application that handles significant data.

Fowler’s Patterns of Enterprise Application Architecture (Addison-Wesley, 2002) is old and remains valuable for understanding the architectural patterns — Active Record, Data Mapper, Repository, Unit of Work, Service Layer, Façade — that web frameworks implement and that experienced engineers reach for by name. The patterns are framework-agnostic and have proven durable.

The OWASP Testing Guide and OWASP Application Security Verification Standard (both free at owasp.org) are the authoritative application security references. Supplementing them with the OWASP Top 10 documentation provides the vocabulary for discussing and auditing application security.

Newman’s Building Microservices (2nd ed., O’Reilly, 2021) and Monolith to Microservices cover the dominant architectural choice in contemporary cloud-deployed applications — when microservices make sense, how to decompose a monolith, how to handle inter-service communication and data ownership. Read alongside the argument for monolith-first development (see Traps).

For frontend specifically, Dodds’s Epic React course and Testing JavaScript course (kentcdodds.com, paid but frequently discounted) provide the most comprehensive structured modern React development education. The courses are built around patterns rather than specific APIs, making them more durable than documentation-based learning.

MDN Web Docs (developer.mozilla.org, free) is the authoritative reference for web standards — HTML, CSS, JavaScript, Web APIs. It is the first place to check for accurate information about any web standard, more reliable than any book for current behavior.

Book/Resource Role Type
Kleppmann, Designing Data-Intensive Applications Backend and distributed systems depth Depth
Fowler, Patterns of Enterprise Application Architecture Architectural patterns vocabulary Depth
OWASP Testing Guide + ASVS (free) Application security reference Reference
Newman, Building Microservices (2nd ed.) Microservices architecture Depth
MDN Web Docs (free) Authoritative web standards reference Reference
React Docs (free) Modern React official learning path Practice
TypeScript Handbook (free) TypeScript language reference Reference
Crockford, JavaScript: The Good Parts JavaScript conceptual foundation Auxiliary
Beck, Test-Driven Development Testing discipline foundation Depth
Haverbeke, Eloquent JavaScript (3rd ed., free) JavaScript depth Entry
Osmani, Learning JavaScript Design Patterns (free) JavaScript patterns Depth

Courses and Lectures

The Odin Project (theodinproject.com, free) is the most comprehensive free full-stack web development curriculum. Its project-based approach and careful progression from fundamentals through frameworks, databases, and deployment — rather than jumping directly to framework tutorials — produces genuine understanding.

Full Stack Open (fullstackopen.com, free, University of Helsinki) provides structured coverage of modern JavaScript web development: React, Node.js, MongoDB, GraphQL, TypeScript, React Native. University-quality with project assignments.

Josh W Comeau’s CSS for JavaScript Developers (joshwcomeau.com, paid) is the most effective treatment of CSS for developers who think algorithmically — explaining the layout algorithms (normal flow, flexbox, grid, positioned layout, stacking contexts) rather than just the properties, which is the mental model that makes CSS predictable.

The PostgreSQL documentation and specifically the PostgreSQL Internals section are the best free resources for understanding what a production relational database is actually doing — how query planning works, how indexes are used, how MVCC provides isolation, how VACUUM works.

Course Platform Type
The Odin Project (free) theodinproject.com Practice
Full Stack Open (free) fullstackopen.com Entry
Josh W Comeau CSS for JS Developers joshwcomeau.com Depth
PostgreSQL documentation (free) postgresql.org/docs Reference

Practice, Tools, and Current Sources

Browser DevTools (built into every browser) are the essential diagnostic tools for web development. Proficiency with the Network tab (inspect HTTP requests, headers, timing), the Elements/Inspector tab (inspect and modify DOM and CSS), the Console (execute JavaScript, inspect errors), the Performance tab (flame graphs, rendering timeline), and the Application tab (inspect cookies, local storage, service workers) transforms debugging from guessing to evidence-based diagnosis.

Postman or Insomnia (both have free tiers) provide GUI environments for constructing and testing HTTP requests to APIs. Using these tools — alongside curl on the command line — to interact with APIs directly, before writing application code, clarifies what the API provides and eliminates the question of whether a problem is in the API or in the client code.

PostgreSQL with EXPLAIN ANALYZE is the most productive database learning environment. Creating tables, writing queries, running EXPLAIN ANALYZE to see the query plan, and comparing the plan before and after adding indexes makes the abstract concepts of query optimization concrete. The pgbench tool provides simple load testing.

Playwright (Microsoft, free) or Cypress (free tier) provide browser automation for end-to-end testing. Writing a Playwright test that exercises a user journey — login, navigate to a page, fill a form, verify the result — and running it in CI produces automated verification that the application works from the user’s perspective.

Reading production open-source web applications — Discourse (a forum platform, 50k+ lines of Ruby and JavaScript), GitLab (a full-featured platform), Cal.com (a scheduling application) — is one of the most concentrated ways to develop architectural judgment. These applications have been written and evolved by experienced teams under production constraints and demonstrate the patterns and compromises that real applications require.

Resource Platform Type
Browser DevTools (free, built-in) Chrome / Firefox / Safari Practice
Postman / Insomnia / curl (free) Various Practice
PostgreSQL + EXPLAIN ANALYZE (free) postgresql.org Practice
Playwright / Cypress (free) playwright.dev / cypress.io Practice
Discourse / GitLab / Cal.com source GitHub Reference
Chrome Lighthouse (free, built-in) Chrome DevTools Practice
web.dev Learn Performance (free) web.dev Practice
HTTP Archive / Web Almanac (free) httparchive.org Reference

Traps

Trap Why it misleads Better response
Framework-first learning Learning React, or Django, or Rails, before understanding HTTP, SQL, and how browsers work produces practitioners who can follow tutorials and fail on problems the tutorial did not anticipate. Frameworks abstract the substrate; without the substrate, the abstraction is opaque. Learn HTML, CSS, and vanilla JavaScript before a JavaScript framework. Learn SQL and basic database operations before an ORM. Write a simple HTTP server in your language’s standard library before a web framework. These foundations make frameworks legible and their conventions meaningful.
Cargo-culting microservices Microservices solve real problems at scale: independent deployability, team autonomy, technology heterogeneity. They introduce real costs at smaller scale: network calls instead of function calls, distributed tracing instead of stack traces, eventual consistency instead of ACID transactions, operational complexity that requires dedicated platform engineering. Many applications that are microservices should be monoliths. Martin Fowler’s “MonolithFirst” essay articulates why: start with a monolith, identify the natural seam lines as the system grows, and extract services along those seams if deployment independence becomes necessary. The anti-pattern is beginning with microservices before understanding the domain well enough to draw the right boundaries.
ORMs as an abstraction ceiling ORMs make common database operations more convenient and can make complex operations more obscure. N+1 query problems — one query per row in a result set, instead of one query for all rows — are the most common ORM-introduced performance problem, and they are invisible at the ORM API level. The solution is not to avoid ORMs, but to understand what SQL the ORM is generating. Enable query logging in your ORM for development. Run EXPLAIN ANALYZE on slow queries. Learn to write the SQL that the ORM is generating, so you can identify when the ORM’s choice is wrong and override it. The ORM is a convenience, not a substitute for understanding the database.
Neglecting security until deployment Secure development habits are harder to retrofit than to establish from the start. An application built with parameterized queries from the beginning has no SQL injection surface; an application whose queries are retrofitted has some percentage of missed cases. An application that implements proper authentication from the beginning has no authentication bypass surface; one whose authentication is patched has gaps in the coverage. Apply OWASP’s security controls at design time: parameterized queries instead of string interpolation, output encoding instead of trusting input, authentication checks in middleware rather than per-handler, authorization checks on every data access. These are not features to add; they are disciplines to establish.
Ignoring the CSS layout model CSS is often approached as property soup — “add properties until it looks right.” This produces brittle layouts that break under different screen sizes, content lengths, or browser rendering. The CSS layout model has structure: normal flow, flexbox, grid, and positioned layout are distinct algorithms with specific behaviors. Study the CSS layout algorithms rather than the properties. The properties make sense once you understand what algorithm applies them. Josh W. Comeau’s CSS for JavaScript Developers restructures CSS learning around the algorithms, which is the mental model that makes layout predictable.
Using AI assistance to avoid understanding AI coding assistants produce working code for many common patterns. Developers who use them to avoid understanding what the code does produce applications they cannot debug when something unexpected happens. AI-generated code that is wrong or subtly insecure is indistinguishable from correct code without the understanding to evaluate it. Use AI assistance to accelerate work in areas where competence already exists, not to bypass learning in areas where it does not. The debugging question “why is this not working?” requires understanding what it is supposed to be doing; understanding comes from studying the underlying concepts, not from accepting AI output uncritically. When an AI generates code you do not understand, that is a signal to understand it before using it in production.

中文

Web 与应用开发是一门构建运行在网络之上、并被人使用的软件的学科——网站、Web 服务、移动应用、API,以及云端部署系统。这个描述覆盖了当代软件工程工作的绝大部分,这意味着这门学科的宽度并不是偶然的:它正是软件工程师这份工作实际要求的东西。挑战在于,单纯的广度并不等于掌握。一个实践者知道 HTML、一个 JavaScript 框架、一个后端框架、一个数据库驱动和一个部署工具,只是拼起了一套工具箱;他还未必理解这些工具在做什么,为什么它们会以这种方式相互作用,以及如何推理新情况。

把工具箱知识转化为真正掌握的,是对机制和原则层面的理解。HTTP 不只是浏览器使用的协议;它是一份契约,定义了什么是 Web request,headers 可以包含什么,缓存如何工作,认证 headers 如何流动,以及 status codes 如何传达语义。关系数据库不只是存储数据并响应 SQL 的东西;它是一个具有特定事务语义、特定索引结构和特定隔离级别的系统,其中索引结构决定查询性能,隔离级别决定并发事务可以看到什么。前端框架也不是让浏览器更新屏幕的魔法;它管理 component tree,通过协调 virtual DOM,或使用细粒度响应性判断什么发生了变化,并发出最少 DOM 操作,使浏览器与应用状态同步。理解工具正在做什么,会让故障可诊断,让架构决策有依据,也让工具迁移成为可能,因为底层概念可以迁移。

本指南其余部分在直接意义上都是本章的前置知识。数据库(§4.4)解释数据库实际上在做什么。计算机网络(§4.3)覆盖 Web 应用建立其上的协议。操作系统(§4.2)覆盖服务器运行的环境。安全(§4.5)覆盖 Web 应用暴露出的攻击面。HCI(§6.2)覆盖如何设计 Web 应用暴露给用户的界面。分布式系统(§4.6)覆盖云端部署、多服务应用在规模化之后是什么。Web 开发作为一门学科整合了所有这些内容;它自己的特定贡献,是这种整合本身,以及数十年实践中形成的应用层模式。

前置知识:编程(§2.1)——至少熟练掌握一门语言。网络(§4.3)——HTTP、DNS、TLS。数据库(§4.4)——SQL、事务。安全(§4.5)——应用安全问题与本章直接重叠。

从静态页面到 AI 集成应用

World Wide Web 由 Tim Berners-Lee 于 1989–1991 年在 CERN 发明,最初是为物理学家设计的信息共享系统。最初设计很简单:HTML 文档,HTTP 作为传输协议,URL 作为地址。第一个 Web 浏览器 WorldWideWeb(后来改名为 Nexus)也由 Berners-Lee 完成。原初 Web 完全是静态的:服务器提供预先写好的 HTML 文件,浏览器显示它们,没有交互机制,也没有动态内容。这种简单性促成了爆发式采用。到 1993 年,CERN 使 Web 技术免费供任何人使用;两年之内,网站数量已经达到数万。

通向动态内容的第一步是 Common Gateway Interface(CGI),它于 1993 年标准化。CGI 允许 Web 服务器执行任意程序,并把程序输出作为 Web 内容返回——程序可以查询数据库、计算响应,并返回 HTML。这很慢(每个请求启动一个新进程),也很原始(用字符串操作生成 HTML),但它证明了这个概念。Perl 成为主导性 CGI 语言,因为它的文本处理能力非常适合生成 HTML。1990 年代中期 CGI 脚本爆发式增长,而且往往写得很差、不安全,这也催生了关于 Web 应用安全的最早一批系统性思考。

1990 年代后期的服务端脚本语言——PHP(1994)、ASP(1996)、ColdFusion 及其同时代技术——让动态 Web 开发更容易。尤其是 PHP:它把代码直接嵌入 HTML 文件中,并拥有简单部署模型(把 .php 文件放到服务器上,它就会运行)。这种可用性推动了采用;早期 WordPress、Wikipedia 和 Facebook 都由 PHP 驱动。2000 年左右的 PHP 应用架构是单体的:所有代码都在一台服务器上,通常每个页面对应一个文件,数据库查询内联在 HTML 生成过程中,认证和授权则是临时拼接的逻辑。它在小规模时有效,在大规模时会崩溃。

Ruby on Rails 框架(2004)是一次关键介入,它为一代实践者把系统性架构思维引入 Web 开发。David Heinemeier Hansson 从 Basecamp 中抽取出 Rails,并将其作为一个 opinionated framework 发布,框架体现了 Model-View-Controller 架构、convention over configuration,以及 DRY(Don’t Repeat Yourself)原则。你不需要选择文件放在哪里,也不需要选择数据库表如何命名——Rails 有约定,遵循这些约定,就可以快速构建很大一类 Web 应用。Rails 的 ORM ActiveRecord 把 SQL 抽象成 Ruby 对象。Scaffolding 会生成样板代码。那个展示十五分钟内构建一个 blog 的 Rails demo,是 Web 开发史上最有效的框架营销,并推动了巨大采用。

Ajax——Asynchronous JavaScript and XML,这个名称由 Jesse James Garrett 于 2005 年提出,但相关技术自 1997 年 Internet Explorer 4 起就已经可用——改变了交互模型。此前每个操作都需要完整页面刷新;现在 JavaScript 可以在后台发出 HTTP 请求,并更新页面的一部分。Google Maps(2005)和 Gmail(2004)展示了这带来的可能性:应用开始感觉响应迅速、像原生应用,而不是一页一页跳转。Web 不再只是文档媒介,而变成了应用平台。随后出现的框架——用于 DOM 操作的 jQuery(2006),以及用于客户端结构的 Backbone.js(2010)——构成了第一代前端 JavaScript 架构。

Ryan Dahl 于 2009 年创建的 Node.js,把 JavaScript 放到了服务器端。其动机是 I/O 性能:Node.js 不用线程阻塞等待 I/O 操作(数据库查询、网络请求),而是使用 event loop,让一个线程异步处理大量并发 I/O 操作。这使 Node.js 对高并发、I/O-bound 工作负载异常高效,并带来一个次要好处:开发者可以在客户端和服务器端使用同一种语言。Node.js 生态系统 npm 爆发式增长;到 2010 年代中期,npm 已经是现存最大的包仓库。

React(2013)由 Facebook 作为开源库发布,它引入了 component model 和 virtual DOM diffing,塑造了之后十年的前端开发。核心思想是:UI 是状态的函数。给定当前应用状态,UI 可以被确定性地渲染;当状态变化时,库会计算最小 DOM 更新集合,使显示与状态同步。这种模型——声明式 UI 是状态函数,单向数据流——相比 jQuery 时代的命令式 DOM 操作,使大型应用更可预测,也更容易推理。React 采用速度很快:到 2016 年,它已经成为主导性前端库;到 2020 年,React 生态(用于服务端渲染的 Next.js、用于移动端的 React Native,以及丰富的组件库生态)已经成为行业许多场景中的默认栈。

TypeScript(Microsoft,2012,自 2017 年起广泛采用)为 JavaScript 增加了静态类型。其采用模式很不寻常:JavaScript 在整个历史中都是动态类型语言,而在拥有数百万包和数十亿行代码的生态中,为它事后添加类型,需要一种渐进方式。TypeScript 的可选、渐进类型方式成功了。到 2023 年,大多数主要 JavaScript 项目已经迁移到 TypeScript,类型检查已经成为标准开发工作流的一部分,数千个 JavaScript 包的类型定义也由社区维护。TypeScript 并没有改变 JavaScript 能做什么;它改变的是开发者在运行代码之前,能够静态知道什么。

Serverless 和 edge computing 时代,大约从 2014 年 AWS Lambda 开始,并在 2010 年代后期和 2020 年代加速,改变了部署模型。开发者不再 provision 和管理服务器,而是部署运行在托管基础设施上的函数。平台处理扩展(自动从零扩展到数千实例)、运维问题(无需服务器管理)和计费(按调用付费,而不是按服务器付费)。Edge deployment——在地理上靠近用户的内容分发网络节点上运行代码——把这一点扩展为更接近用户的计算,从而降低全球分布式应用的延迟。Cloudflare Workers、Vercel Edge Functions 和类似平台使 edge deployment 变得可用。

AI 集成时代从 2020 年 OpenAI API 发布开始真正到来,并在 2022–2023 年 ChatGPT 以及后续强语言模型 API 大量出现后急剧加速。对 Web 应用开发来说,这引入了一种新模式:应用代码调用 AI APIs 来完成自然语言处理、内容生成、代码辅助和推理任务。RAG(Retrieval-Augmented Generation)模式——检索相关上下文并传给语言模型——成为构建 AI 搜索和问答功能的标准方式。Agent 模式——AI 系统可以调用工具(API、数据库、代码执行),并执行一系列行动——则成为下一前沿。框架工具(LangChain、LlamaIndex、Vercel AI SDK)快速成熟,full-stack developer 的技能集现在也包括把语言模型能力集成进生产应用的能力。

架构、性能与全栈

HTTP 层:理解协议

HTTP 是 Web 应用开发的底层基质。每个 Web request 都是一次 HTTP transaction:一个 method(GET、POST、PUT、DELETE、PATCH)、一个 URL、headers,以及可选 body。每个 response 都有 status code、headers 和 body。每个缓存决策、每种认证方案、每条 CORS policy 都通过 HTTP headers 运作。一个不理解 HTTP 的 Web 开发者,是在使用一个机制不透明的工具。

HTTP/1.1(1997)引入了持久连接(一个 TCP 连接可以服务多个请求)和 chunked transfer encoding。HTTP/2(2015)引入了 multiplexing(一个 TCP 连接上有多个并发请求)和 header compression,显著降低了加载大量资源页面的延迟。HTTP/3(2022)用 QUIC 替代 TCP,消除了 head-of-line blocking,并改善了丢包网络上的性能。

REST(Representational State Transfer)是大多数 Web API 所体现的架构风格。它的定义性约束包括:无状态性(每个请求都包含处理所需的全部信息,服务器不保存 session state)、统一接口(基于资源的一致 URL,并使用标准 HTTP verbs),以及分层系统(load balancers 和 caches 等中间层对请求透明)。REST 的实践好处包括:URL 结构可预测、可以利用 HTTP 缓存,并通过标准 methods 和 status codes 实现互操作。REST 的实践弱点包括:over-fetching(endpoint 返回所有字段,而客户端只需要其中一部分)和 under-fetching(需要多个请求才能组装相关数据)。GraphQL(2015,Facebook)通过允许客户端在单个 query 中准确请求所需字段来处理这些问题。

Authentication 和 authorization 是彼此正交的 HTTP 关切。Authentication 确认身份:是谁在发出这个请求?Authorization 确认权限:这个身份是否被允许做这件事?基于 cookie 的 sessions(传统方法)把 session state 存在服务端;浏览器提交 session cookie,服务器查找关联用户。基于 token 的认证——JWT(JSON Web Tokens)是最常见形式——把 session state 以签名 token 形式存储在客户端;服务器验证签名并读取 claims。OAuth 2.0 提供的是 delegated authorization 协议:允许用户授权第三方应用访问其在另一个服务上的账户,而不共享凭据。

状态管理:应用开发中最困难的问题

应用状态是决定应用显示什么、以及如何响应用户行动的数据。正确管理状态——维护显示数据与真实数据之间的一致性,处理并发更新,从失败中恢复,高效更新——是大多数应用 bug 所在之处。

在前端,状态管理最初是临时性的(jQuery 时代),随着应用变复杂才被系统化。Flux(Facebook,2014)和 Redux(2015)引入了单向数据流:actions 描述发生了什么,reducers 根据旧状态和 action 推导出新状态,store 保存当前状态,view 由 store 派生。这使状态变更显性化、可追踪。代价是冗长。React 的 Context API 和 hooks(useState、useReducer、useContext)使本地和半全局状态管理在不引入单独库的情况下变得更自然。后来生态变得分裂——Zustand、Jotai、Recoil、Valtio 等——这反映了关于正确模型的真实分歧。

Server state 和 client state 是不同关切。Client state——例如“这个 modal 是否打开”这样的 UI 状态——属于组件。Server state——从 API 获取的数据,本地缓存,与服务器同步——具有不同特性:当服务器变化时必须被 invalidated,可能是 stale 的,也可能被多个组件共享。React Query 和 SWR 这类库专门处理 server state management,开箱提供缓存、后台重新获取和 optimistic updates。

Real-time state——必须反映服务器端变化的状态——需要持久连接机制。WebSocket 在一个持久 TCP 连接上提供全双工通信;当数据变化时,server events 会推送数据,无需轮询。Server-Sent Events(SSE)通过 HTTP 提供单向 server-to-client streaming,在不需要双向通信时比 WebSocket 更简单。Long polling(客户端发出请求,服务器保持请求打开直到有变化)则是在 WebSocket 不可用环境中的 fallback。

性能:测量取代直觉的地方

Web 性能以与用户体验相关的指标衡量:Time to First Byte(TTFB)、First Contentful Paint(FCP)、Largest Contentful Paint(LCP)、Cumulative Layout Shift(CLS)和 Interaction to Next Paint(INP)。Google 的 Core Web Vitals 于 2021 年成为搜索排名因素,使这些指标具有经济意义:慢页面会失去搜索可见性。Web 开发中,可测性能与商业结果之间的联系异常清晰,因此性能优化是可操作的。

前端性能改进遵循一个层级。网络优化减少传输大小(minification、compression、image optimization、HTTP/2 multiplexing、CDN caching),并减少 round trips(resource hints、preloading、用于离线和 prefetch 的 service workers)。JavaScript bundle size 会直接影响 parse 和 execution time;tree shaking(移除未使用代码)、code splitting(只加载当前 route 所需代码)和 lazy loading(按需加载组件)是标准技术。Critical rendering path optimization 确保浏览器能够尽快渲染 above-the-fold 内容,方法是最小化 render-blocking resources。

后端性能瓶颈需要通过 profiling 诊断。Web 应用中的大多数性能问题都与数据库有关:缺少 indexes 导致 full table scans,N+1 query patterns(列表中每个结果发一次查询),ORM 抽象层隐藏查询成本导致查询数量过多。多级缓存——database query cache、application-level cache(Redis、Memcached)、CDN edge cache——可以降低数据库负载并改善响应时间。Connection pooling 确保建立新的数据库连接这种昂贵操作不会主导请求延迟。

数据库连接池是一个很好的例子:它在教程级开发中不可见,但在生产中至关重要。每个 Web server instance 维护一组持久数据库连接;进入的请求从池中获取一个连接,使用它,然后归还。没有连接池,每个请求都要打开和关闭一个 TCP 连接,并完成 handshake——对于一个只需数毫秒的查询,这会带来数百毫秒开销。连接池大小是关键调优参数:太小,请求会排队等待连接;太大,则会超过数据库服务器的连接限制。

学习这一部分会改变什么

Web 与应用开发会改变实践者如何构建真正被使用的软件。

第一个变化,是协议素养:能够阅读并推理 HTTP 层发生了什么。一个能在浏览器 DevTools 中阅读 HTTP headers、理解 401 和 403 response 分别意味着什么以及为什么不同、能追踪 CORS error 到其来源、理解 Cache-Control header 语义的实践者,能够诊断大量生产问题;而这些问题对只停留在框架 API 层面的实践者是不可见的。

第二个变化,是架构判断。单体和 microservices 之间的选择,server-side rendering 和 client-side rendering 之间的选择,关系数据库和 document store 之间的选择,REST 和 GraphQL 之间的选择——这些都是工程决策,其后果会在产品开发的多年过程中展开。学习过这些权衡的实践者,会基于实际需求做出决策,而不是基于当前流行趋势。

第三个变化,是把安全意识作为一种持续姿态。SQL injection、cross-site scripting、cross-site request forgery、broken authentication、excessive data exposure——OWASP Top 10 vulnerabilities 不是罕见边缘情况。它们会出现在缺乏安全意识、但由熟练工程师构建的生产应用中。真正内化 OWASP 漏洞类别的实践者,会本能地清理输入、使用 parameterized queries、验证 JWT signatures、对 API responses 应用最小权限原则,并实现正确 CORS policies,而不需要单独的安全审查阶段。

第四个变化,是推理生产环境行为的能力。开发环境可控且宽容;生产环境不是。学习过生产运维——log aggregation、metrics、distributed tracing、error tracking——的实践者,会在开发期间,而不是部署之后,思考 observability。他们会在出问题之前就为代码添加 instrumentation,定义有意义的 metrics,并把 logs 结构化到可查询。

资源

书籍与文本

Kleppmann 的 Designing Data-Intensive Applications(O’Reilly,2017)——本指南中反复引用——是理解后端和分布式系统最有价值的单本文本。它对数据库、复制、分布式事务、流处理和一致性的处理,比任何 Web 开发文本都更深入,并且直接相关于任何处理大量数据的应用。

Fowler 的 Patterns of Enterprise Application Architecture(Addison-Wesley,2002)已有年头,但仍然有价值,因为它解释了 Web 框架实现、经验工程师也会按名称调用的架构模式——Active Record、Data Mapper、Repository、Unit of Work、Service Layer、Façade。这些模式与框架无关,并已经证明足够持久。

OWASP Testing GuideOWASP Application Security Verification Standard(二者均可在 owasp.org 免费获取)是权威应用安全参考。配合 OWASP Top 10 文档,可以提供讨论和审计应用安全的词汇。

Newman 的 Building Microservices(第 2 版,O’Reilly,2021)和 Monolith to Microservices 覆盖了当代云端部署应用中的主导架构选择——microservices 什么时候合理,如何拆分单体,如何处理服务间通信和数据所有权。阅读时应同时阅读 monolith-first development 的论证(见 Traps)。

对于前端,Dodds 的 Epic React 课程和 Testing JavaScript 课程(kentcdodds.com,付费但经常打折)提供最完整的现代 React 开发结构化教育。这些课程围绕模式构建,而不是围绕特定 API,因此比基于文档的学习更持久。

MDN Web Docs(developer.mozilla.org,免费)是 Web 标准的权威参考——HTML、CSS、JavaScript、Web APIs。关于任何 Web 标准的准确信息,它都是第一查询地点,比任何书都更可靠地反映当前行为。

书籍/资源 作用 类型
Kleppmann, Designing Data-Intensive Applications 后端与分布式系统深度 深入
Fowler, Patterns of Enterprise Application Architecture 架构模式词汇 深入
OWASP Testing Guide + ASVS(免费) 应用安全参考 参考
Newman, Building Microservices(第 2 版) Microservices 架构 深入
MDN Web Docs(免费) 权威 Web 标准参考 参考
React Docs(免费) 现代 React 官方学习路径 实践
TypeScript Handbook(免费) TypeScript 语言参考 参考
Crockford, JavaScript: The Good Parts JavaScript 概念基础 辅助
Beck, Test-Driven Development 测试纪律基础 深入
Haverbeke, Eloquent JavaScript(第 3 版,免费) JavaScript 深度 入门
Osmani, Learning JavaScript Design Patterns(免费) JavaScript 模式 深入

课程与讲座

The Odin Project(theodinproject.com,免费)是最完整的免费 full-stack Web 开发课程。它采用项目式方法,并从基础到框架、数据库和部署仔细推进——而不是直接跳到框架教程——能够产生真正理解。

Full Stack Open(fullstackopen.com,免费,University of Helsinki)系统覆盖现代 JavaScript Web 开发:React、Node.js、MongoDB、GraphQL、TypeScript、React Native。它具有大学课程质量,并配有项目作业。

Josh W Comeau 的 CSS for JavaScript Developers(joshwcomeau.com,付费)是面向算法型思维开发者的最有效 CSS 处理方式——它解释 layout algorithms(normal flow、flexbox、grid、positioned layout、stacking contexts),而不只是列举属性;这正是让 CSS 变得可预测的心智模型。

PostgreSQL documentation,尤其是 PostgreSQL Internals 部分,是理解生产级关系数据库实际在做什么的最佳免费资源——query planning 如何工作,indexes 如何被使用,MVCC 如何提供隔离,VACUUM 如何工作。

课程 平台 类型
The Odin Project(免费) theodinproject.com 实践
Full Stack Open(免费) fullstackopen.com 入门
Josh W Comeau CSS for JS Developers joshwcomeau.com 深入
PostgreSQL documentation(免费) postgresql.org/docs 参考

实践、工具与当前资料

Browser DevTools(内置于每个浏览器)是 Web 开发的必要诊断工具。熟练使用 Network tab(检查 HTTP requests、headers、timing)、Elements/Inspector tab(检查和修改 DOM 与 CSS)、Console(执行 JavaScript、检查 errors)、Performance tab(flame graphs、rendering timeline)和 Application tab(检查 cookies、local storage、service workers),会把调试从猜测转变为基于证据的诊断。

PostmanInsomnia(二者都有免费层)提供构造和测试 HTTP requests 到 APIs 的 GUI 环境。使用这些工具——同时配合命令行中的 curl——在编写应用代码之前直接与 APIs 交互,可以澄清 API 提供了什么,并消除“问题是在 API 里,还是在客户端代码里”的不确定性。

PostgreSQL with EXPLAIN ANALYZE 是最高效的数据库学习环境。创建表、编写 queries、运行 EXPLAIN ANALYZE 查看 query plan,并比较添加 indexes 前后的 plan,会使 query optimization 的抽象概念变得具体。pgbench 工具提供简单 load testing。

Playwright(Microsoft,免费)或 Cypress(免费层)提供浏览器自动化,用于端到端测试。编写一个 Playwright test 来执行一条用户路径——登录、导航到页面、填写表单、验证结果——并在 CI 中运行它,可以从用户视角自动验证应用确实工作。

阅读生产级开源 Web 应用——Discourse(论坛平台,50k+ 行 Ruby 和 JavaScript)、GitLab(全功能平台)、Cal.com(日程安排应用)——是发展架构判断力最高密度的方式之一。这些应用由经验团队在生产约束下编写并演化,展示真实应用所需的模式与妥协。

资源 平台 类型
Browser DevTools(免费,内置) Chrome / Firefox / Safari 实践
Postman / Insomnia / curl(免费) Various 实践
PostgreSQL + EXPLAIN ANALYZE(免费) postgresql.org 实践
Playwright / Cypress(免费) playwright.dev / cypress.io 实践
Discourse / GitLab / Cal.com source GitHub 参考
Chrome Lighthouse(免费,内置) Chrome DevTools 实践
web.dev Learn Performance(免费) web.dev 实践
HTTP Archive / Web Almanac(免费) httparchive.org 参考

陷阱

陷阱 为什么会误导 更好的回应
框架优先学习 在理解 HTTP、SQL 和浏览器如何工作之前就学习 React、Django 或 Rails,会培养出能跟教程、但在教程没预料到的问题上失败的实践者。框架抽象底层基质;没有底层基质,抽象就是不透明的。 在学习 JavaScript 框架之前,先学习 HTML、CSS 和原生 JavaScript。在学习 ORM 之前,先学习 SQL 和基础数据库操作。在学习 Web 框架之前,先用你所用语言的标准库写一个简单 HTTP server。这些基础会让框架变得可读,也会让它们的约定有意义。
照搬式 microservices Microservices 确实能在规模化场景中解决真实问题:独立部署、团队自主、技术异质性。但它们在较小规模中也引入真实成本:网络调用替代函数调用,distributed tracing 替代 stack traces,eventual consistency 替代 ACID transactions,运维复杂度需要专门平台工程支持。许多采用 microservices 的应用,本应是 monoliths。 Martin Fowler 的 “MonolithFirst” 文章解释了原因:从单体开始,随着系统增长识别自然边界,如果部署独立性变得必要,再沿这些边界抽取服务。反模式是在尚未充分理解领域、无法画出正确边界之前,就从 microservices 开始。
把 ORM 当成抽象天花板 ORM 让常见数据库操作更方便,也会让复杂操作更不透明。N+1 query problems——结果集中每一行发一次 query,而不是一次 query 取回所有行——是 ORM 引入的最常见性能问题,并且在 ORM API 层面不可见。解决方案不是避免 ORM,而是理解 ORM 正在生成什么 SQL。 在开发环境中启用 ORM query logging。对慢查询运行 EXPLAIN ANALYZE。学习编写 ORM 正在生成的 SQL,这样才能识别 ORM 的选择什么时候是错的,并覆盖它。ORM 是便利工具,不是理解数据库的替代品。
到部署前才考虑安全 安全开发习惯一开始建立,比事后修补容易得多。一个从一开始就使用 parameterized queries 的应用没有 SQL injection 攻击面;一个事后改造查询的应用,总会有一定比例遗漏。一个从一开始就正确实现认证的应用没有认证绕过面;一个事后补 authentication 的应用会有覆盖缺口。 在设计阶段应用 OWASP 安全控制:用 parameterized queries 替代字符串插值,用 output encoding 替代信任输入,把认证检查放在 middleware 中而不是每个 handler 中零散处理,在每次数据访问时做 authorization checks。这些不是要添加的功能,而是要建立的纪律。
忽视 CSS layout model CSS 经常被当成 property soup——“不断加属性,直到看起来对”。这会产生脆弱布局,在不同屏幕尺寸、内容长度或浏览器渲染下崩坏。CSS layout model 有结构:normal flow、flexbox、grid 和 positioned layout 是具有特定行为的不同算法。 学习 CSS layout algorithms,而不只是属性。一旦你理解了哪个算法在应用这些属性,属性才会变得有意义。Josh W. Comeau 的 CSS for JavaScript Developers 会围绕算法重组 CSS 学习,这正是使布局可预测的心智模型。
用 AI 辅助逃避理解 AI coding assistants 可以为许多常见模式生成可运行代码。开发者如果用它们逃避理解代码在做什么,就会构建出一旦出现意外情况就无法调试的应用。AI 生成代码中的错误或微妙安全问题,如果没有评估它的理解能力,就无法与正确代码区分。 在已经具备能力的领域中使用 AI 辅助来加速工作,而不是在不具备能力的领域中绕过学习。调试问题“为什么这不起作用?”要求你理解它本来应该做什么;理解来自学习底层概念,而不是不加批判地接受 AI 输出。当 AI 生成了你不理解的代码,这就是一个信号:在投入生产之前,先理解它。