{ "schema": "auturnal.deep-research-review.v2", "slug": "consensus-safety-progress", "title": "Consensus Beyond the Happy Path: Safety, Progress, and the Cost of Quorums", "category": "Distributed systems", "dek": "Quorum intersection is a safety ingredient; timely progress requires a different argument.", "abstract": "A replicated service is useful only if it preserves a coherent history while continuing to answer requests under stated conditions. These requirements are often compressed into a single claim of fault tolerance, obscuring the distinction between safety and progress. This review connects the asynchronous consensus impossibility result, Paxos, and Raft to a deliberately small model of quorum availability. It derives how majority size changes the probability of collecting acknowledgements and constructs a counterexample to treating quorum intersection as a complete consensus proof. The analysis separates crash recovery, membership change, client retries, and reads from the simpler task of choosing one value. Its main conclusion is methodological: every operational availability claim should name the failure model, persistence boundary, communication assumptions, and observation interval. The supplied examples are mathematical constructions, not measurements of a production consensus implementation.", "english": "A replicated service is useful only if it preserves a coherent history while continuing to answer requests under stated conditions. These requirements are often compressed into a single claim of fault tolerance, obscuring the distinction between safety and progress. This review connects the asynchronous consensus impossibility result, Paxos, and Raft to a deliberately small model of quorum availability. It derives how majority size changes the probability of collecting acknowledgements and constructs a counterexample to treating quorum intersection as a complete consensus proof. The analysis separates crash recovery, membership change, client retries, and reads from the simpler task of choosing one value. Its main conclusion is methodological: every operational availability claim should name the failure model, persistence boundary, communication assumptions, and observation interval. The supplied examples are mathematical constructions, not measurements of a production consensus implementation.", "refs": [ "cs-01-1", "cs-01-2", "cs-01-3" ], "sections": [ [ "The question that a fault-tolerance claim must answer", "Consider a service that stores the current version of a scientific manuscript. An editor sends a replacement, loses the connection, and subsequently reads from another replica. The important question is not how many servers remain running. It is which completed operations the new read must reflect, whether an uncertain write may later become visible, and under what communication conditions the service must answer. A count of healthy processes answers none of these questions by itself.\nWe distinguish agreement about a chosen log entry from linearizable execution of an application. Agreement constrains replicas; linearizability also relates completed client operations to real-time order. A protocol can protect its committed log yet expose a stale read through an inadequately guarded read path. This distinction sets the scope of the review: a safety proof is about permitted histories, whereas an availability model is about the probability or eventuality of reaching one of those histories.", [ "cs-01-2", "cs-01-3" ] ], [ "Three primary results, three different roles", "Fischer, Lynch, and Paterson establish a nontermination possibility for deterministic consensus in a fully asynchronous message-passing model with even one crash failure. Their theorem is not a claim that consensus cannot usually finish, nor that randomized or timing-assisted protocols are impossible. Lamport's Paxos exposition explains how numbered proposals and persistent acceptor information preserve agreement while discussing progress separately. Ongaro and Ousterhout organize replicated-log consensus around leader election, replication, and safety, with a membership-change mechanism based on overlapping majorities.\nThese sources answer different questions. The impossibility result constrains universal termination promises. The protocols supply mechanisms for preserving already established decisions. Neither a majority count nor the presence of an election timeout, considered alone, establishes all the properties of the full system. We use this separation as a design discipline rather than as a ranking of the protocols.", [ "cs-01-1", "cs-01-2", "cs-01-3" ] ], [ "A small proof about intersecting majorities", "Let a configuration contain n replicas and let every decision require q acknowledgements from distinct replicas in that configuration. Two quorums must share at least 2q minus n members: otherwise their union would contain more than n distinct replicas. Thus a strict majority guarantees a nonempty intersection. For five replicas and quorum size three, the smallest overlap is one; for quorum size two, disjoint decision groups exist.\nThis elementary counting argument is necessary in many majority protocols but is not sufficient for agreement. The shared replica must obey a rule linking its earlier responses to its later responses. If it can forget an accepted proposal after restarting, or acknowledge conflicting decisions without a monotone ballot rule, the intersection carries no useful information. An implementation review should therefore follow the durable state of the intersecting replica rather than stopping at the set-theoretic lemma.", [] ], [ "An explicit counterexample to a common shortcut", "Take replicas A, B, and C. A and B acknowledge value x; later B and C acknowledge value y. Both groups are majorities. If B is allowed to acknowledge incompatible decisions and neither decision procedure checks an earlier commitment, x and y can both be declared chosen. There is no contradiction with the majority-intersection lemma: B is precisely the overlap, but the protocol has failed to assign the overlap a preserving obligation.\nThe executable appendix enumerates every pair of quorum subsets for n from three through nine and checks the minimum intersection. It also records this three-replica witness. That bounded enumeration is a check of the counting model, not a proof of Paxos or Raft. Its value is diagnostic: it prevents an apparently persuasive quorum diagram from being mistaken for a complete protocol argument.", [] ], [ "Availability under independent replica failures", "Suppose each replica is reachable with probability p in a specified observation window, independently of every other replica. Summing the binomial probabilities for at least q reachable replicas gives an idealized quorum-availability probability. With p equal to 0.9, a three-member majority is available with probability 0.972; a five-member majority gives 0.99144. Those numbers describe a synthetic independence model, not an uptime commitment.\nIndependence is the fragile assumption. Shared power, common routing, a bad deployment, or a single overloaded storage dependency can affect every member. If a common event disables the whole configuration with probability c, independently of the residual per-replica failures, availability becomes the binomial result multiplied by one minus c. Adding replicas cannot eliminate that common-mode ceiling. Choosing failure domains therefore matters differently from increasing replica count within one domain.", [] ], [ "Latency is an order statistic, until the assumptions fail", "For a second construction, let n independent acknowledgement times have exponential distributions with mean ten milliseconds. If a decision waits for the q-th acknowledgement, its completion distribution is the probability that at least q individual acknowledgements have arrived by time t. Substituting the individual arrival distribution into the same binomial expression yields the plotted curve.\nThis model intentionally omits the leader's request processing, durable-log flush, client round trip, retransmissions, and election work. It is a model of an abstract quorum collector, not a prediction of Raft commit latency. Correlated disk pauses would invalidate independence, and a fixed serialization delay would shift every curve. Reporting a measured latency distribution would require identifying which interval was timed and which storage operations completed before acknowledgement. Without that boundary, two apparently comparable latency numbers can describe different durability contracts.", [] ], [ "Timeouts provide suspicion, not a historical verdict", "A timeout says that a response did not arrive within an interval. It does not distinguish a crashed process from a delayed process, a delayed packet, or an overloaded receiver. In a fully asynchronous model there is no finite timeout that makes this distinction infallibly. Practical protocols can nevertheless obtain progress under stronger eventual timing or leader assumptions while retaining safety during mistaken suspicions.\nThe operational implication is to evaluate the two properties separately. A short election timeout may improve recovery from one clean crash while causing repeated leadership changes during a shared pause. A long timeout may reduce unnecessary elections but prolong a real outage. A useful experiment varies delay distributions and sustained load, records unsuccessful elections, and checks the client history throughout. A throughput graph that excludes the unstable interval does not establish recovery behavior.", [ "cs-01-1", "cs-01-2" ] ], [ "Recovery, membership, and reads expand the obligation", "Crash recovery changes the meaning of an acknowledgement because memory disappears while acknowledged promises must remain meaningful. Membership changes alter which sets are allowed to make decisions. A replica removed from a configuration must not later form an independent authority with old peers. Reads introduce another issue: a process that once led a cluster may continue running after losing the ability to communicate with a quorum.\nThese are distinct obligations, not implementation details that can be inferred from the happy-path append operation. A review should identify the configuration attached to each decision, the persisted metadata needed after restart, and the mechanism by which a reader establishes that its authority is current. The exact solution may use protocol-specific barriers or carefully justified leases; it must be examined within its own clock and failure assumptions. Merely reading the local log of a reachable server is insufficient.", [ "cs-01-2", "cs-01-3" ] ], [ "A proposed evaluation that could falsify the claim", "Start with a sequential specification of the application and collect invocation and response intervals, including timeouts as uncertain outcomes rather than automatic failures. Generate faults at persistence boundaries, not only by stopping idle processes. Force a leader to acknowledge a write, lose connectivity, and then receive a read while another group elects a leader. Repeat with membership transitions and delayed messages from earlier configurations.\nCorrectness and service quality should be reported on the same histories. The denominator for availability must include requests issued during recovery. Record the failure detector parameters, retry policy, log-flush policy, and topology. A history checker can reject a bad execution, but passing a finite collection of histories cannot establish universal correctness. A formal specification and a bounded model checker offer complementary evidence, provided their abstraction excludes no behavior relevant to the asserted property.", [] ], [ "Conclusion: make each guarantee carry its assumptions", "Consensus becomes easier to assess when every statement names its quantifiers. A safety claim constrains all permitted executions of a model; a progress claim names the timing or fairness conditions under which completion follows; an empirical availability claim refers to a workload and observation period. Confusing these forms is the source of many exaggerated reliability promises.\nOur quantitative examples show why more replicas can improve an independent-failure model while doing little against common causes, and why waiting for a quorum is only one component of end-to-end latency. The counterexample shows why intersection must carry persistent protocol state. A defensible deployment argument joins those elements without substituting any one for the others: the mathematics, implementation boundary, and measured environment must describe the same service.", [] ] ], "section_kinds": [ "source_based_review", "source_based_review", "author_analysis_or_verification_proposal", "author_analysis_or_verification_proposal", "author_analysis_or_verification_proposal", "author_analysis_or_verification_proposal", "source_based_review", "source_based_review", "author_analysis_or_verification_proposal", "author_analysis_or_verification_proposal" ], "equations": [ { "tex": "|Q_1\\cap Q_2|\\ge 2q-n", "explanation": "For two subsets of size q of the same n-member configuration. A positive lower bound ensures intersection, not agreement without additional protocol rules." }, { "tex": "A(n,q,p)=\\sum_{k=q}^{n}\\binom{n}{k}p^k(1-p)^{n-k}", "explanation": "Independent per-replica reachability p within a declared window; failures are crash or communication unavailability, not Byzantine behavior." }, { "tex": "A_c=(1-c)A(n,q,p)", "explanation": "A common event disables every replica with probability c; residual independent reachability is conditional on no common event." }, { "tex": "\\Pr[T_{(q)}\\le t]=A\\!\\left(n,q,1-e^{-t/\\tau}\\right)", "explanation": "Ideal quorum collector with iid exponential acknowledgement times of mean tau; the plots use tau = 10 ms." } ], "table": { "headers": [ "Claim", "Required evidence", "What it does not establish" ], "rows": [ [ "Intersecting quorums", "Subset argument plus durable response rules", "Progress under arbitrary delay" ], [ "Linearizable application", "Sequential specification and client histories", "Unlimited throughput" ], [ "Operational availability", "Failures, workload, window, and denominator", "A universal termination theorem" ] ] }, "protocol": [ "Specify the exact application state machine and completion boundary before generating load.", "Enumerate quorum intersections and document the persistent obligation at each shared replica.", "Inject crash and communication faults before and after durable acknowledgements.", "Include stale leaders, reconfiguration, retries, and reads in collected histories.", "Check histories against the sequential specification; retain timeouts as potentially completed operations.", "Report availability and latency including recovery intervals and common-cause failures." ], "limitations": [ "The binomial model assumes independent reachability except for one explicitly modeled common event.", "The latency model is a quorum-collector abstraction and excludes actual storage, network, and leader implementation costs.", "The executable subset enumeration does not verify a consensus implementation.", "Three selected sources do not cover every consensus family, Byzantine model, or recent optimization." ], "evidence": [ { "source": "cs-01-1", "result": "Deterministic asynchronous consensus can admit a nonterminating execution with one possible crash.", "boundary": "The theorem's model and universal termination requirement matter; it is not a bound on typical latency.", "locator": "Abstract; asynchronous model and impossibility proof." }, { "source": "cs-01-2", "result": "Raft separates replicated-log responsibilities and describes overlapping-majority configuration changes.", "boundary": "The paper does not establish correctness of arbitrary read paths or modified implementations.", "locator": "USENIX paper; leader election, log replication, safety and membership sections." }, { "source": "cs-01-3", "result": "Paxos uses numbered proposals and retained acceptor state to preserve chosen values.", "boundary": "Progress is a separate issue; acknowledgements without the stated rules do not suffice.", "locator": "Sections 2.1–2.5 and state-machine discussion." } ], "research_questions": [ "How strongly are failures correlated across the intended deployment's actual fault domains?", "What client-history behaviors occur during configuration changes and stale-leader reads?", "Which persistence and scheduling delays dominate observed quorum latency?" ], "kind": "ai_assisted_research_guide", "author": "Codex (AI)", "original_language": "en", "created_date": "2026-10-22", "published_date": "2026-10-22T13:02:56+00:00", "publication_status": "published_commissioned_review", "version": 5, "previous_version": "/guides/consensus-safety-progress/v4/", "analytic": "consensus-safety-progress", "peer_reviewed": false, "journal_publication": false, "disclosure": { "authorship": "Written and technically self-reviewed by Codex (AI). References identify the original researchers; the analytical constructions and proposed evaluation designs are attributed to this article.", "figures": "Covers are conceptual ImageGen illustrations. Technical figures are computed from declared models or finite examples, with Python and CSV supplied. They are not measurements of deployed products or reproduced performance benchmarks.", "review": "This local draft has no independent peer-review or journal-acceptance record. Bounded executable examples check the stated small models; they are not formal verification of the cited systems." }, "review_scope": "A focused analytical review of three selected foundational primary sources, with original worked examples. This is not an exhaustive or systematic survey of the literature through 2026. Source access and version notes are recorded with the bibliography.", "revision_note": "Initial local English manuscript; no server upload or publication action.", "source_slug": "consensus-safety-progress", "source_revision": "41dc5ea7bf1db9868e2ee5b121f7021c823cc018", "source_article_sha256": "e75c774b5784d9aab3c9770130e94eef9d8d516c70e7addac4a30e62d3e22886", "sources": [ { "id": "cs-01-1", "key": "FLP", "title": "Impossibility of distributed consensus with one faulty process", "authors": [ { "given": "Michael J.", "family": "Fischer" }, { "given": "Nancy A.", "family": "Lynch" }, { "given": "Michael S.", "family": "Paterson" } ], "journal": "Journal of the ACM", "year": 1985, "doi": "10.1145/3149.214121", "url": "https://groups.csail.mit.edu/tds/papers/Lynch/jacm85.pdf", "verified_date": "2026-10-22", "metadata_source": "Crossref deposit, title checked against primary record", "author_list_complete": true, "csl_type": "article-journal", "access_scope": "Primary record, abstract and selected accessible text; not an exhaustive full-text or supplementary-material audit.", "article_slug": "consensus-safety-progress", "source_article_slug": "consensus-safety-progress" }, { "id": "cs-01-2", "key": "Raft", "title": "In Search of an Understandable Consensus Algorithm", "authors": [ { "given": "Diego", "family": "Ongaro" }, { "given": "John", "family": "Ousterhout" } ], "journal": "USENIX ATC", "year": 2014, "doi": "", "url": "https://www.usenix.org/conference/atc14/technical-sessions/presentation/ongaro", "verified_date": "2026-10-22", "metadata_source": "primary publisher or author record", "author_list_complete": true, "csl_type": "paper-conference", "access_scope": "Primary record, abstract and selected accessible text; not an exhaustive full-text or supplementary-material audit.", "article_slug": "consensus-safety-progress", "source_article_slug": "consensus-safety-progress" }, { "id": "cs-01-3", "key": "Paxos", "title": "Paxos Made Simple", "authors": [ { "given": "Leslie", "family": "Lamport" } ], "journal": "ACM SIGACT News", "year": 2001, "doi": "", "url": "https://lamport.azurewebsites.net/pubs/paxos-simple.pdf", "verified_date": "2026-10-22", "metadata_source": "primary publisher or author record", "author_list_complete": true, "csl_type": "article-journal", "access_scope": "Primary record, abstract and selected accessible text; not an exhaustive full-text or supplementary-material audit.", "article_slug": "consensus-safety-progress", "source_article_slug": "consensus-safety-progress" } ], "original_source_url": "/guides/assets/v9-computer-science-articles-consensus-safety-progress-json-e75c774b5784d9aa.json", "pdf": { "document_id": "5fbb0f13-2c12-5c09-91ee-389e5709090f", "version_id": "374daea1-6d54-51af-84c5-b18a00957385", "media_type": "application/pdf", "authentication_required": true, "download_audited": true }, "presentation_change": "Complete version-bound translated reading views and selectable authenticated original/translated PDFs; original scientific text and evidence retained.", "source_locale": "en", "source_version_id": "0e8493a3-1bbf-551f-8138-0a26e03e037a", "translations": [ { "locale": "zh", "translation_id": "201e45de-db5a-58f5-aef5-37df317c8374", "translated_content_sha256": "e9148cc0b3d2ba8cc71abbed57c33208291b6e124e59d7493c77f2e61b1ab3a1", "sha256": "a14cfd491df4e432674a97870fbb345e730b513c85aa3206db77805732b9a425", "byte_length": 904175 }, { "locale": "es", "translation_id": "e6543dae-a805-500f-85b0-a40c78683f84", "translated_content_sha256": "ae721eefb99b953bc908c69f390f1b4bc692001ac2693864bd52f55a98fda36b", "sha256": "cb41f2f865d62d940ce923200214223c4ec87f9f4c997f4c7c4fd573cc6d7d70", "byte_length": 651568 }, { "locale": "ja", "translation_id": "760638d4-3428-5a9b-b550-59a4248ce293", "translated_content_sha256": "43b4c5de45ec339c523a74c547817d21b6648aa8cbb2b1c8c7c7b780cff77981", "sha256": "14ae65bc0618d1148251e19ed3a9efd9c7c75412a0e0f5d10b178b9adff3d81b", "byte_length": 847542 }, { "locale": "ar", "translation_id": "54550ec4-36ee-52fd-a1fc-a34448dda9d6", "translated_content_sha256": "a65122bdde8a9683b0a1e1b564a7a8a9243bf55b0cab75c3bf7cd158433bc694", "sha256": "62c8423b09bc8d108b67a1f8653120c70137e3ee8ebb69110db3cd5afc4ab9b0", "byte_length": 647385 } ] }