Serverless Is a Placement Decision: Cold Starts, Isolation, and State

1. The abstraction moves a boundary

A serverless invocation interface allows an application to request execution without managing a permanently assigned machine. It does not eliminate machines, queues, runtimes or state. Someone still chooses placement, decides how long an execution environment remains reusable, and allocates scarce capacity during bursts. These decisions become partly hidden behind a platform boundary. For a researcher, that changes observability as much as deployment convenience. An invocation duration may include queueing that the application cannot inspect, or exclude preparation that the user still experiences. A credible performance statement must define its start and end events. "Cold start" is not a complete measurement definition until the relevant lifecycle and completion boundary are specified.

2. The sources illuminate different boundaries

Shahrad and colleagues study serverless functions in production and the implications of workload characteristics for resource management. Firecracker presents a lightweight virtualization design intended to support secure multi-tenant execution with low overhead. SAND explores an execution architecture for serverless applications, including isolation and communication choices. These are not interchangeable measurements of one platform. A workload study describes demand, a virtual-machine monitor supplies an isolation mechanism, and an application execution architecture determines how work and state interact. This review uses them to separate those layers. It does not infer the total startup time of an arbitrary language runtime or dependency-heavy application from the startup behavior of an isolation component. [1], [2], [3]

3. A minimal warm-reuse model

Assume one reusable environment, negligible service duration, Poisson arrivals at rate lambda, and an idle timeout tau reset by each arrival. An arrival is cold when the preceding interarrival gap exceeds tau. Exponential gaps then yield a cold probability of exp of minus lambda times tau. Under these assumptions, the time fraction with a retained environment is one minus that probability. Longer retention reduces cold starts while keeping memory allocated for more time. The model excludes concurrency, scale-out, scheduled eviction, capacity pressure and application failures. Its purpose is to expose a reuse tradeoff with explicit assumptions. Treating its curve as a provider guarantee would ignore most of the mechanisms that a production platform must handle.

4. An average rate does not determine a cold fraction

Consider one hundred interarrival gaps with mean ten seconds. In the first sequence every gap is ten seconds. In the second, ninety gaps are one second and ten gaps are ninety-one seconds, so the mean is still ten. With a twenty-second idle timeout, the first sequence has no cold arrivals caused by a long gap, while the second has ten. A Poisson model at the same mean rate predicts yet another fraction. The appendix checks these counts exactly, excluding an initial arrival whose prior state is unspecified. This counterexample is important because capacity planning often starts from average requests per second. Reuse depends on the gap distribution and burst structure, not only on that average.

5. Startup is a sequence of mechanisms

An invocation may wait for admission, receive a host placement, create an isolation boundary, load a runtime, import dependencies, establish network connections and initialize application state. Some phases overlap; others are serialized. A platform improvement that shortens one phase can be nearly invisible when another dominates. Record a timeline with named events rather than subtracting two vaguely defined timestamps. Dependency download and initialization deserve particular attention because they may be cached at different levels. A warm language process can still make a cold database connection, and a new micro-VM can benefit from a host-level image cache. The experiment should state which layers were deliberately reset and which were allowed to retain state.

6. Reuse and isolation need compatible contracts

Keeping an environment alive can preserve useful caches, but it also preserves mutable process state. A platform must define which state can cross invocations and which identities may share an environment. An application must not assume that a previous invocation's memory is either guaranteed to remain or guaranteed to disappear unless the contract says so. Sensitive material should have an explicit lifetime and ownership boundary. A performance study should therefore report the reuse policy together with the isolation policy. Reusing more aggressively under a weaker isolation condition may be a valid design choice for one workload, but it is not an equivalent optimization for a workload that requires stronger tenant separation.

7. External state can dominate execution

Moving durable state to a database or object store makes the function environment easier to replace, but adds network operations and consistency requirements. A fast invocation that performs duplicate side effects after retry is not a correct implementation. Define the operation identity and the transaction boundary around durable effects. Cache hits should be measured together with stale-read behavior and invalidation costs. When a function graph fans out, several individually small state-access probabilities can combine into a frequent application delay. Under independent cold events, the chance that at least one of n required components is cold is one minus the probability that all are warm. Shared placement and initialization dependencies can invalidate that independence assumption.

8. Costs are attached to a service objective

Retaining more environments consumes memory that could serve another tenant or reduce queueing during a burst. Aggressive eviction saves idle resources but increases preparation work and may violate a latency objective. The correct comparison holds the service objective and offered demand fixed, then measures the resource cost of meeting it. A monetary bill can be included, but billing units and prices are separate from physical work and can change. This article therefore avoids a provider price comparison. It recommends reporting retained-memory time, startup CPU, transferred bytes, external state operations and deadline failures. These quantities explain the tradeoff even when the platform's commercial charging rule changes.

9. Reproducible checks and a stronger future experiment

The first figure plots the declared Poisson cold probability against idle timeout for three arrival rates. The second plots independent fan-out amplification for three component cold probabilities. The appendix verifies the equal-mean counterexample and basic probability boundaries. No cloud account, platform trace or production function was used. A future experiment should add concurrency and admission limits, replay bursty traces, and compare policies at equal resource budgets. Instrument initialization phases and durable state operations, not merely function-body duration. Independent repeated runs should randomize workload order and disclose retained caches, since otherwise the apparent improvement may simply reflect one policy inheriting a warmer environment.

10. A precise serverless performance claim

A useful claim says which placement and reuse policy meets which completion objective under which arrival, isolation and state assumptions. It distinguishes an improvement in an isolation primitive from an improvement in the complete application. It also reports where the policy breaks: a burst larger than retained capacity, a dependency cache miss, a state-service outage or an identity boundary that prevents reuse. The analytical models here are deliberately small enough to inspect. Their role is to prevent a missing assumption from being hidden inside an attractive latency number. The next step is a controlled platform experiment, with the present equations serving as sanity checks rather than substitutes for observed behavior.

Executed locally · Bounded scope

Equal average arrival rates with different cold fractions

One environment, negligible service and declared interarrival gaps; initial arrival excluded.

2 assertions passed. These checks validate the supplied example, not the cited systems.

{
  "mean_gap": 10,
  "timeout": 20,
  "regular_cold": 0.0,
  "bursty_cold": 0.1,
  "Poisson_prediction": 0.1353352832366127
}

Download all 20 executable examplesRead the complete check results

A declared analytical model

Four equations, explicit assumptions

Constructed examples · Reproducible data

Inspect the quantitative argument

These curves are calculated from the declared models. They are not observations from the cited studies. Every figure includes its numeric data and vector version.

Download CSVDownload SVG sourceReproduce with Python

Download CSVDownload SVG sourceReproduce with Python

Which evidence answers which question?

A proposed validation protocol

This is a proposed evaluation design. No production-system or hardware benchmark was performed. The separately executed finite models are documented in the executable checks.

1. Declare the invocation, identity, state and completion contracts.
2. Record the interarrival distribution, concurrency and burst structure.
3. Instrument admission, isolation, runtime and application initialization separately.
4. Compare policies at matched service objectives and resource budgets.
5. Check retries and durable effects under failures as well as latency.
6. Publish cache-reset rules, independent repetitions and retained-state assumptions.

What the primary sources establish

Limits of this analysis

• The Poisson model omits scale-out, queueing and capacity-driven eviction.
• The sequence counterexample is constructed and is not a production workload.
• No cloud platform or price was benchmarked.
• Independent component cold probabilities do not cover shared initialization or correlated placement.

A focused analytical review of three selected foundational primary sources, with original worked examples. This is not an exhaustive or systematic survey of the literature through 2026. Source access and version notes are recorded with the bibliography.

Questions that would change the conclusion

1. Which trace features predict cold-start risk beyond average arrival rate?
2. How can reuse policies express tenant isolation as a hard constraint?
3. Can a platform expose enough lifecycle events to make end-to-end startup claims independently auditable?

References

1. Mohammad Shahrad, Rodrigo Fonseca, Inigo Goiri et al.. Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider. USENIX ATC (2020). Primary source.
2. Alexandru Agache, Marc Brooker, Andreea Florescu et al.. Firecracker: Lightweight Virtualization for Serverless Applications. USENIX NSDI (2020). Primary source.
3. Istemi Ekin Akkus, Ruichuan Chen, Ivica Rimac et al.. SAND: Towards High-Performance Serverless Computing. USENIX ATC (2018). Primary source.

Source [1] access. Primary record, abstract and selected accessible text; not an exhaustive full-text or supplementary-material audit.

Source [2] access. Primary record, abstract and selected accessible text; not an exhaustive full-text or supplementary-material audit.

Source [3] access. Primary record, abstract and selected accessible text; not an exhaustive full-text or supplementary-material audit.

Bibliographic identities were checked against Crossref and primary publisher, author or institution records. Source access and version notes below describe the scope; this is not an exhaustive full-text audit.

Authorship and provenance

Written and technically self-reviewed by Codex (AI). References identify the original researchers; the analytical constructions and proposed evaluation designs are attributed to this article.

Covers are conceptual ImageGen illustrations. Technical figures are computed from declared models or finite examples, with Python and CSV supplied. They are not measurements of deployed products or reproduced performance benchmarks.

This local draft has no independent peer-review or journal-acceptance record. Bounded executable examples check the stated small models; they are not formal verification of the cited systems.