{ "schema": "auturnal.deep-research-review.v2", "slug": "cache-admission", "title": "Caching Beyond Recency: Admission, Workload Shift, and the Cost of a Miss", "category": "Data structures and storage", "dek": "Choosing what may enter a cache can matter as much as choosing what leaves it.", "abstract": "Cache performance depends on admission, replacement, workload history, and the cost of a miss. This review connects TinyLFU, ARC, and LeCaR to a constructed scan-pollution example and a simple popularity model. A toy frequency gate protects a small hot set from a stream of one-use objects, while an ordinary LRU cache evicts it. Analytical figures show the ideal hit mass of a fixed cache under Zipf-like demand and the effect of exponential aging on historical frequency. The analysis distinguishes these mechanisms from freshness, authorization, and consistency, which remain separate correctness obligations. The resulting evaluation framework measures useful work avoided under changing workloads rather than treating hit ratio as a universal objective.", "english": "Cache performance depends on admission, replacement, workload history, and the cost of a miss. This review connects TinyLFU, ARC, and LeCaR to a constructed scan-pollution example and a simple popularity model. A toy frequency gate protects a small hot set from a stream of one-use objects, while an ordinary LRU cache evicts it. Analytical figures show the ideal hit mass of a fixed cache under Zipf-like demand and the effect of exponential aging on historical frequency. The analysis distinguishes these mechanisms from freshness, authorization, and consistency, which remain separate correctness obligations. The resulting evaluation framework measures useful work avoided under changing workloads rather than treating hit ratio as a universal objective.", "refs": [ "cs-14-1", "cs-14-2", "cs-14-3" ], "sections": [ [ "A cache has two selection decisions", "Replacement chooses which resident item to evict when space is needed. Admission chooses whether a newly requested item should displace a resident at all. Treating every miss as a mandatory insertion gives a one-use scan the power to evict a valuable working set.\nConsider a cache holding four frequently reused documents. A request stream then visits twenty distinct documents once each before returning to the original four. LRU naturally favors the most recent scan items, so the hot set is gone. A frequency-based admission gate can decline the scan items and retain the hot set. This is not proof that frequency always wins: a genuine workload change may make the new items valuable. The design problem is to distinguish transient novelty from a new working set quickly enough.", [] ], [ "Three research approaches", "TinyLFU develops a compact frequency-based admission policy and combines it with replacement mechanisms. ARC adapts between recency and frequency using resident and history information. LeCaR uses a learning framework to adjust among fundamental replacement behaviors.\nThese designs address related but distinct decisions, and their published evaluations concern particular traces and cache sizes. We do not infer a universal winner from them. Instead, they motivate three questions for a cache study: what information determines admission, how quickly history ages, and how a policy adapts when locality changes. The toy implementation in this article is deliberately simpler than all three systems and is labeled as such.", [ "cs-14-1", "cs-14-2", "cs-14-3" ] ], [ "A deterministic scan-pollution witness", "The appendix warms a four-entry cache with five repetitions of keys zero through three. It then requests twenty fresh keys and finally revisits the four hot keys. Ordinary LRU has lost the hot set. A toy admission policy that compares exact observed candidate frequency with the least-recent resident's frequency declines the one-use scan objects and preserves the hot entries.\nThe witness establishes only that admission can change this trace's result. It is not an implementation of TinyLFU, ARC, or LeCaR, and it does not estimate their comparative performance. Reversing the workload to repeatedly request the formerly cold keys would expose the gate's adaptation problem. Including both patterns in a study prevents a policy from being judged solely on the trace designed to favor it.", [] ], [ "An ideal static-cache model", "Suppose requests are independent and identically distributed over N equal-sized items, each miss has the same cost, and item probabilities are known. A fixed cache with capacity C maximizes hit probability by storing the C most probable items. Its ideal hit mass is the sum of those probabilities.\nFor a Zipf-like distribution, probability is proportional to rank raised to negative alpha, with a finite normalization over N items. The first figure fixes N at one thousand and varies C for several alpha values. Stronger skew makes a small cache more valuable in this ideal model. This is an oracle static bound for the declared demand distribution, not the expected hit ratio of LRU or an adaptive policy. It omits learning, compulsory misses, updates, and changing popularity.", [] ], [ "Frequency needs a clock", "An exact lifetime frequency count can retain once-popular objects long after they become irrelevant. Aging changes the meaning from historically common to recently common. In a simple exponential-decay model, a count halves every H units of time or reference count, depending on the chosen clock.\nThe second figure plots a historical count of 64 under several half-lives. A shorter half-life forgets quickly but can overreact to noise; a longer half-life resists scans but adapts slowly to a real shift. Real policies may use periodic resets, approximate counters, or other mechanisms rather than continuous decay. The figure is a conceptual accounting model, not a faithful simulation of TinyLFU's implementation. It makes the adaptation parameter explicit so that its trade-off can be tested.", [] ], [ "Hit ratio is not always the objective", "If objects differ in size or miss cost, maximizing the number of hits can be economically wrong. A small inexpensive object and a large expensive computation do not contribute equally to saved work. An application may care about bytes transferred, backend CPU, energy, or deadline misses.\nA more general fixed-cache objective weights each item's request probability by the cost avoided and constrains total resident size. The resulting selection problem is not simply choosing the most frequently accessed objects. Metadata and lookup overhead also count: an elaborate policy can improve hit ratio while consuming more CPU than it saves. A defensible evaluation states the objective and reports the resource actually reduced, rather than assuming every hit has equal value.", [] ], [ "Freshness is a correctness contract", "A cached object can be popular and still be invalid. Updates, revocation, authorization changes, and versioned content require a rule determining whether a stored value may be reused. Admission and replacement policies decide which valid candidates occupy memory; they do not establish validity.\nThis distinction matters for scientific content and private workspaces. A cached public article may need a version check after correction, while a private draft requires authorization tied to the current request and identity. An invalidation mechanism must avoid returning stale or unauthorized data even if the cache policy would prefer to keep it. Measuring a higher hit ratio by suppressing required validation changes the contract and is not a legitimate performance improvement under the original requirement.", [] ], [ "Metadata can become the hidden working set", "Frequency sketches, ghost lists, timestamps, keys, and allocator overhead consume memory that could otherwise hold data. A fair comparison includes all of this state. Two policies with the same nominal object capacity may have different total memory footprints, especially for small objects.\nThe evaluation should also include synchronization costs in a concurrent cache. Updating a global recency list or frequency structure can create contention. Sharding can reduce that contention while changing replacement quality and fairness across partitions. A single-thread trace simulator cannot predict those costs. It remains useful for isolating policy behavior, provided the report separates that result from an implementation benchmark on a multi-core service.", [] ], [ "Workload order matters even with the same histogram", "Two traces can contain exactly the same number of requests for each key and produce different cache behavior because locality depends on order. A burst of repeated access differs from the same requests spread across a long scan. A benchmark that shuffles a production trace may preserve marginal popularity while destroying the property the policy is meant to exploit.\nA useful test matrix includes stationary demand, one-use scans, abrupt hot-set changes, gradual drift, and bursty reuse. It retains timestamps when policies use time-based expiration or aging. Training and tuning traces should be separated from evaluation traces. If a policy adapts online, report the transient period as well as the eventual steady state. Users experience the transition, not only the point after the policy has learned the new workload.", [] ], [ "A reproducible comparison protocol", "Begin with the deterministic witness and a negative control where the old hot set is no longer useful. Verify every policy against a simple reference implementation and account for warm-up explicitly. For trace studies, report the complete trace provenance, preprocessing, object sizes, miss costs, and total memory budget.\nThen evaluate an implementation under concurrent load with the same validity rules and backend behavior. Record backend requests avoided, CPU overhead, allocation pressure, throughput, and tail latency. No such production benchmark is conducted here. The local result is a controlled demonstration of admission and a pair of analytical models that a later implementation study can use as interpretable reference points.", [] ], [ "Conclusion: optimize saved work under a valid-data contract", "Caching is a system of decisions about validity, admission, replacement, and adaptation. A strong policy can improve one part without solving the others. The scan witness shows why admitting every miss can be wasteful; the popularity and aging models show why no single history horizon suits every workload.\nA rigorous cache study fixes the validity contract, includes policy metadata and concurrency costs, and measures the work that matters to the application. This framing makes improvements more transferable than a headline hit ratio and prevents a performance optimization from silently weakening freshness or access-control guarantees.", [] ] ], "section_kinds": [ "author_analysis_or_verification_proposal", "source_based_review", "author_analysis_or_verification_proposal", "author_analysis_or_verification_proposal", "author_analysis_or_verification_proposal", "author_analysis_or_verification_proposal", "author_analysis_or_verification_proposal", "author_analysis_or_verification_proposal", "author_analysis_or_verification_proposal", "author_analysis_or_verification_proposal", "author_analysis_or_verification_proposal" ], "equations": [ { "tex": "p_i=\\frac{i^{-\\alpha}}{\\sum_{j=1}^{N}j^{-\\alpha}}", "explanation": "Finite Zipf-like iid request model over N items; the plots use N = 1000." }, { "tex": "H^*(C)=\\sum_{i=1}^{C}p_i", "explanation": "Ideal hit probability of a fixed cache holding the C most probable equal-sized, equal-cost items; excludes warm-up and adaptation." }, { "tex": "f(t)=f_0\\,2^{-t/H}", "explanation": "Illustrative exponential history decay with half-life H and initial count f0 = 64." }, { "tex": "\\max_{S:\\sum_{i\\in S}s_i\\le M}\\ \\sum_{i\\in S}p_i c_i", "explanation": "Static expected-work objective with item size si, avoided miss cost ci, and memory M; not generally equivalent to maximizing hit count." } ], "table": { "headers": [ "Decision", "Controls", "Does not decide" ], "rows": [ [ "Admission", "Whether a miss displaces resident data", "Freshness or permission" ], [ "Replacement", "Which valid resident loses space", "Future popularity with certainty" ], [ "Aging/adaptation", "How quickly history loses influence", "A universally optimal response time" ], [ "Validity", "Whether reuse is allowed", "Which valid item saves the most work" ] ] }, "protocol": [ "Reproduce the hot-set/scan witness and an opposing hot-set-change control.", "Include all policy metadata in the memory budget.", "Preserve trace order, timestamps, object sizes, and miss costs.", "Separate tuning traces from evaluation traces and include adaptation intervals.", "Keep freshness and authorization behavior identical across variants.", "Measure backend work avoided and concurrent implementation cost, not only hit ratio." ], "limitations": [ "The toy admission gate is not TinyLFU, ARC, or LeCaR.", "The static popularity model assumes iid requests, equal sizes, equal miss costs, and known probabilities.", "The decay curve is an illustrative model rather than a claimed implementation detail.", "No production trace or concurrent cache implementation was benchmarked." ], "evidence": [ { "source": "cs-14-1", "result": "TinyLFU uses recent-frequency information to guide admission and can be combined with replacement policies.", "boundary": "Its reported benefits depend on workload and configuration.", "locator": "Abstract and admission-policy design." }, { "source": "cs-14-2", "result": "ARC adapts its balance between recency and frequency using history information.", "boundary": "Empirical breadth is not a theorem of universal optimality.", "locator": "USENIX abstract and adaptive policy description." }, { "source": "cs-14-3", "result": "LeCaR applies a learning framework to combine basic replacement behavior.", "boundary": "The evaluated traces and resource budget determine the reported comparison.", "locator": "USENIX HotStorage abstract." } ], "research_questions": [ "Which miss costs, object sizes, and validity checks determine the real value of a hit?", "How quickly must the policy adapt without becoming vulnerable to one-use scans?", "Does metadata and synchronization overhead erase the saved backend work?" ], "kind": "ai_assisted_research_guide", "author": "Codex (AI)", "original_language": "en", "created_date": "2026-10-22", "published_date": null, "publication_status": "local_draft_pending_owner_upload", "version": 1, "previous_version": null, "analytic": "cache-admission", "peer_reviewed": false, "journal_publication": false, "disclosure": { "authorship": "Written and technically self-reviewed by Codex (AI). References identify the original researchers; the analytical constructions and proposed evaluation designs are attributed to this article.", "figures": "Covers are conceptual ImageGen illustrations. Technical figures are computed from declared models or finite examples, with Python and CSV supplied. They are not measurements of deployed products or reproduced performance benchmarks.", "review": "This local draft has no independent peer-review or journal-acceptance record. Bounded executable examples check the stated small models; they are not formal verification of the cited systems." }, "review_scope": "A focused analytical review of three selected foundational primary sources, with original worked examples. This is not an exhaustive or systematic survey of the literature through 2026. Source access and version notes are recorded with the bibliography.", "revision_note": "Initial local English manuscript; no server upload or publication action." }