An interactive field guide to model size, tokens, memory, speed, reasoning effort, open weights, deployment, routing, and the physical cost of an AI answer.
A model is not the app, a parameter is not a fact, and “tokens per second” is not a universal speed limit. This interactive guide follows one prompt through the model layer so you can see what changes when a provider offers a lighter model, more reasoning effort, downloadable weights, or an on-device route.
Here is the mistake that makes most AI conversations unnecessarily confusing: we use the word model for almost everything between a text box and an answer.
ChatGPT and Claude are products. GPT-5.6 and Claude 5 are model families. A dated API identifier can point to a checkpoint. “High effort” is a serving instruction. Fable and Mythos can wrap the same underlying model in different access and safety policies. A router can send two apparently identical product requests down different paths.
Those distinctions are not trivia. They determine who controls the system, what it costs, where it can run, what an outsider can inspect, and whether comparing two environmental figures is even meaningful.
Answers: How many learned numerical values a disclosed checkpoint stores.
Does not answer: Intelligence, active compute, training quality, memory precision, or serving speed.
Answers: For a sparse model, the approximate subset routed into computation for one token.
Does not answer: Total weights that must be stored, moved, sharded, or licensed.
Answers: The advertised maximum sequence budget under specified conditions.
Does not answer: Reliable recall, usable document length, KV-cache footprint, or answer quality.
Answers: Queueing plus prompt ingestion and other work before streaming begins.
Does not answer: How quickly the rest of the answer arrives.
Answers: Decode speed under a stated tokenizer, hardware, load, precision, and serving path.
Does not answer: Prompt-processing speed, task completion, aggregate throughput, or cross-tokenizer reading speed.
Answers: How much work a whole serving system completes across concurrent users.
Does not answer: The streaming speed experienced by one user.
This page freezes public evidence through August 13, 2026. Model catalogs move fast, so every current claim carries a date and every absent disclosure stays absent. A model card that says “hardware unknown” is more useful than an attractive guess.
Large language models do not read and write words in the way people do. A tokenizer divides text into pieces, maps those pieces to numbers, and the model repeatedly estimates a distribution for what token should come next. The 2017 Transformer paper introduced the attention-centered architecture behind that loop; current systems add years of changes in data, scale, multimodality, sparse routing, training, tools, and serving.
Token boundaries depend on the tokenizer and language. A short English word may be one token; an unfamiliar name, emoji, line of code, or word in another language may split differently. That is why a million-token context window is not a million words, and why output tokens per second are not automatically comparable across model families.
One decoding loop
Move through a simplified generation cycle. This is an architectural teaching view—not a trace of hidden reasoning, a provider tokenizer, or one production request.
The model predicts the next token.
Illustrative token pieces
These hand-authored pieces explain the concept. They are not output from OpenAI, Anthropic, Google, Apple, or another production tokenizer.
Step 1 of 5
A tokenizer turns text into model-specific integer pieces. Another model can split the same characters differently.
Schematic activity · not neurons, parameters, or compute utilization
There are two distinct phases when a hosted language model answers. Prefill processes the input and constructs working state for the sequence. Decode produces output one token at a time. A long document can therefore make the first token slow even when the answer streams quickly once it begins.
The product may also retrieve documents, call a search service, inspect an image, run code, invoke a safety classifier, or ask another model for help. The clean mental model is not “prompt → model.” It is:
product experience = router + model or models + context + tools + policy systems + serving hardware
There is no standards body assigning every model to small, medium, or frontier. “Lightweight” usually means designed around a constraint: memory, power, latency, price, or a device’s thermal envelope. The result might be genuinely small, distilled from a larger teacher, quantized to fewer bits, sparsely activated, specialized for one task, or several of those at once.
“Frontier” is relative. It usually describes the most broadly capable systems near the current capability boundary, not a fixed parameter threshold. Even regulation uses its own definitions for its own purposes; a legal training-compute threshold is not a consumer buying guide.
A smaller system can outperform a larger one on the task it was trained for. OpenAI’s InstructGPT research found its 1.3-billion-parameter instruction-tuned model was preferred to 175-billion-parameter GPT-3 on the prompts studied. That did not make the smaller model universally more capable. It showed why parameter count is not a scoreboard.
The model-size decoder
Compare disclosed architectures. Total parameters affect stored weights; an active-parameter figure uses the publisher’s stated routing scope, which can be per token or per request. Neither number is a universal intelligence score.
Select a disclosed model
Alibaba / Qwen · mixture of experts
Weights to store
30B total
Displayed active figure
3B active
Read the publisher wording below for its routing scope; “active” is not universally a per-token measurement.
Precision disclosure
30B total, 3B activated per token.
Deployment boundary
Total weights still need storage even though per-token compute routes through a subset.
What still decides “lightweight”: modality, context, precision, memory, latency target, batching, software, device envelope, training, and the job. Equal active-parameter counts do not imply equal operations, memory traffic, quality, or latency. A quantized large model can fit locally; a small multimodal model can still be computationally demanding.
The scale lab keeps several meters separate:
In a dense model, almost all major weight blocks participate for each token. In a conventional token-routed mixture-of-experts model, a learned router chooses a subset of expert blocks for each token; other sparse designs can route at the prompt or request level. “Expert” is a mathematical name; it does not mean one little neurologist and one little tax lawyer live inside the network. Sparse activation can reduce arithmetic while the full collection of weights still has to live somewhere across memory and machines.
The field is wider than the three names in a typical consumer model picker. OpenAI, Anthropic, Google DeepMind, xAI, Meta, Apple, Amazon, Microsoft, Mistral, Alibaba’s Qwen team, DeepSeek, Moonshot AI, Cohere, and NVIDIA all publish or operate consequential model families. Their roles differ.
A company can research and train a model, publish its weights, operate an API, distribute it through a cloud marketplace, wrap it in an app, or license it to someone else. Those are separate relationships. Google can design TPUs, operate data centers, train Gemini, and ship the Gemini app. Apple can design models and silicon, run local inference, operate Private Cloud Compute software, and still use outside cloud and model technology for other routes. A startup can build a product around several outside models without training a frontier checkpoint.
Model makers
Search companies that design or train model families. A distributor, cloud, application, chip supplier, and model maker can be different organizations—or several roles inside one vertically integrated company.
Showing 14 of 14 reviewed makers.
United States
Proprietary general-purpose, reasoning, coding, image, and realtime families.
Named families
GPT-5.6 · Codex · GPT Image · realtime and speech models
Boundary: Current prices, context, and product modes are public; parameters, active parameters, and request-level hardware/location are not.
United States
Proprietary general-purpose and agentic Claude model classes with tiered capability and access.
Named families
Claude Haiku · Claude Sonnet · Claude Opus · Claude Fable / Mythos
Boundary: System cards and cloud portfolio are public; parameter counts and per-request deployment remain private.
United States / United Kingdom
Multimodal proprietary Gemini/media models plus downloadable Gemma weights.
Named families
Gemini · Gemma · Veo · Imagen · Lyria
Boundary: TPU portfolio and selected models are public; a consumer request is not mapped to a TPU generation or data center.
United States
Task-integrated on-device and Private Cloud Compute models optimized with Apple silicon.
Named families
Apple Foundation Models · Xcode predictive completion
Boundary: Apple publishes unusually detailed architecture for selected generations, but feature-to-model and request-to-site routing stay private.
United States
Open-weight Llama families alongside proprietary product, media, and ranking models.
Named families
Llama · Muse · SAM · generation and recommendation models
Boundary: Llama weights and architecture are downloadable under a custom license; this does not expose all training inputs or Meta product routing.
United States
Proprietary Grok language models plus separate Imagine image/video and Voice APIs, with vertically integrated Memphis compute.
Named families
Grok · Imagine · Voice
Boundary: Model catalog and campus portfolio are public; parameters and the cluster serving one request are not.
United States
Proprietary foundation and action models distributed through AWS.
Named families
Nova · Titan · Nova Act
Boundary: Bedrock exposes many outside models too; catalog presence does not make Amazon their maker.
United States
Small/open-weight Phi research and product models plus proprietary Microsoft models.
Named families
Phi · MAI
Boundary: Microsoft also distributes OpenAI and partner models; Foundry availability is not model authorship.
France
Proprietary services and permissively licensed downloadable dense/MoE families.
Named families
Mistral · Ministral · Magistral · Voxtral
Boundary: License and model cards vary by checkpoint; “Mistral model” is not one uniform openness status.
China
Broad multilingual dense and MoE family spanning local through cloud deployment.
Named families
Qwen
Boundary: Qwen3.8-27B uses Apache 2.0 while the 2.4T-A95B and Flash-Next checkpoints use custom Qwen licenses, so artifact and license status must be checked per checkpoint.
China
Reasoning and general-purpose sparse models with downloadable weights and API service.
Named families
DeepSeek V4 Pro · DeepSeek V4 Flash
Boundary: Weight licensing, hosted checkpoints, aliases, and API routing are separate; running the weights yourself does not reproduce the hosted service.
China
Large sparse agentic and multimodal models with app, API, and weight distribution.
Named families
Kimi K3
Boundary: The app may add tools, memory, and orchestration beyond the downloadable checkpoint.
Canada
Enterprise, multilingual, retrieval, and sovereign-deployment models.
Named families
Command · Aya · Embed · Rerank
Boundary: Open-weight releases coexist with proprietary services and product orchestration.
United States
Open model families co-designed with NVIDIA training, optimization, and serving software.
Named families
Nemotron · Cosmos · specialist safety and retrieval models
Boundary: NVIDIA can be model maker, chip designer, framework supplier, and distributor in different routes; those roles must remain separate.
The atlas shows public model makers without pretending their disclosure practices are equal. A service-only model can publish prices and context while withholding parameter count. A downloadable model can expose weights while withholding the training corpus. A company can disclose a cloud portfolio without revealing which provider served your request.
This distinction matters to reporters too. “Available through AWS” is not evidence that AWS trained the model. “Runs on NVIDIA” is not evidence for a specific GPU, site, batch, or user request. “Owned by Google” is not evidence that every Google AI feature calls Gemini.
The phrase open model is so overloaded that it is nearly useless without an artifact list.
The Open Source Initiative’s Open Source AI Definition requires the freedoms to use, study, modify, and share an AI system, along with access to parameters, code, and sufficiently detailed data information in the preferred form for modification. Downloadable weights alone do not establish all of that.
The openness ledger
Inspect an abbreviated artifact view and the license for a named release. “Open,” “open source,” “open model,” and “open-weight” are not interchangeable labels.
Model weights
publishedInference code
publishedTraining code
partialTraining-data information
partialEvaluations
publishedLicense
Llama 4 Community License
Restrictions and boundary
Custom acceptable-use and large-platform provisions; inspect the exact license.
Abbreviated artifact view: this matrix does not inventory tokenizers, data-processing code, intermediate checkpoints, documentation, or every downstream right and restriction. The Open Source Initiative definition asks whether people can use, study, modify, and share the system with model parameters, code, and sufficient data information. This lab records publisher artifacts; it is not individualized legal advice.
The x-ray asks what you can actually obtain:
Meta’s Llama weights, for example, are downloadable under a custom community license. Mistral 3 and Qwen3 publish model weights under Apache 2.0, while DeepSeek has released selected weights under MIT terms. Those facts are valuable, but a license on tensors does not magically reveal the full dataset or reproduce the training run. The interface therefore uses service-only, open-weight, and cautious artifact-by-artifact labels instead of one green “open source” badge or a legal compliance verdict.
The honest first answer is: more information is required.
A useful lower bound for raw weight storage is:
parameters × bits per parameter ÷ 8
At sixteen bits, eight billion parameters occupy about 14.9 GiB before runtime overhead. Seventy billion occupy about 130.4 GiB. At four bits, the same raw floors are roughly 3.7 GiB and 32.6 GiB. Those examples use binary gibibytes; the interactive lab displays decimal gigabytes, so its number is slightly larger for the same bytes. None of those numbers means “this model will run on this GPU.”
The fit calculator
Calculate a raw weight floor, then add a user-selected KV-cache assumption. The sum is a known arithmetic floor—not a complete memory estimate, supported deployment, speed forecast, or hardware recommendation.
Weights only
0.75GB
Assumed KV
2.15GB
Known arithmetic floor
2.9GB
Raw capacity marker · not operational
1× 24 advertised GB containers
0.75 GB weights + 2.15 GB assumed KV = 2.90 GB known arithmetic floor
Decimal GB means 10⁹ bytes here, not GiB (2³⁰ bytes). The marker only divides a known floor by advertised capacity; it is not a feasible hardware count.
The raw weight floor already includes learned embedding weights when the published parameter total includes them. Real inference also needs quantization metadata, activations, temporary workspaces, serving software, and the KV cache: stored attention state for prior tokens. KV memory grows with context length, batch size, layer count, key/value heads, head dimension, and cache precision. Different architectures compress or share it differently.
That is why the lab reports raw capacity floors, not a definitive GPU count. It lets you place those floors next to advertised memory capacities while leaving operational fit unresolved. The resulting container count only divides a known arithmetic floor by nominal capacity; it is not a feasible hardware count, and no universal overhead percentage covers every architecture and serving stack. A model that technically loads can still be unusably slow, starve its KV cache, exceed a device’s power budget, or need an interconnect to span several accelerators.
CPUs remain excellent at flexible control flow and orchestration. GPUs dominate large-model training and much cloud inference because they combine massive parallel matrix throughput with high memory bandwidth. NPUs trade generality for power-conscious acceleration of supported neural operations, making them attractive inside phones and laptops. Most real systems use all three, plus storage, RAM, and network fabric.
When someone says a model runs at 300 tokens per second, ask five questions: for whom, on what hardware, at what concurrency, with which tokenizer, and after how much waiting?
The latency dashboard
Change four teaching assumptions. The arithmetic separates first-token delay, decode speed, end-to-end time, and aggregate load; it does not predict any named model or provider.
TTFT
1.2 s
Decode phase
6.4 s
Arithmetic total
7.6 s
Hypothetical aggregate
25 tok/s
Simple product, not measured throughput
Illustrative stream
A useful answer arrives after prompt ingestion, queueing, scheduling, and repeated decoding.
Schematic, capped visual word stream: these chunks are not real tokenizer output and their playback does not simulate the selected TTFT or TPS.
Not a benchmark: decode time is max(output tokens − 1, 0) ÷ TPS because TTFT already includes the first output token. The concurrency result is only a hypothetical multiplication, not measured aggregate throughput. Real concurrency changes queueing, batching, cache use, per-user speed, and hardware utilization. Cross-model TPS also depends on tokenizer, prompt/output length, precision, runtime, speculative decoding, network, and measurement percentile.
The lab separates the metrics that NVIDIA’s serving documentation also treats separately:
Under the common convention that TTFT ends when the first output token arrives, a response containing O output tokens takes approximately TTFT + (O - 1) × ITL. Counting O complete inter-token intervals adds one interval that did not occur.
Batching can improve fleet throughput while making one person wait longer. A very fast decoder can still lose a task by producing a needlessly long answer or requiring a retry. A speed number from one accelerator, quantization, prompt distribution, batch policy, and percentile cannot be transferred to a different deployment.
That is also why OpenAI’s “up to 750 tokens per second” preview for GPT-5.6 Sol must stay attached to its limited Cerebras serving route, and Google’s reported Flash-Lite speed must retain the benchmark source and conditions. They are evidence about particular offerings—not intrinsic physical constants inside the model.
Provider naming combines several independent ideas: capability class, generation, modality, access policy, product plan, and sometimes a routing mode. The decoder lets you pull those labels apart.
Name versus identity
Choose a maker and inspect dated records. Product menus compress several technical and commercial dimensions into short names.
Fable and Mythos are not two capability rungs. Anthropic says Fable 5 and Mythos 5 are configurations of the same underlying model; safeguards and trusted-access rules distinguish them.
fast / economical
High-volume, latency-sensitive work and subagents.
Access: Claude products, API, AWS, Google Cloud, and Microsoft Foundry; surface availability varies.
$1/M input · $5/M output
Identity boundary: A model class and pinned checkpoint, not a guarantee of hardware, location, energy, or water use.
Released or disclosed Oct 15, 2025
balanced frontier
Fast frontier model for scaled agentic work.
Access: Claude products, API, AWS, Google Cloud, and Microsoft Foundry.
$2/M input · $10/M output
Identity boundary: Current standard API price at the review cutoff; batch, cache, long-context, fast-mode, and cloud-route charges can differ.
Released or disclosed Jun 30, 2026
premium frontier
Complex agentic coding, enterprise work, and long-horizon reasoning.
Access: Claude products, API, AWS, Google Cloud, and Microsoft Foundry; API-only fast mode is a separately priced research preview.
$5/M input · $25/M output
Identity boundary: Opus is a recurring capability class; claude-opus-5 is a pinned model ID, while low through max effort changes inference-time work.
Released or disclosed Jul 24, 2026
safeguarded Mythos-class
Legacy Mythos-class model superseded by Fable 5.1 on September 1, 2026 but still available.
Access: Listed as a still-available legacy model on the Claude API, AWS, Google Cloud, and Microsoft Foundry after the Fable 5.1 launch; product-plan access varies.
$10/M input · $50/M output
Identity boundary: Same underlying model as Mythos 5; classifiers and fallbacks mean the selected Fable label need not prove which model handled every request.
Released or disclosed Jun 9, 2026
trusted-access Mythos-class
Defensive cybersecurity and selected scientific work with some safeguards lifted; succeeded by Mythos 5.1 on September 1, 2026.
Access: Invitation-only Project Glasswing for approved customers—not a normal public tier.
$10/M input · $50/M output
Identity boundary: Same underlying model as Fable 5, with different safeguards and access—not a universally stronger set of weights.
Released or disclosed Jun 9, 2026
safeguarded Mythos-class
Long-running agentic coding, knowledge work, and multistep research; successor to Fable 5.
Access: Claude API, Amazon Bedrock, Claude Platform on AWS, Google Cloud, and Microsoft Foundry; product-plan access varies. Retirement not sooner than September 1, 2027.
$10/M input · $50/M output
Identity boundary: Same underlying model as Mythos 5.1 with cybersecurity and biology safeguards; the $0.25/M cache-read price and always-on adaptive thinking are pricing and runtime facts, not evidence of a different set of weights.
Released or disclosed Sep 1, 2026
trusted-access Mythos-class
Successor to Mythos 5 for Project Glasswing participants in cyberdefense and life-sciences verification programs.
Access: Project Glasswing participants only, through Anthropic, AWS, or Google Cloud account teams—not a public tier.
$10/M input · $50/M output
Identity boundary: Same underlying model as Fable 5.1 with different safeguards and trusted-access rules—not a separately trained stronger checkpoint.
Released or disclosed Sep 1, 2026
Anthropic is the clearest 2026 case. Haiku, Sonnet, and Opus are recurring capability and price classes. As of this page’s cutoff, Anthropic’s public comparison spans Haiku 4.5, Sonnet 5, Opus 5, and Fable 5. Fable and Mythos are not two ordinary rungs above Opus. Anthropic says Fable 5 and restricted Mythos 5 use the same underlying Mythos-class model, capabilities, specifications, and pricing. The difference is the surrounding safeguards and who may access the less-restricted configuration.
Fable’s classifiers can decline or reroute some sensitive-domain requests to Opus. Mythos is limited to vetted cyber-infrastructure and selected scientific programs. That makes this a model-policy lesson: changing the safety configuration and access contract can create a different service even when the underlying model is the same. The public description does not establish that every surrounding classifier or policy component lives inside identical weight files.
OpenAI’s Sol, Terra, and Luna are separate GPT-5.6 models aimed at frontier, balanced, and high-volume work. Selecting “high” effort on Sol does not turn it into another checkpoint. Google’s Flash-Lite and Flash names describe service classes, but they do not disclose parameter sizes or guarantee a particular TPU. xAI’s Grok product includes distinct language, image/video, and voice services rather than one endpoint that does everything.
Even a released checkpoint is not an infinite utility. Capacity has to be reserved, regions and cloud marketplaces have to deploy it, safety and legal reviews differ by surface, and product plans ration expensive serving routes.
The availability gate
Capacity, safety, product packaging, tool fit, lifecycle, and governance create different catalogs. Select a gate to see what it controls—and what it does not prove.
Selected gate
A premium model can consume more expensive serving capacity, longer reasoning time, or scarce launch supply, so subscriptions and APIs meter it differently.
Dated example
Fable 5 moved between included access and usage credits while Anthropic expanded capacity.
What this does not prove: that the public explanation reveals every internal capacity pool, fallback, classifier, region, checkpoint, or commercial decision.
20
available
1
limited
2
restricted
0
planned
0
historical
Availability can depend on:
Fable 5 is a useful warning against treating a launch post as permanent state. Anthropic suspended and later redeployed capacity under changing access and credit rules. A page that does not preserve those dates turns an operational history into a contradiction.
Sometimes the answer is quite specific. Apple says its roughly three-billion-parameter 2025 on-device foundation model used aggressive quantization and memory techniques. Apple’s 2026 AFM3 family separates a dense 3B on-device Core model, a sparse 20B Core Advanced model that activates 1–4B parameters depending on the request, and cloud models for harder work. Its Private Cloud Compute architecture is a privacy and service system, not merely the phone model moved into a server.
Where models run
Filter disclosed device, cluster, portfolio, and customer-selected paths. A named cloud or campus still does not identify the server, accelerator, grid, or cooling system used by one answer.
provider portfolio · training and inference
Operators
Microsoft Azure · Oracle / Stargate · AWS · CoreWeave
Hardware boundary
Mixed and not request-disclosed
Individual request
Not publicly allocated
Azure is primary while OpenAI documents additional contracted capacity across other providers.
A partnership or campus announcement cannot locate a GPT-5.6 request or identify its chip.
provider portfolio · training and inference
Operators
AWS · Google Cloud · Microsoft Azure · xAI / SpaceX capacity
Hardware boundary
AWS Trainium, Google TPU, NVIDIA GPU, and undisclosed mixes
Individual request
Not publicly allocated
AWS is the primary partner; Anthropic publicly documents material capacity with several operators.
No source maps Haiku, Sonnet, Opus, Fable, or Mythos request shares to a cloud, region, campus, or accelerator.
provider portfolio · training and inference
Operators
Google · Google Cloud
Hardware boundary
Google TPU families plus other fleet hardware
Individual request
Not publicly allocated
Google says TPUs train and serve Gemini across its infrastructure.
The exact TPU generation, zone, and energy/water context for one request are not exposed.
device · inference
Operators
Device owner · Apple software stack
Hardware boundary
Eligible Apple silicon
Individual request
Not publicly allocated
Selected Apple Foundation Models execute on the phone, tablet, or computer.
Feature selection, model generation, quantization, battery energy, charging grid, and device thermals vary.
provider portfolio · inference
Operators
Apple · Google Cloud extension
Hardware boundary
Apple silicon servers; NVIDIA GPUs disclosed for the Cloud Pro extension
Individual request
Not publicly allocated
Larger Apple model requests can use Apple-operated PCC, with a 2026 Google Cloud extension for Cloud Pro.
Apple does not publish request shares, regions, campuses, or universal feature routing.
named cluster · training and inference
Operators
xAI
Hardware boundary
NVIDIA GPU generations disclosed at portfolio level
Individual request
Not publicly allocated
xAI identifies Memphis data centers with Grok training and serving.
Named cluster is still not a per-request server, rack, GPU, electricity, or water trace.
customer selected · inference
Operators
User, enterprise, cloud, device, or inference host
Hardware boundary
Selected by the deployer
Individual request
Not publicly allocated
Downloadable weights can be quantized, fine-tuned, and served in many environments.
The model maker may not know where a downloaded derivative runs; the app must disclose its own deployment.
For hosted frontier models, public evidence is usually a portfolio rather than a GPS trace. Anthropic names AWS as its primary cloud and also documents Google TPU, Microsoft/NVIDIA, and xAI capacity relationships. OpenAI names Azure as primary alongside Oracle/Stargate, AWS, and CoreWeave capacity. Google describes Gemini running across its own TPU infrastructure. None of those facts locates an individual prompt.
The map therefore does not drop a fake pin on “the Claude data center.” It shows the disclosed level of precision: device, provider portfolio, distributed cluster, or unknown. A cloud contract is not a workload allocation; a campus announcement is not proof that the model you selected ran there.
Apple Intelligence can run on-device because the local models and features are designed around Apple’s hardware, memory, power, and product constraints. Harder or broader tasks can cross into Private Cloud Compute or a separately disclosed external service. The fact that one local feature works without a warehouse of GPUs does not imply that the same device can train—or even serve—the largest general-purpose model.
It does not turn on more neurons. It is a provider-specific control over how much inference-time deliberation, checking, tool behavior, or hidden reasoning work the service may spend. The exact implementation is usually private and can change by model version.
Checkpoint versus effort
Pair an illustrative task with an effort setting. The bars show relative teaching levels only; they do not predict hidden tokens, quality, latency, price, or energy for a provider.
Illustrative inference-time budget
Medium
Routine work with several constraints or modest tool use.
Model checkpoint
Defines the released model identity, weights, architecture, and capability envelope—even when much of that remains undisclosed.
Effort or thinking setting
Lets an eligible checkpoint spend a different inference-time budget or use a different runtime policy. Vendor implementations differ.
For “Summarize several documents”: The suggested medium starting point is a teaching heuristic. Escalate after a failed evaluation, not because the longest setting feels safest.
Higher effort can be useful when a task genuinely benefits from decomposition, verification, or search. It can be wasteful on extraction, formatting, classification, or an email you will immediately edit. It usually raises latency and cost; it can also overthink an easy request. Where mistakes are expensive, independent evidence, domain evaluation, and accountable human review matter more than simply moving an effort slider upward.
The labels are not portable. OpenAI’s low, medium, and high do not establish the same budget as xAI’s similarly named settings or Anthropic’s adaptive thinking. Output verbosity is separate again. A concise answer can involve substantial internal work, while a long answer can come from a low-effort route.
The right question is not “Which setting is smartest?” It is “What is the cheapest, fastest route that clears the quality and risk threshold for this task?”
Even if one route used less arithmetic under tightly matched conditions, that would not establish a measured water ratio. No public dataset lets us hold task, success rate, tokens, effort, cache, batching, hardware, location, weather, cooling system, and accounting boundary constant between those two services.
Energy and water
Use the evidence checklist for the same-task Haiku-versus-GPT-5.6 Sol question. Checking every field designs a study; it does not manufacture a result.
Haiku versus GPT-5.6 Sol
5 required boundaries are still missing. Public model names and API prices cannot fill them.
company estimated
0.24 Wh · 0.03 gCO₂e · 0.26 mL onsite freshwater
Scope: Point-in-time May 2025 median under Google’s stated serving boundary and 2024 fleet-average factors.
Excludes: Training, external networks, end-user devices, data storage, request-level location, indirect electricity-generation water, and embodied hardware water.
executive claim
0.34 Wh · 0.000085 gal water (≈0.32 mL)
Scope: Brief executive statement without a published methodology.
Excludes: Model mix, reasoning effort, token counts, hardware, location, cooling boundary, training, and independent audit.
national model
66B L direct · nearly 800B L indirect grid water
Scope: National estimate for all data centers, not AI alone.
Excludes: Facility PPAs and behind-the-meter generation in the indirect model; model/provider/request allocation.
A fleet prompt estimate, an executive average, and a national data-center model have incompatible scopes. Do not rank models by placing these three numbers in one chart.
The evidence cards deliberately refuse to calculate the comparison. Google reported that a median Gemini Apps text prompt in May 2025 used 0.24 Wh and 0.26 mL of onsite freshwater under its stated fleet boundary. Sam Altman separately wrote that an average ChatGPT query used 0.34 Wh and about 0.32 mL of water, but supplied no comparable methodology. Those values differ in model mix, time, boundary, and transparency. They are not a Gemini-versus-ChatGPT benchmark.
The Lawrence Berkeley National Laboratory’s U.S. data-center report estimated 66 billion liters of direct water consumption and nearly 800 billion liters of indirect power-sector water consumption for all U.S. data centers in 2023. Dividing those totals by a guessed number of AI prompts would manufacture precision.
For one completed task, impact can change with retries, hidden reasoning, output length, router fan-out, tool calls, cache hits, utilization, accelerator efficiency, facility PUE, cooling WUE, and the grid. On-device inference may avoid data-center cooling for that request, but it still uses device electricity and hardware with a manufacturing footprint.
The impact lab is therefore a study-boundary checklist, not a model leaderboard or calculator. It shows what a matched comparison would have to measure and keeps fleet estimates, executive claims, and national data-center totals visibly separate; checking every box still does not produce an energy or water result.
Using the most expensive model for every task is not the same as using AI responsibly. It can increase delay, cost, and resource use without improving the result.
Minimum sufficient model
Choose four requirements. The result recommends a model class and escalation policy—not a brand, benchmark winner, or guarantee of quality, cost, privacy, or environmental impact.
Suggested starting class
This is a reasonable default when privacy, modality, and latency do not force a narrower choice.
Orthogonal oversight requirement
Name when a person must verify, approve, override, or stop the workflow. A model recommendation never assigns accountability.
Escalation rule
Escalate for a documented failure mode; downgrade after the task becomes repeatable.
This teaching router contains no live prices, private prompts, automated provider calls, or claim that a smaller model always uses less energy.
The teaching router starts with the task rather than a brand:
This is a teaching heuristic, not a reconstruction of any provider’s proprietary router. Its real lesson is that good routing requires evaluation. Teams should test representative tasks, define the failure cost, record latency and price, and promote a request only when the smaller route misses the target.
The most revealing fields are often the empty ones. Which checkpoint does a consumer alias actually reference? What fraction of requests hits each router branch? Which support models and tools fan out behind one answer? What precision and hardware serve each tier? How often does a safety classifier substitute another model? Which regions host the service, and what are their measured PUE and WUE?
Reporter mode
Model names, prices, defaults, access rules, and serving routes change quickly. This frozen edition preserves the disclosure date and leaves parameters, hardware, locations, and request allocation unknown when providers do not publish them. Research frozen Sep 4, 2026.
12 reporting leads
For each product mode, what exact checkpoint or router served the request, and when did that mapping change?
What share of requests are automatically escalated, downgraded, refused, or routed to a fallback model?
Which plan, geography, cloud, safety class, capacity pool, and feature gates each model?
Are “fast,” “balanced,” and “frontier” separate checkpoints, quantizations, serving tiers, or inference-time settings?
For sparse models, what are total parameters, active parameters per token, expert count, routing policy, and deployed precision?
What are p50 and p99 TTFT, inter-token latency, output TPS per user, total system TPS, input/output lengths, concurrency, hardware, runtime, and date?
How much memory goes to weights, KV cache, activations/workspace, runtime, batching, and fault-tolerance reserve?
Which training-data information, code, checkpoints, optimizer state, evaluations, and license terms are actually available?
Which named clouds, clusters, accelerator families, and regions train and serve each model—and what allocation remains undisclosed?
What is the measured energy and water boundary for a model class: prompt, completed task, tool-using workflow, tenant, region, or fleet?
When a safeguard triggers, is the request refused, rewritten, monitored, or served by another model—and is the user told?
How often would a right-sized smaller model complete the same evaluated task with acceptable quality, latency, and expert oversight?
68 dated sources
The downloadable ledger behind this page keeps those unknowns intact. They are reporting leads, not defects to paper over. Product makers could make the model layer far more legible by publishing alias histories, routing shares, per-tier latency distributions, hardware and region ranges, matched energy and water boundaries, and licenses expressed as artifact matrices rather than one “open” adjective.
If you retain six ideas, keep these:
The model layer feels intangible because its useful objects are software and numbers. But every choice in it changes the infrastructure below: memory moved, accelerators occupied, networks crossed, heat rejected, and capacity reserved. Understanding those choices is how the AMICE stack stops being a five-layer diagram and becomes a system you can interrogate.