What will your AI feature actually cost?

Compare token costs across 16 current models, see when serving an open-weights model on your own GPUs beats paying per token, and size GPUs for serving or training. Every assumption is visible and every price is dated and sourced.

Prices as of —

Your workload

Start from a preset, then make it yours.

Requests per day

Input tokens

Output tokens

Per request. Output includes any reasoning tokens.

Input served from cache 60% A shared system prompt or document prefix that repeats across requests.

Sent through batch APIs 0% Work that can wait up to a day, at half price.

Monthly active users (optional)

Show providers

Monthly cost by model, cheapest first
Model	Tier	Per request	Per month	Per user / mo	Split

Fresh input Cached input Output

Serve it yourself

Open-weights model

Weights

KV cache

GPU

Price from

$ per GPU-hour

Sequences in flight per replica Bigger batches raise throughput until memory runs out.

Average utilisation 60% Traffic is uneven; few fleets average above 60–70%.

Ops & engineering overhead 0%

Compare with API model

Monthly cost as traffic grows

API Self-hosted (whole replicas, one always on) Break-even

Memory to serve a model

Weights plus the KV cache for every sequence in flight.

Model

Weights

KV cache

Context tokens

Sequences

GPU	Memory	GPUs needed	$/hour

Compute to train a model

The standard estimate: 6 × parameters × training tokens.

Parameters (B)

Active per token (B)

Training tokens (billions)

GPU

Cluster size

Model FLOPs utilisation 35% Meta's Llama 3.1 405B run works out at about 35% on H100s.

API token costs

For each model, a request costs:

fresh input × input price + cached input × cached price + output × output price

Batched requests use each provider's batch prices (half the standard rate). Cache writes are ignored: with a reused prefix they are a small one-off. Prompts above a model's long-context threshold switch to its long-context prices where the provider publishes them. A month is 30 days.

Self-hosting an open-weights model

Generating tokens is usually limited by memory bandwidth, not compute: every step reads the weights and every sequence's KV cache. So one decode step takes the longer of

(weights read + sequences × KV cache per sequence) ÷ (bandwidth × efficiency) 2 × active parameters × sequences ÷ (peak FLOPS × utilisation)

KV cache per token = 2 × layers × KV heads × head size × bytes, from each model's published config.
Mixture-of-experts models read only the experts a batch touches: a share of 1 − (1 − active/total)^batch of the weights.
Prompt processing (prefill) is compute-bound: 2 × active parameters FLOPs per prompt token, at 50% utilisation.
Bandwidth efficiency 70%, decode compute utilisation 40%, and 85% scaling per extra GPU in a replica.
A replica is the fewest of 1, 2, 4 or 8 GPUs that hold the weights plus 8 sequences of cache, using 90% of memory.
Cost per request = replica cost per second × seconds of replica time per request ÷ average utilisation.
Self-hosting is billed in whole replicas with at least one always on, so it only wins above a break-even volume.

These are first-order estimates. Real serving stacks (vLLM, TensorRT-LLM, SGLang) add continuous batching, speculative decoding and prefix caching, so treat the result as a range and measure your own workload before committing.

Memory and training

Serving memory = weights (parameters × bytes per parameter) + KV cache (per token × context × sequences).

Training compute uses the standard rule of thumb from scaling-law research:

FLOPs = 6 × active parameters × training tokens GPU-hours = FLOPs ÷ (peak BF16 FLOPS × MFU) ÷ 3600

Calibration: Llama 3.1 405B trained on 15.6T tokens. At 35% MFU on H100s this gives 30.4M GPU-hours; Meta reported 30.84M. Smaller models use GPUs less efficiently: Meta's figures for the 8B and 70B imply roughly 14% and 25%. Mixed-precision Adam needs about 16 bytes per parameter before activations, which sets the minimum number of GPUs even with full sharding.

Sources

GPU specs are dense (no sparsity) BF16 and FP8 tensor throughput and memory bandwidth from NVIDIA's datasheets. Prices change often; every number lives in js/data.js with its source, so updating them is a one-file change.