Cloud GPU pricing explained (2026): on-demand vs spot vs reserved

Who sells GPU time, and how

Three kinds of sellers set today's market:

• Hyperscalers (AWS, GCP, Azure) — list prices, enterprise contracts, and H100-class instances typically at $4–7 per GPU-hour before committed-use gymnastics.
• GPU marketplaces — thousands of independent hosts auctioning capacity. Deep supply and low medians, but the price is a moving target and the host quality is a distribution, not a promise.
• Fixed-price GPU clouds — the newer category we belong to: capacity bought in bulk from verified datacenters, resold at a published flat rate. Our rule is mechanical: marketplace median × 0.70, rounded down, re-checked weekly. Today that puts the H100 SXM at $1.587/hr against a $2.27 median.

What actually moves prices

GPU-hour prices are supply economics with a silicon accent:

• Generation launches — every Blackwell shipment pushes Hopper prices down a step; the 90-day drift on H100 medians is visible in any public feed.
• VRAM per card — memory sells the hour. 80 GB parts hold a price floor long after their FLOPS are matched by consumer cards, because model sizes grew faster than compute needs.
• Electricity and density — hosts with cheap power and dense racks undercut; that is why medians differ by region and why our fleet skews to power-cheap regions.
• Bursts of demand — a hot open-weights release can double marketplace spot prices for a week. Fixed pricing exists precisely to opt out of that volatility.

On-demand vs interruptible vs reserved

The industry uses "spot" for our interruptible tier, but classic spot is an auction: you bid, you win, a higher bid evicts you. A flat −50% with stop-not-destroy semantics behaves very differently in practice — the discount is predictable, so pipelines can be designed around it instead of around bid strategy.

The break-even math

Two formulas cover 90% of purchasing decisions:

reserved beats on-demand when:
    utilisation > res_rate / od_rate            (= 65% here)

interruptible beats on-demand when:
    (1 + overhead) x 0.50 < 1
    i.e. restart overhead under 100% of runtime — checkpointing
    every 15 min on a 6 h job is ~4% overhead, not 100%.

Concrete: a fine-tune that needs 200 GPU-hours of A100 SXM4 per month —

• on-demand: 200 × $0.583 = $117;
• interruptible with 5% restart overhead: 210 × $0.291 = $61 — the obvious winner;
• reserved only wins here at 475+ monthly hours (65% of 730).

Where bills quietly grow

• Egress — hyperscalers charge $0.05–0.12/GB out; moving a 2 TB dataset off can cost more than the training run. Flat $0.01/GB (ours) or free egress changes which workflows are even viable.
• Idle storage — forgotten stopped instances bill their disks forever. Our dashboard surfaces runway and stopped-disk burn; check whatever provider you use for the same view.
• Hourly rounding — per-hour billing turns a 61-minute job into 2 hours. Per-second billing is worth 0–49% on short jobs — the shorter the job, the bigger the gap.
• Multi-GPU premiums — some sellers price 8× machines above 8× the single price. Check the multiplication; here it is exactly linear.

A buyer's checklist

1. Compute $/hr per GPU, not per instance, and normalise VRAM (a RTX 5090 hour buys 32 GB; an 80 GB card should justify its multiple).
2. Ask what happens at eviction: auction re-price, destroy, or stop-with-disk?
3. Price the full loop: GPU + storage-month + egress of your artefacts.
4. Prefer sellers who publish their pricing rule — if the rule is secret, it can move against you.
5. Benchmark once: an hour of testing on a $0.131 interruptible RTX 4090 answers throughput questions no spec sheet can.