Use case · LLM training
LLM training on cloud GPUs: the same silicon, no auction
Pre-training and continued pre-training live on HBM parts: H100 SXM at $1.587/GPU-hr fixed (market median $2.27), NVLink in-node, InfiniBand across nodes, and interruptible capacity at half price for every checkpointed epoch.
The training cards, ranked
Full comparison in the H100 vs H200 vs B200 guide.
| Tier | GPU | VRAM | On-demand | Interruptible | Why this card | Action |
|---|---|---|---|---|---|---|
| Good | A100 SXM4 | 80 GB | $0.583 | $0.291 | Proven 80 GB HBM2e workhorse — unbeatable $/step for ≤13B experiments and LoRA-heavy labs. | Deploy |
| Better | H100 SXM | 80 GB | $1.587 | $0.793 | The default: FP8 transformer engine, 3.35 TB/s, deep supply in every major region. | Deploy |
| Best | B200 | 192 GB | $4.204 | $2.102 | 192 GB HBM3e and Blackwell FP4/FP8 — shortest wall-clock when the deadline is the budget. | Deploy |
Cost of a real run, line by line
Continued pre-training, 8× H100 SXM, 72 hours, interruptible with volume checkpoints:
| GPU time | 8 × 72 h × $0.793 | = | $456.77 |
|---|---|---|---|
| Checkpoint volume 500 GB | 3 days × $40.00/mo | = | $4.00 |
| Dataset ingress 400 GB | 400 × $0.01 | = | $4.00 |
| Total | ≈ | $465 |
The market-median equivalent of the GPU line alone: about $653 at spot auctions you have to babysit — or $1,306 on-demand.
Patterns that make it cheap
- Checkpoint to a volume every N steps — interruptions restart free, from disk.
- Tune on 1×, train on 8× — debug the config at one-eighth the burn rate.
- Pack with FP8/BF16 — the H100/H200 transformer engine is the discount nobody uses.
- Destroy, keep the volume — datasets stay warm at $0.08/GB/mo, GPUs bill zero.
Templates: PyTorch, axolotl (FSDP-ready) — or your own image with a full VM for exotic stacks.
LLM training GPUs: FAQ
Sizing math: VRAM requirements guide.
Which GPU should I train an LLM on?
For serious runs: H100 SXM ($1.587/hr) is the price-performance default, H200 adds 141 GB for longer context and bigger micro-batches, B200 leads when wall-clock time is the constraint. For sub-13B experiments, 8× A100 nodes at $0.583/GPU-hr are hard to beat.
On-demand or interruptible for training?
Interruptible, almost always — training checkpoints anyway, and −50% compounds over hundreds of GPU-hours. A 72-hour run on 8× H100 costs about $457 interruptible vs $914 on-demand. Keep on-demand for the final, deadline-bound run.
How do multi-node runs connect?
Single machines scale to 8× with NVLink. Past that, clusters connect 8-GPU nodes over InfiniBand with NCCL pre-tuned — see the clusters page for pod pricing and all-reduce figures.