Use case · fine-tuning
Fine-tune LLMs on cloud GPUs: your data, an evening's budget
Adapter methods moved fine-tuning from cluster territory to single-card territory: QLoRA a 8B model for about $0.20 on an interruptible RTX 4090, or a 70B overnight on one 80 GB card. Per-second billing means the meter stops with the last step.
The fine-tuning ladder
| Tier | GPU | VRAM | On-demand | Interruptible | Why this card | Action |
|---|---|---|---|---|---|---|
| Good | RTX 5090 | 32 GB | $0.318 | $0.159 | 32 GB GDDR7 — QLoRA up to ~34B on one card, twice the memory bandwidth of the 4090. | Deploy |
| Better | A100 SXM4 | 80 GB | $0.583 | $0.291 | 80 GB HBM2e — the classic LoRA/full-FT node; 8× SXM4 scales without drama. | Deploy |
| Best | H100 PCIE | 80 GB | $2.147 | $1.073 | 80 GB + FP8 — full fine-tunes finish in roughly half the A100 wall-clock. | Deploy |
Budget corner: a 24 GB RTX 4090 at $0.131/hr interruptible handles 8B–13B QLoRA beautifully.
What fits where (QLoRA, 4-bit base)
| Base model | VRAM needed | Cheapest single card | ≈ per training hour |
|---|---|---|---|
| Llama 3.1 8B | ~11 GB | RTX 3090 24 GB | $0.052 |
| Qwen 2.5 14B | ~17 GB | RTX 4090 24 GB | $0.131 |
| Qwen 2.5 32B | ~26 GB | RTX 5090 32 GB | $0.159 |
| Llama 3.1 70B | ~48 GB | H100 PCIe 80 GB | $1.073 |
Interruptible rates shown — fine-tunes checkpoint, so pay half. Full tables in the VRAM guide.
A clean fine-tuning loop
-
Stage data on a volume
Dataset + output dir live on a $0.08/GB/mo volume, not the disposable instance.
-
One YAML, one command
axolotl template: axolotl train qlora.yml — checkpoints stream to the volume.
-
Merge, serve, destroy
Merge adapters, point a vLLM instance at the volume, destroy the trainer. Billing: zero.
Fine-tuning GPUs: FAQ
The full copy-paste run lives in the QLoRA walkthrough.
What does a typical fine-tune cost?
A QLoRA pass over 10k instruction pairs on Llama-3.1-8B takes ~1.5 h on one RTX 4090 — about $0.20 interruptible. A 70B QLoRA on one 80 GB card is an overnight run in the tens of dollars. Full-parameter 8B on 8× A100: low hundreds.
LoRA, QLoRA or full fine-tune?
QLoRA first: 4-bit base + trainable adapters fits 8B in under 12 GB and 70B in ~48 GB, and quality is usually within a point of full tuning for instruction tasks. Go full-parameter only when the domain shift is large — then rent A100/H100 nodes for hours, not weeks.
Which template should I start from?
axolotl — one YAML covers LoRA/QLoRA/full, DeepSpeed and FSDP, and the walkthrough guide is written against it. Kohya covers the image-LoRA side.