How long does a QLoRA fine-tune take?

The run in this guide — Llama 3.1 8B, 10k instruction pairs, 3 epochs — takes about 95 minutes on one RTX 4090. Cost at the interruptible rate: roughly $0.21.

Do I lose quality using QLoRA instead of full fine-tuning?

For instruction/style adaptation, usually a point or less on task metrics — the LoRA paper lineage and years of community results back this. Large domain shifts (new language, code dialects) are where full fine-tunes still earn their 10× cost.

Can I fine-tune with my data staying private?

Your dataset lives on your instance and volume, is deleted when you delete them, and payment is crypto with no KYC. Nothing about the run is shared — see the security page for the isolation model.

How to fine-tune Llama 3.1 8B with QLoRA on a single GPU

Setup: one instance, one volume

Everything durable goes on a volume; the GPU instance stays disposable:

infrastructure, 2 commands

powergpu volume create --name ft --size 20 --region eu-west-1
powergpu launch --gpu rtx-4090 --type interruptible \
  --template axolotl --volume ft:/ft --disk 40
# ✓ i-c41b9a02 running · $0.131/hr

Why interruptible: axolotl checkpoints to /ft, so an interruption costs one resume, not the run. Why 24 GB is enough: QLoRA holds the 8B base in 4-bit (~5.5 GB) + adapters + optimizer (~3 GB) + activations — peaks near 14 GB at batch 4, sequence 2048. A RTX 3090 at $0.052/hr does it too, ~35% slower.

The dataset format

data.jsonl

// /ft/data.jsonl — one instruction pair per line
{"instruction": "Summarize this ticket for an engineer.",
 "input": "Customer reports…",
 "output": "P1 — checkout API 500s when…"}

Quality beats volume decisively at this scale: 2,000 clean, consistent pairs outperform 50k scraped ones. Dedupe, strip formatting noise, keep outputs in exactly the voice you want back.

The axolotl config, annotated

qlora.yml — the whole config

# /ft/qlora.yml
base_model: meta-llama/Llama-3.1-8B-Instruct
load_in_4bit: true            # the Q in QLoRA
adapter: qlora
lora_r: 32                    # capacity of the adapter; 16-64 is the sane band
lora_alpha: 64                # 2x r is the boring, correct default
lora_target_linear: true      # all linear layers — better than picking modules

datasets:
  - path: /ft/data.jsonl
    type: alpaca
val_set_size: 0.05

sequence_len: 2048
micro_batch_size: 4
gradient_accumulation_steps: 4   # effective batch 16
num_epochs: 3
learning_rate: 2e-4
lr_scheduler: cosine
warmup_ratio: 0.03

gradient_checkpointing: true  # ~30% VRAM back for ~20% speed
flash_attention: true
bf16: true

output_dir: /ft/out
save_steps: 100               # ← interruption insurance
logging_steps: 10

Run and watch

the run

axolotl train /ft/qlora.yml
# step 10/561 · loss 1.842 · 14.1 GB VRAM · 3.4 it/s
# …
# step 561/561 · loss 0.914 · eval_loss 0.987 · 94 min

Read the curves like this: train loss should fall fast then flatten; eval loss following it down means learning, eval loss rising while train falls means memorising — stop at the divergence (or raise val_set_size and lower epochs). For a first run, 3 epochs on 10k pairs almost never overfits an 8B.

Merge, test, serve

from adapters to endpoint

# merge adapters into standalone weights (on /ft, survives the instance)
axolotl merge-lora /ft/qlora.yml --lora-model-dir /ft/out

# smoke-test locally
python -m vllm.entrypoints.openai.api_server \
  --model /ft/out/merged --max-model-len 4096 &
curl localhost:8000/v1/chat/completions -d '{…}'

# then destroy the trainer and serve properly
powergpu destroy i-c41b9a02 --yes
powergpu launch --gpu rtx-5090 --template vllm \
  --volume ft:/ft:ro --env MODEL=/ft/out/merged

Serving card choice is a different optimisation than training — the vLLM guide picks it properly.

The final bill

GPU, ~96 min interruptible	$0.21
Volume 20 GB, 2 days	$0.107
Total	≈ $0.32

A custom-behaviour 8B for the price of a sandwich. The same recipe scales: 14B on the same card, ~34B on a 32 GB card, 70B on an 80 GB card overnight — the VRAM guide has the ceilings.

Put the numbers to work

Every price in this guide is our live rate — fixed, ≥30% under the market median, billed per second. Deploy the exact setup above from the console in about 30 seconds, paid in crypto, no card and no KYC.