Fine-tuning, and when not to

Three ways to adapt an LLM

A simple rule: if the model does not know something, give it the information (RAG). If it knows but behaves wrongly (wrong tone, wrong format, unreliable on a narrow task), show it how (prompting first, fine-tuning if prompting is not enough).

Supervised fine-tuning (SFT)

You prepare pairs of input and ideal output, often hundreds to thousands, in a file such as JSONL (one JSON object per line). Training nudges the weights so the model produces outputs like yours. Examples: replying to customers in your brand's tone, converting free-text addresses into a fixed format, classifying medical codes the way your team does.

{"prompt": "Classify: parcel not delivered after 7 days", "completion": "delivery"}
{"prompt": "Classify: charged twice for order 5521", "completion": "billing"}

Parameter-efficient fine-tuning: LoRA

Updating all billions of weights needs huge GPUs. LoRA (low-rank adaptation) freezes the original weights and trains small extra matrices added to some layers, often well under 1% of the parameters. QLoRA also stores the frozen model in 4-bit form to save memory. This makes fine-tuning open models possible on a single good GPU, and the small adapter files are easy to store and swap.

Data quality beats data size

• Every example should be correct and in exactly the format you want; the model copies mistakes too.
• Cover the real variety: short and long inputs, Hindi and English, hard and edge cases.
• Remove duplicates and personal data you are not allowed to use.
• Keep a separate test set and compare the fine-tuned model with the base model plus a good prompt.

Distillation is a related idea: a large model's outputs are used to train a smaller, cheaper model for one task. Check the larger model's licence and terms before doing this.

Common mistakes

• Fine-tuning to add facts (like this year's policy). The model may still mix them up; use RAG.
• Fine-tuning before trying a good prompt with a few examples.
• No baseline comparison, so nobody knows whether fine-tuning helped.
• Training on messy or inconsistent examples and getting a messy model.