Retrieval-augmented generation (RAG) and fine-tuning solve different problems. RAG gives a model access to external knowledge at request time. Fine-tuning changes model behaviour by training on examples.
Use RAG when knowledge changes
RAG is usually appropriate for policies, product documentation, contracts, internal knowledge and other sources that evolve. It also supports source attribution and access-control patterns more naturally than embedding knowledge into model weights.
Use fine-tuning when behaviour is the problem
Fine-tuning can help with consistent format, classification behaviour, domain-specific style or repeated task patterns when prompting alone is insufficient. It is not automatically the best way to “teach the model your documents”.
Evaluate before choosing
Compare alternatives using a representative test set. Measure accuracy, factuality, latency, cost, maintainability and governance. In many enterprise systems, the right solution is a combination: retrieval for current knowledge plus prompting or tuning for behaviour.
Do not ignore operational cost
The best architecture is the one that meets quality requirements at an acceptable total cost of ownership, including indexing, monitoring, evaluation, model usage and maintenance.