1.5 Customization Spectrum: Prompting vs RAG vs Fine-Tuning vs Continued Pre-training
Key Takeaways
- Use RAG for current traceable facts and weight customization for repeatable behavior or domain adaptation.
- Validate provenance, privacy, schema, holdout quality, and regression risk before starting a customization job.
- Do not apply the obsolete rule that every custom Bedrock model requires Provisioned Throughput.
1.5 Customization Spectrum: Prompting vs RAG vs Fine-Tuning vs Continued Pre-training
Generative AI teams can change application behavior at several layers. Prompting changes runtime instructions. Retrieval-augmented generation (RAG) supplies current external evidence. Fine-tuning adapts model behavior from examples. Continued pre-training or parameter-efficient techniques can adapt a model more deeply to a domain. The correct choice follows from the type of gap, not from the assumption that more customization is always better.
Choose the least invasive effective technique
| Technique | Best fit | Data shape | Main operational concern |
|---|---|---|---|
| Prompt engineering | Role, format, tone, reasoning scaffold, and a few demonstrations | Instructions and examples in the request | Token cost, prompt injection, regression control |
| RAG | Current or private facts that need traceable sources | Documents plus metadata in a retrieval system | Retrieval quality, authorization, freshness, citations |
| Supervised fine-tuning | Repeated task behavior, terminology, style, or output patterns | Curated input-output examples | Dataset quality, overfitting, evaluation, deployment support |
| Continued pre-training or PEFT | Broad domain adaptation where the base model lacks domain language or patterns | Large domain corpus or adapter-training data | Compute cost, catastrophic forgetting, lifecycle complexity |
Prompting is the fastest starting point because it does not alter weights. Use clear system instructions, structured delimiters, examples, and output schemas. Prompting is weak for large changing knowledge sets because every fact must fit in context and remain current.
RAG also leaves weights unchanged. The application retrieves authorized passages at request time and gives them to the model as evidence. RAG is the normal choice for policies, product data, legal material, and other changing facts. It can produce citations, but retrieval does not guarantee truth: weak chunking, missing metadata filters, or an unfaithful generator can still fail. Evaluate retrieval and generation separately.
Supervised fine-tuning learns from labeled examples. It is useful when a base model repeatedly violates an output pattern or lacks a specialized task behavior despite good prompts. Fine-tuning is not a database update mechanism and should not be the default way to memorize changing facts. Continued pre-training and parameter-efficient adaptation address deeper domain patterns, but require stronger data governance and regression testing.
Prepare and validate customization data
Training data is executable product behavior. Establish provenance, licensing, consent, retention, and access controls before a job begins. Remove duplicates, secrets, personal data that is not required, poisoned examples, and conflicting labels. Split training, validation, and final test data so the final set was not used to tune prompts or hyperparameters.
Validate the exact schema required by the selected model and customization method. Requirements vary; do not assume one universal JSONL prompt-completion format. Check file encoding, record limits, supported Regions, base-model eligibility, and service-role permissions. Record the source snapshot and preprocessing code so the model can be reproduced.
Evaluation should compare the customized candidate with the base model and with a prompt- or RAG-based alternative. Include task success, structured-output validity, hallucination rate, safety, latency, and total cost. Test general capabilities that might regress, not only the narrow training task.
Decision scenarios
Use prompting when the model already understands the task and needs clearer instructions or a few demonstrations. Use RAG when an answer depends on current private facts. Use supervised fine-tuning when hundreds or thousands of reviewed examples define a stable behavior that prompting cannot achieve economically. Consider deeper adaptation only when the base model systematically lacks domain language or patterns and the expected value justifies the added governance.
Combinations are common. A fine-tuned model can still use RAG for current facts. A RAG system still needs prompt engineering. A specialized classifier can route to a general model. Evaluate the simplest alternative beside the customized candidate so the team can quantify whether changing weights adds enough value.
Dataset splits must prevent leakage. Near-duplicate documents, templated customer messages, or multiple versions of the same case can make a held-out score look better than real generalization. Deduplicate before splitting, keep time-based or entity-based boundaries where appropriate, and reserve incident cases for regression testing.
Anti-pattern check
Do not customize solely to shorten a prompt until measured token savings exceed training, evaluation, and hosting costs. Do not use fine-tuning to bypass authorization or safety. Do not train on model outputs without quality review and provenance. Each of these choices can make a demo appear simpler while transferring risk into opaque weights.
A policy assistant must answer from regulations that change every day and show source evidence. Which technique best addresses the changing facts?