Author

aiterms

What is RAG vs fine-tuning?

RAG (retrieval-augmented generation) and fine-tuning are two different ways to make a language model more useful: RAG gives the model external information at answer time, while fine-tuning changes the model itself using training data.

Why it matters

If your app needs accurate answers about fast-changing or private information, RAG is usually the first thing to try because it can use your documents without retraining the model. If you want the model to consistently follow a format, style, or task pattern, fine-tuning can be the better fit.

In practice, most teams use:

• RAG for knowledge lookup, citations, and freshness
• Fine-tuning for behavior shaping, output format, and domain-specific patterns
• Both when they need good behavior and access to internal knowledge

How it works

RAG

1. A user asks a question.
2. The system retrieves relevant passages from a search index, vector database, or document store.
3. Those passages are added to the model’s prompt.
4. The model answers using that retrieved context.

The key idea is that the model is not “learning” the documents permanently; it is being fed the right information for this request.

Fine-tuning

1. You prepare examples of desired inputs and outputs.
2. You train the model further on that dataset.
3. The model weights are updated so it tends to produce similar outputs in the future.

The key idea is that the model itself changes. That can improve consistency on a narrow task, but it does not automatically give the model access to a live knowledge base.

The practical tradeoff

• RAG is better when information changes often, must be sourced, or lives in many documents.
• Fine-tuning is better when you want the model to behave differently, not just know more.
• RAG can be updated by changing the index; fine-tuning usually requires another training run.

Tiny concrete example

Suppose a support assistant must answer policy questions.

• With RAG: the assistant retrieves the latest refund policy PDF and answers from it.
• With fine-tuning: the assistant learns to answer in a concise support style, but it still may not know the latest refund policy unless that policy was in training data.

Example:

User: “Can I refund an annual plan after 14 days?”
RAG system: retrieves the current policy text and answers, “No, annual plans are refundable only within 14 days.”
Fine-tuned system: may answer in the right tone and format, but could still be wrong if the policy changed after training.

Common pitfalls / when NOT to use it

• Do not use fine-tuning as a replacement for fresh knowledge. If the facts change, RAG is usually safer.
• Do not assume RAG fixes bad reasoning. Retrieval helps with context, but the model can still misunderstand or hallucinate.
• Do not fine-tune just to “add documents.” That is usually inefficient and harder to maintain than indexing the documents.
• Do not expect RAG to enforce style or tool-use behavior by itself. If you need a strict output format, fine-tuning or strong prompting may help more.
• Do not skip evaluation. Both approaches can fail in different ways, so test accuracy, latency, and cost on real examples.

In short: use RAG to give the model the right information; use fine-tuning to change how the model behaves.

Related terms

• What is retrieval-augmented generation (RAG)?
• What is a vector database?
• What is an embedding?
• What is semantic search?
• What is chunking (and semantic chunking)?
• What is hybrid search (BM25 + vector)?
• What is a reranker / cross-encoder?
• What is top-k retrieval?
• What is a knowledge base for RAG?
• What is contextual / late chunking?

同じ著者の記事

What is prompt engineering?

Prompt engineering is the practice of designing and refining the text you give to a language model so it produces more useful, reliable, and specific outputs. Large language models are sensitive to how you ask. A vague request can produce a vague answer; a well-structured prompt can improve format, scope, tone, and task performance. You reach for prompt engineering when you want to: get a model to follow instructions more consistently, steer it toward a role, style, or output format, reduce back

papoo.work

What is contextual / late chunking?

Contextual chunking, also called late chunking, is a way to split a document into retrieval chunks after a model has already encoded the whole document, so each chunk keeps some awareness of the surrounding context. Normal chunking cuts text first and then embeds each piece alone. That is simple, but it can make chunks semantically thin: a paragraph may be ambiguous without the surrounding sections, headings, or earlier definitions. Late chunking helps when you want better retrieval quality

papoo.work

What is a knowledge base for RAG?

A knowledge base for RAG is the collection of documents or records a system retrieves from to ground an LLM’s answer in relevant source material. RAG, short for retrieval-augmented generation, works best when the model can search a useful, trustworthy set of content before it writes. That content is the knowledge base. You’d reach for one when the model needs to answer questions from: company docs support articles policies and handbooks product specs research notes tickets or tickets-like record

papoo.work

What is top-k retrieval?

Top-k retrieval is a way to search a collection and return the k most relevant items for a query, instead of trying to sort or inspect everything. Top-k retrieval is the standard first step in search, recommendation, and retrieval-augmented generation (RAG). You use it when you want a fast shortlist: the most relevant documents, passages, products, images, or candidates, without paying the cost of full ranking over the entire corpus. In practice, teams reach for top-k retrieval when: the dat

papoo.work

What is a reranker / cross-encoder?

A reranker, often implemented as a cross-encoder, is a model that takes a query and a candidate result together and scores how well they match. A reranker solves the “good enough first pass, better final answer” problem. In search and retrieval systems, you usually start with a fast retriever that finds a few dozen or few hundred candidate documents. That first stage is optimized for speed, not perfect ranking. A reranker then looks at those candidates more carefully and reorders them so the mos

papoo.work

What is hybrid search (BM25 + vector)?

Hybrid search is a retrieval method that combines keyword search with vector similarity search so you can find documents that match both the exact words a user typed and the broader meaning of the query. Pure keyword search is good when the exact term matters. Pure vector search is good when wording varies but meaning is similar. In practice, many real queries need both. You reach for hybrid search when: users may search with exact product names, error codes, or technical terms the same

papoo.work

What is retrieval-augmented generation (RAG)?

Retrieval-augmented generation, or RAG, is a way to make a language model answer questions using relevant external documents it retrieves first, instead of relying only on what it memorized during training. RAG helps when you want answers that are: grounded in your own content, such as docs, policies, tickets, or product knowledge easier to update than retraining a model more likely to mention current or proprietary information In practice, teams reach for RAG when a model needs to answer “what

papoo.work

What is a vector database?

A vector database, also called a vector store, is a system for storing and searching data by its numerical embeddings so you can find items that are “most similar” rather than exactly matching text or IDs. Regular databases are great when you know the exact thing you want: a user ID, a product SKU, a timestamp range. But many AI features need semantic search or nearest-neighbor retrieval: “find questions like this one,” “retrieve passages relevant to this prompt,” or “recommend items

papoo.work

What is an embedding?

An embedding is a vector embedding: a list of numbers that represents something—like a word, sentence, image, or user—in a way a model can compare mathematically. Embeddings turn messy, human data into a form machines can use for: Similarity search: “Find documents like this one.” Recommendations: “Show items close to what the user liked.” Clustering and classification: “Group related things together.” Retrieval for LLMs: “Fetch the most relevant context before generating an

papoo.work

What is semantic search?

Semantic search is a way to find information by meaning, not just by matching the exact words you typed. Traditional keyword search is good when you know the exact term to look for. Semantic search helps when users phrase the same idea in different words, use synonyms, or ask a question instead of naming a document. You’d reach for it when: search queries are messy or conversational, exact wording varies a lot across documents, you want to retrieve “about the same thing” rather than “contains th

papoo.work