Author

aiterms

What is a base model vs instruct model?

A base model is a pretrained language model that predicts and generates text, while an instruct model is the same kind of model further tuned to follow human instructions more reliably.

Why it matters

This distinction helps you choose the right model for the job:

• Base models are useful when you want raw next-token generation, custom fine-tuning, or maximum flexibility for downstream tasks.
• Instruct models are useful when you want the model to answer questions, follow prompts, summarize, write code, or chat with less prompt engineering.

In practice, most product teams start with an instruct model unless they specifically need to adapt a base model themselves.

How it works

A base model is usually trained first on a large text corpus with a simple objective: predict the next token. That pretraining teaches broad language patterns, facts, and some reasoning behavior, but it does not by itself make the model especially good at obeying user instructions.

An instruct model starts from that base model and is then fine-tuned on instruction-following examples. This often includes supervised fine-tuning on prompt-response pairs, and sometimes additional alignment methods such as preference optimization or RLHF. The goal is to make the model more helpful, more direct, and less likely to ignore the user’s intent.

So the relationship is usually:

1. Pretrain a base model
2. Fine-tune it for instruction following
3. Deploy the tuned model as an instruct model

The exact training recipe varies by lab and model family, so “instruct model” is a practical label, not a single formal algorithm.

Tiny concrete example

If you ask:

“Write a polite email declining a meeting.”

• A base model might continue the text in many possible directions, including unrelated ones.
• An instruct model is more likely to produce a complete email that directly answers the request.

Example instruct-style output:

Hi Sam, thanks for the invitation. I’m not able to make the meeting, but I’d be happy to review the notes afterward.

Common pitfalls / when NOT to use it

• Don’t assume “base” means worse. Base models can be better for custom adaptation, controlled generation, or research workflows.
• Don’t assume “instruct” means factually correct. It follows instructions better, but it can still hallucinate.
• Don’t fine-tune an instruct model when you need a clean starting point. If you plan to do heavy domain adaptation, a base model may be the better foundation.
• Don’t use the terms interchangeably. “Chat,” “assistant,” and “instruct” are related but not identical; different vendors use them loosely.
• Don’t expect a base model to behave like a chatbot out of the box. It may need prompt formatting or additional tuning.

Related terms

Related terms

• What is fine-tuning?
• What is LoRA (low-rank adaptation)?
• What is RLHF?
• What is instruction tuning?
• What is distillation?
• What is quantization?

同じ著者の記事

What is speculative decoding?

Speculative decoding is a text-generation trick that lets a fast “draft” model propose several next tokens and then uses a larger “target” model to verify them, often reducing end-to-end latency without changing the final output distribution. Autoregressive LLMs normally generate one token at a time, and each token requires a full forward pass through the big model. That is accurate, but it can be slow. Speculative decoding matters when you want lower latency or higher throughput without switchi

papoo.work

What is tokens per second (throughput)?

Tokens per second, or throughput, is the rate at which a language model generates or processes tokens over time. Throughput is a practical way to describe model speed. If you care about user experience, cost efficiency, or server capacity, tokens per second tells you how much text the system can handle in a given time. You’d use it when comparing models, sizing infrastructure, or setting expectations for streaming responses. In practice, teams often care about both throughput and latency

papoo.work

What is time to first token (TTFT)?

Time to first token (TTFT) is the time between sending a generative AI request and receiving the model’s first output token. TTFT is the part of latency users feel first. If it is high, the system can seem “stuck” even if the full answer eventually streams quickly. You usually care about TTFT when you want: a responsive chat or copilot experience good perceived performance in streaming UIs to compare serving setups, models, or prompts to separate “slow to start” from “slow to finish” In practice

papoo.work

What is streaming (token streaming)?

Streaming, or token streaming, is when an AI model sends you its output incrementally as it is generated instead of waiting until the whole answer is finished. This solves the “dead air” problem: users see text quickly, which makes apps feel faster and more interactive. It is especially useful for chatbots, copilots, and any UI where a short time-to-first-token matters more than waiting for the full response. In practice, teams use streaming when they want: faster perceived latency, a typing-lik

papoo.work

What is batching / continuous batching?

Batching is the practice of grouping multiple requests together so a model can process them more efficiently, and continuous batching is the version where new requests are added to that group while others are still running. Large models are expensive to run because the same GPU is often underused if you serve one request at a time. Batching improves throughput by sharing compute across many requests. Continuous batching goes a step further: it keeps the GPU busy by letting the server dynamically

papoo.work

What is an inference endpoint / serving API?

An inference endpoint, or serving API, is a network service you send inputs to in order to get model predictions back in real time. It solves the problem of turning a trained model into something applications can actually use. You reach for an inference endpoint when you want: a web or mobile app to call a model on demand a backend to classify, rank, summarize, or generate text without running the model itself a standardized way to deploy, scale, monitor, and secure model requests In practice, m

papoo.work

What is quantization?

Quantization is the process of representing numbers with fewer bits, so a model or computation uses less memory and can often run faster. Large neural networks store weights, activations, and sometimes key/value caches as floating-point numbers. That is accurate, but expensive. Quantization reduces that cost by converting some of those values to lower-precision formats, such as 8-bit integers or even 4-bit representations. You usually reach for quantization when you want one or more of these: lo

papoo.work

What is distillation?

Distillation is a way to train a smaller model to imitate a larger, better one, so you keep much of the quality while reducing cost, latency, or memory use. Distillation solves a very practical problem: the model you want may be too slow, too expensive, or too large to deploy everywhere. You reach for distillation when you want one or more of these: lower inference cost faster responses smaller on-device models simpler deployment a model that approximates a stronger system without retraining fro

papoo.work

What is instruction tuning?

Instruction tuning is a way to train a language model to follow natural-language requests more reliably by fine-tuning it on examples of instructions paired with good responses. A base language model is good at predicting text, but that does not automatically make it good at doing what a user asks. Instruction tuning helps bridge that gap. You’d reach for it when you want a model to respond more helpfully to prompts like: “Summarize this email” “Write SQL for this schema” “Explain this code as i

papoo.work

What is RLHF?

RLHF, or reinforcement learning from human feedback, is a way to train a model so its outputs better match what people prefer, not just what is statistically likely. A plain language model learns to predict text. That is useful, but it does not automatically make the model helpful, honest, safe, or aligned with user intent. RLHF is used when you want to steer a model toward human preferences: better answers, fewer toxic outputs, more helpful refusals, and responses that fit product goals. In

papoo.work