Author

aiterms

What is structured output (JSON mode)?

Structured output, often called JSON mode, is a way to ask an LLM to return data in a fixed machine-readable format instead of free-form text.

Why it matters

When you want an answer that another program can safely parse, plain chat text is fragile. A model might add extra words, change field names, or wrap the result in an explanation.

Structured output helps when you need:

• predictable API responses
• reliable extraction into downstream systems
• cleaner automation for forms, workflows, or tools
• outputs that match a schema, such as an object with required fields

In practice, teams reach for it when they want the model to behave more like a data generator than a conversational assistant.

How it works

There are two closely related ideas:

1. JSON mode: the model is constrained to emit valid JSON text.
2. Structured outputs / schema-constrained output: the model is constrained not just to JSON, but to JSON that matches a specified structure, such as required keys, types, and nesting.

Official implementations vary by provider, but the general mechanism is the same: you give the model instructions plus a format constraint, and the system either:

• guides generation so the output stays in the allowed shape, or
• validates the output against the schema and retries or rejects if it does not fit.

This is different from “please respond in JSON” in plain natural language. A prompt alone may help, but it does not guarantee correctness. Structured output features exist to make that guarantee much stronger.

A useful mental model: the model is still generating text, but the platform is steering or checking that text against a contract.

Tiny concrete example

Suppose you want to extract a support ticket summary:

Request

Return JSON with keys: priority (low, medium, high), category, and summary.

Response

{
  "priority": "high",
  "category": "billing",
  "summary": "Customer was charged twice for the same invoice."
}

That output is easy for your code to parse because the shape is predictable.

Common pitfalls / when NOT to use it

• Don’t use it when you want creative text. If the task is drafting, brainstorming, or open-ended conversation, strict structure can be unnecessary friction.
• Don’t assume JSON mode alone validates your business rules. It may ensure valid JSON, but not that values are semantically correct for your app.
• Be careful with overcomplicated schemas. Very deep or highly nested structures can make prompts harder to maintain.
• Don’t skip downstream validation. Even with structured output, production systems should still validate required fields and types.
• Know the provider’s exact meaning. “JSON mode” and “structured outputs” are not always identical across APIs; some enforce syntax only, others enforce a schema.

Common pitfalls / when NOT to use it.

• Do not use it for open-ended creative writing.
• Do not assume it replaces application-side validation.
• Do not overload the schema with unnecessary fields.

Related terms

• What is prompt engineering?
• What is a system prompt?
• What is few-shot prompting?
• What is zero-shot prompting?
• What is chain-of-thought prompting?
• What is context engineering?
• What is a context window?
• What is prompt caching?
• What is prompt injection?

同じ著者の記事

What is prompt caching?

Prompt caching is a way to reuse the cost and latency of processing the same prompt prefix instead of sending or recomputing it from scratch every time. If your app sends a long, repeated system prompt, policy block, tool schema, or document context on many requests, you can end up paying to re-process the same text over and over. Prompt caching helps reduce that waste. In practice, teams reach for it when: a large part of the prompt stays the same across requests, they want lower latency for re

papoo.work

What is a context window?

A context window is the amount of text a language model can read and keep in mind at once while it generates a response. The context window sets the practical limit for what you can ask a model to consider in a single turn: your prompt, system instructions, chat history, retrieved documents, and the model’s own recent output all compete for that space. In practice, it matters when you want the model to: follow long instructions, answer questions about a large document, maintain conversation stat

papoo.work

What is context engineering?

Context engineering is the practice of deliberately assembling the right information, instructions, tools, and examples into an AI model’s context so it can do a task reliably. Large language models do not “know” what you want unless you provide it in the prompt, surrounding messages, retrieved documents, tool outputs, or examples. Context engineering helps you get better accuracy, fewer hallucinations, and more consistent behavior by shaping what the model sees before it answers. In practice, i

papoo.work

What is chain-of-thought prompting?

Chain-of-thought prompting, often abbreviated CoT, is a way of asking a language model to produce intermediate reasoning steps before giving the final answer. It is used to improve performance on tasks that benefit from multi-step reasoning, such as arithmetic, logic, and some question-answering problems. In practice, teams reach for CoT when a direct one-line answer is more likely to be wrong than a stepwise solution. It can also make model behavior easier to inspect: if the model shows its

papoo.work

What is zero-shot prompting?

Zero-shot prompting is asking an AI model to do a task without giving it any worked examples first. It is the simplest way to use a large language model: you describe the task in plain language and let the model infer the pattern from its pretraining. That makes it useful when you want: a fast baseline before writing better prompts simple classification, extraction, or drafting tasks lower prompt length and less example curation a test of whether the model already “gets” the task well enough In

papoo.work

What is few-shot prompting?

Few-shot prompting, also called in-context learning, is a way of asking a language model to do a task by giving it a few example inputs and outputs in the prompt. Few-shot prompting helps when you want the model to follow a pattern, format, or classification rule without training or fine-tuning a new model. You’d reach for it when: the task is simple but the output needs to be consistent, you can show the model 1–5 good examples, you want to prototype quickly before investing in fine-tuning

papoo.work

What is a system prompt?

A system prompt is the instruction text that sets the assistant’s role, behavior, and boundaries before the user’s message is processed. System prompts help you control how an LLM behaves without changing the model itself. They’re useful when you want consistent tone, safety rules, formatting, or task-specific behavior across many requests. In practice, teams use system prompts to: keep outputs on-brand, enforce policies or guardrails, define what the assistant should and should not do, reduce a

papoo.work

What is prompt engineering?

Prompt engineering is the practice of designing and refining the text you give to a language model so it produces more useful, reliable, and specific outputs. Large language models are sensitive to how you ask. A vague request can produce a vague answer; a well-structured prompt can improve format, scope, tone, and task performance. You reach for prompt engineering when you want to: get a model to follow instructions more consistently, steer it toward a role, style, or output format, reduce back

papoo.work

What is contextual / late chunking?

Contextual chunking, also called late chunking, is a way to split a document into retrieval chunks after a model has already encoded the whole document, so each chunk keeps some awareness of the surrounding context. Normal chunking cuts text first and then embeds each piece alone. That is simple, but it can make chunks semantically thin: a paragraph may be ambiguous without the surrounding sections, headings, or earlier definitions. Late chunking helps when you want better retrieval quality

papoo.work

What is a knowledge base for RAG?

A knowledge base for RAG is the collection of documents or records a system retrieves from to ground an LLM’s answer in relevant source material. RAG, short for retrieval-augmented generation, works best when the model can search a useful, trustworthy set of content before it writes. That content is the knowledge base. You’d reach for one when the model needs to answer questions from: company docs support articles policies and handbooks product specs research notes tickets or tickets-like record

papoo.work