Structured Output: JSON Schema, Templates and Variables, Multilingual Output

Today's Goals

1. Explain why structured output must rely on a schema rather than a line saying "please output JSON"
2. Write a prompt as a template with variables, and explain how to split variables from constants
3. Write a script that extracts structured fields from natural language, and handle parse failures

For two days the ticket has been written for a human reader: the model outputs prose, you glance at it, and if you like it you use it. Starting today the reader changes — it is a program. A program does not glance. It wants fixed fields in fixed places. Today is about turning the model's output from a paragraph into a table, and about what to do when the table comes back filled in wrong. When you have read the walkthrough and finished the lab, come back to the top of the page and check off the three goals.

Plain-Language Walkthrough

Why a program cannot read the model's answer

The colleague reports the week's API errors in a sentence — "a few hundred, mostly 400s" — versus you handing them a three-column form (status code / count / percentage) to fill in. The sentence is perfectly comprehensible to a human, and it stalls the moment you try to put it in a spreadsheet. How many is a few hundred? How many is "a couple of 500s"? What share is "mostly"? You have to go back and ask another round. Hand them the three-column form up front and you type the result straight in, with no follow-up at all.

The model's free-text output is that sentence. It might say "312 errors total, of which 400 accounts for 289," or "errors are concentrated on 400 (289 occurrences)," or it might helpfully append "you may want to check your parameter validation." All three mean the same thing, and for your program to dig the number 289 out of them you have to write a pile of regular expressions that stop working the next time the model phrases it differently. That is the fundamental tension between free text and a fixed shape: text has endless ways to say a thing, and a program accepts exactly one.

Most people's first instinct is to append "please output in JSON format" to the prompt. That line helps, and it is not enough. The model will output JSON, and it may also wrap that JSON in explanatory prose, prepend a line saying "Sure, here are the results:", write numbers as strings, spell the failureStatus you asked for as failure_status, add a field you never requested, or drop the key entirely when the array should have been empty. Every one of those makes the code after JSON.parse reach into nothing. "Please output JSON" constrains it to being JSON; it does not constrain which JSON.

A JSON schema is the form you hand the model

The fix is to actually draw the form and hand it over: what the fields are called, what type each one is, which are mandatory, and which values an enumerated field may take. The standard way of writing that form is a JSON schema. It was designed for validating data, and both major APIs now let you hand it straight to the model so the output is constrained into that shape as it is generated, rather than generated first and prayed over afterward.

Take the task that runs through the course. We want to pull an endpoint, an HTTP method, per-field validation rules, a failure status code, and a one-line summary out of a requirement description. The schema looks like this:

{
  "type": "object",
  "additionalProperties": false,
  "required": ["endpoint", "method", "validations", "failureStatus", "summary"],
  "properties": {
    "endpoint": { "type": "string" },
    "method": { "type": "string", "enum": ["GET", "POST", "PATCH", "PUT", "DELETE"] },
    "validations": {
      "type": "array",
      "items": {
        "type": "object",
        "additionalProperties": false,
        "required": ["field", "type", "required", "rule"],
        "properties": {
          "field": { "type": "string" },
          "type": { "type": "string", "enum": ["string", "number", "boolean", "date"] },
          "required": { "type": "boolean" },
          "rule": { "type": "string" }
        }
      }
    },
    "failureStatus": { "type": "integer" },
    "summary": { "type": "string" }
  }
}

Go through what it constrains, one clause at a time. Everything listed in required must be present, so the model cannot omit validations just because the array came out empty. The enum pins method to five values, so it will not emit post or Post. type: integer guarantees failureStatus is a number rather than the string "400". And additionalProperties: false forbids fields you did not ask for. Each clause corresponds to one of the ways the previous section's code reached into nothing.

Two points get overlooked constantly. First, in strict mode every field of every object must appear in required, and every object must carry additionalProperties: false. Both APIs require this, and omitting it earns you a 400 with an error message that is not much fun to read. The way to express an optional field is to allow null in its type, not to remove it from required. Second, description inside a schema is not a comment — it is instructions the model reads. A description like "rule: one sentence describing the validation rule, for example length at most 200" measurably raises the odds that the model fills the field in correctly. Write those descriptions as an extension of the format box on your ticket.

The structured-output switch on each API

The same schema gets wired in differently on each API, and each way carries one limit worth knowing.

On the Anthropic side, the most reliable approach is to force a tool call: hand the schema over as a tool's input definition, then use tool_choice to require that the model call that tool. The model's act of "calling" it is filling in arguments that conform to the schema, and what you pull out of the response is that object, with no free text mixed in anywhere. On the OpenAI side you use the Responses API's text.format, with the type set to json_schema and strict turned on; the model's text output is then constrained to the schema, and what you receive is a string guaranteed to survive JSON.parse.

// One function per provider, each returning an unvalidated arbitrary object.
// Validation belongs to the next section.
async function extractWithAnthropic(system: string, requirement: string): Promise<unknown> {
  const res = await fetch('https://api.anthropic.com/v1/messages', {
    method: 'POST',
    headers: {
      'x-api-key': process.env.ANTHROPIC_API_KEY ?? '',
      'anthropic-version': '2023-06-01',
      'content-type': 'application/json',
    },
    body: JSON.stringify({
      model: 'claude-sonnet-5',
      max_tokens: 1024,
      system,
      messages: [{ role: 'user', content: requirement }],
      tools: [
        {
          name: 'record_spec',
          description: 'Record the extracted endpoint validation spec',
          input_schema: SPEC_SCHEMA,
        },
      ],
      tool_choice: { type: 'tool', name: 'record_spec' }, // forcing the tool means no free text
    }),
  })
  const json = (await res.json()) as { content: Array<{ type: string; input?: unknown }> }
  return json.content.find((c) => c.type === 'tool_use')?.input
}
 
async function extractWithOpenAI(system: string, requirement: string): Promise<unknown> {
  const res = await fetch('https://api.openai.com/v1/responses', {
    method: 'POST',
    headers: { Authorization: `Bearer ${process.env.OPENAI_API_KEY}`, 'content-type': 'application/json' },
    body: JSON.stringify({
      model: process.env.OPENAI_MODEL, // do not hard-code a model; follow the current docs
      instructions: system,
      input: requirement,
      text: { format: { type: 'json_schema', name: 'spec', schema: SPEC_SCHEMA, strict: true } },
    }),
  })
  const json = (await res.json()) as { output: Array<{ content?: Array<{ type: string; text?: string }> }> }
  const text = json.output.flatMap((o) => o.content ?? []).find((c) => c.type === 'output_text')?.text
  return JSON.parse(text ?? 'null') as unknown
}

import json
import os
 
import requests
 
 
# One function per provider, each returning an unvalidated arbitrary object.
# Validation belongs to the next section.
def extract_with_anthropic(system: str, requirement: str):
    res = requests.post(
        "https://api.anthropic.com/v1/messages",
        headers={
            "x-api-key": os.environ["ANTHROPIC_API_KEY"],
            "anthropic-version": "2023-06-01",
            "content-type": "application/json",
        },
        json={
            "model": "claude-sonnet-5",
            "max_tokens": 1024,
            "system": system,
            "messages": [{"role": "user", "content": requirement}],
            "tools": [
                {
                    "name": "record_spec",
                    "description": "Record the extracted endpoint validation spec",
                    "input_schema": SPEC_SCHEMA,
                }
            ],
            "tool_choice": {"type": "tool", "name": "record_spec"},  # forcing the tool means no free text
        },
    )
    blocks = res.json()["content"]
    return next(b["input"] for b in blocks if b["type"] == "tool_use")
 
 
def extract_with_openai(system: str, requirement: str):
    res = requests.post(
        "https://api.openai.com/v1/responses",
        headers={"Authorization": f"Bearer {os.environ['OPENAI_API_KEY']}", "content-type": "application/json"},
        json={
            "model": os.environ["OPENAI_MODEL"],  # do not hard-code a model; follow the current docs
            "instructions": system,
            "input": requirement,
            "text": {"format": {"type": "json_schema", "name": "spec", "schema": SPEC_SCHEMA, "strict": True}},
        },
    )
    for item in res.json()["output"]:
        for part in item.get("content", []):
            if part["type"] == "output_text":
                return json.loads(part["text"])
    return None

Each has one limit. The tool approach occupies the tool channel: if your call already has real tools for the model to choose among, forcing the extraction tool fights with them, and you have to drop the forcing and ask for the shape in the system prompt instead. Strict mode's limit is that it supports only a subset of schema keywords — some, such as complex conditional combinations, are unsupported; the first use of a given schema carries a one-time compilation cost; and there is a ceiling on field count. The shared limit is the important one: a schema constrains shape, not content. It can guarantee failureStatus is an integer; it cannot guarantee that integer is a 4xx. It can guarantee rule is a string; it cannot guarantee the string is not empty. Correct content still comes from writing the prompt clearly, with code validation as the backstop.

Templates and variables: from a paragraph to a function

By now the prompt has a role, a task, a format, constraints, examples, and a schema. It is long, and you will notice that most of it is identical on every call, with only a few things changing: the requirement text being extracted this time, the language to output this time, this team's default failure status code. Anyone who writes code recognizes that pattern — the unchanging part is the function body, the changing part is the parameters. A prompt should be written the same way: a function that takes a few arguments and returns the complete prompt string.

const LANG_NAME: Record<string, string> = { zh: 'Chinese', en: 'English' }
 
// Constants are baked into the template; anything that may change becomes a parameter.
// Callers never need to know what the prompt looks like.
function buildSystemPrompt(lang: string, defaultStatus: number): string {
  const langName = LANG_NAME[lang] ?? LANG_NAME.zh
  return [
    'You are a backend engineer reviewing endpoint designs.',
    'Task: from the requirement description the user gives you, extract the endpoint path, the HTTP method, the validation rule for each request field, and the failure status code.',
    `When the requirement does not mention a failure status code, use ${defaultStatus}. Field types may only be string, number, boolean, or date.`,
    `The summary field must be written in ${langName}; every other field keeps the English identifiers exactly as they appear in the requirement.`,
    'Fill in only fields that explicitly appear in the requirement. Do not invent any.',
  ].join('\n')
}
 
const system = buildSystemPrompt(process.env.OUTPUT_LANG ?? 'zh', 400)

import os
 
LANG_NAME = {"zh": "Chinese", "en": "English"}
 
 
# Constants are baked into the template; anything that may change becomes a parameter.
# Callers never need to know what the prompt looks like.
def build_system_prompt(lang: str, default_status: int) -> str:
    lang_name = LANG_NAME.get(lang, LANG_NAME["zh"])
    return "\n".join([
        "You are a backend engineer reviewing endpoint designs.",
        "Task: from the requirement description the user gives you, extract the endpoint path, the HTTP method, the validation rule for each request field, and the failure status code.",
        f"When the requirement does not mention a failure status code, use {default_status}. Field types may only be string, number, boolean, or date.",
        f"The summary field must be written in {lang_name}; every other field keeps the English identifiers exactly as they appear in the requirement.",
        "Fill in only fields that explicitly appear in the requirement. Do not invent any.",
    ])
 
 
system = build_system_prompt(os.environ.get("OUTPUT_LANG", "zh"), 400)

How do you split variables from constants? By exactly the same test you used on D1 to split the system prompt: what will be the same on the next call is a constant; what might differ is a variable. Role, task description, constraints, and schema are constants. Requirement text, output language, and default status code are variables. Add one engineering consideration on top: fewer variables is better. Every additional variable adds a dimension to the space of possible prompts, which doubles the combinations D4's test set has to cover. If a "variable" has only ever taken one value across all your calls, turn it back into a constant.

Templates have a less obvious benefit too: the prompt acquires an identity. It is a function, with a name, a parameter signature, tests, and a place in version control. When D4 covers prompt versioning, this function is the thing being versioned — not string concatenation scattered across the codebase.

Multilingual output: language is a variable, not a second prompt

Many teams approach multiple languages by copying the prompt and translating it, which leaves them with two files, one per language. The problem surfaces three months later: you change one constraint in the first version, forget the second, the two languages start behaving differently, and you have a hard time noticing.

The previous section's code already shows the right approach: there is only one prompt body, and the output language is a template parameter, read by exactly one line — "the summary field must be written in such-and-such language." Any change to any rule then applies to every language automatically. Note the line about other fields keeping the English identifiers exactly as they appear: it guards a real trap. When you ask for English output, the model will happily translate dueDate into due_date or even deadline, and /todos into /tasks. Which fields follow the language and which never change has to be stated in the template.

There is a companion rule on the schema side: enum values do not follow the language. method is always POST, never a localized verb. type is always date, never a localized noun. Enums are read by programs; only human-facing fields such as summary switch languages. Put plainly: the column headers belong to the program, and only the notes inside the cells belong to the human.

What to do when parsing fails: validate, retry, degrade

The forms the new colleague fills in go wrong too: the status code comes back as 200, the mandatory column is left blank. The colleague fills the form wrong; you point it out and ask for a redo, and if it's still wrong you set it aside marked "needs a human." You do not scrap the whole reporting effort over one bad form. Handling model output works exactly the same way, in three layers.

Layer one, validation. Once you have the object, check the business rules in code, one at a time: endpoint starts with a slash, method is in the enum, failureStatus is a 4xx integer, summary is non-empty, every item of validations has the four keys with the right types. This layer does not duplicate the schema — the schema constrains shape during generation, validation checks content after it. The validation function should return an error list, not a boolean, because the next layer needs that list.

Layer two, retry once. When validation fails, splice the error list into the user message and call again: "the previous output had these problems, please fix them: failureStatus must be a 4xx integer." The essential part is retrying with the reason attached — resend the same thing verbatim and you will most likely get the same failure; tell it what was wrong and it usually fixes it on the first try. Do not retry many times; once is enough. Needing more than two says the problem is not this input but your prompt or your schema, and you should go change the template rather than keep retrying.

Layer three, degrade. If both attempts fail, return an empty value, log it, and let the caller decide whether to skip or route it to a human. The most important rule here is do not throw. A failed extraction is a normal business branch, not a program error, and throwing lets one bad requirement abort an entire batch. Do not settle for the "almost right" result from the previous attempt either — half-correct structured data is more dangerous than no data, because everything downstream treats it as true.

Source Reading

Hands-On Lab

This is the course's first runnable lab, and the first four criteria need no API key. Open labs/prompt-engineering-5days/day-03-structured-extraction, run solution/ first to see the behavior, then go back to starter/ and fill in the four exercise points:

1. In solution/, run pnpm install --ignore-workspace and then MOCK=1 pnpm start. Study the JSON each of the three requirements produces, and watch the third one fail validation and then succeed on retry.
2. Switch to starter/ and run MOCK=1 pnpm start once. Notice that the third requirement's failureStatus is a string and gets through anyway — that is the behavior exercise 3 fixes.
3. Complete exercise 1's schema: the four fields of validations.items, the enums, required, and additionalProperties: false.
4. Complete exercise 2's template: replace the hard-coded language and the hard-coded 400 with the lang and defaultStatus parameters, then run with OUTPUT_LANG=en to verify the summary comes back in English.
5. Complete exercise 3's validation and exercise 4's retry, then rerun the third requirement and confirm you get a correct result after the attempt-1 failure. If you have a key, drop MOCK=1 and compare stability against the real model.

Interview Questions

Today's three questions are in the question bank below, covering how schema constraints work and where they stop, the trade-offs in template variables, and the design of parse-failure fallbacks. The follow-up on the third question is the one production systems get asked most often; derive it yourself before reading the analysis.

Checklist and Tomorrow

• Explain why structured output must rely on a schema rather than a line saying "please output JSON"
• Write a prompt as a template with variables, and explain how to split variables from constants
• Write a script that extracts structured fields from natural language, and handle parse failures
• The first 4 acceptance criteria of the structured extraction script pass
• You can answer at least 2 of the 3 interview questions without looking at the points

Tomorrow (D4) answers a question that has been hanging over us: you changed a version of the prompt, so how do you know whether it got better or worse? "This version feels better" is not usable in production, and we are going to turn it into "this version got three more of ten test cases right." D4 teaches you to build a small test set, write an A/B evaluation script, put version numbers on prompts, and recognize five of the most common anti-patterns. Today's structured output is exactly what makes evaluation possible — when the output is fixed fields, scoring can be done in code.