Featured image: AI-generated conceptual illustration. Sources checked 26 October 2026.
A request can be perfectly valid JSON and still describe the wrong action. It can name a real customer, supply a correctly formatted email address and ask for an unauthorized export. Structured output helps software consume model responses. It does not establish that the response is true, appropriate or permitted.
This distinction matters because structured interfaces often feel more trustworthy than free text. A neat object with typed fields looks ready for automation. Before connecting it to a business action, separate the guarantees of the format from the checks required by the workflow.
What a schema can tell you
A schema describes expected fields, types and allowed structures. It can help require an account identifier, constrain a status to a known set or distinguish a missing value from a supplied one. These properties reduce parsing ambiguity and make application behavior easier to test.
The OpenAI structured-output documentation describes generating outputs that conform to supported schemas. The precise support and behavior depend on the API and model configuration, so implementation teams should use the current documentation rather than assume that every JSON-like response has the same guarantees.
The schema cannot establish the business relationship between fields simply because each field has a valid type. A customer identifier and order identifier may both exist while belonging to different customers. An amount may be numeric while exceeding an authorized limit. An email address may be well formed while belonging to the wrong recipient.
A hypothetical export request
Imagine an internal assistant asked to prepare a summary for an account owner. It produces a structured request containing an account ID, a report type and an email destination. Every field passes the format checks. The destination, however, came from an untrusted document rather than the employee’s instruction or an approved contact record.
If the application treats schema compliance as approval, it may disclose the report to the wrong person. The failure is not malformed JSON. It is a missing relationship between the requested destination and the authority granted for the task.
The application should verify the account scope, report permission and recipient identity before sending. It should also distinguish preparing the report from distributing it. A user who authorized a draft did not necessarily authorize an external message, regardless of how neatly the model expresses the send request.
Build a validation ladder
Parse and shape. Confirm that the response follows the expected structure. Reject unsupported fields where appropriate and handle missing or invalid values explicitly. Do not silently fill critical fields with defaults that change the action’s meaning.
Business meaning. Resolve identifiers against authoritative systems and check relationships between them. Validate units, date ranges and state transitions. A request to close a case should not be accepted merely because closed is an allowed string value.
Authority. Check the authenticated user, resource scope and required approval. Do this in application code at execution time. Do not accept a model-generated field such as approved: true as evidence that an authorized person actually approved anything.
Destination handling. Treat text according to where it will be used. Content rendered in a browser, passed to a database or included in a file path needs appropriate handling for that context. A structured wrapper does not make the string inside it safe to execute or render without controls.
OWASP’s improper-output-handling guidance specifically addresses insufficient validation and handling of model output before downstream use. This is ordinary application security meeting a new source of generated input, not a reason to discard established defensive practices.
Keep generated text away from interpreter boundaries
Avoid turning arbitrary model text into a shell command or database query when a narrow operation can express the business task. A tool that accepts a validated record identifier and a permitted field change is easier to constrain than a general instruction string that downstream code interprets.
Where dynamic content is necessary, use the relevant safe interface and validation rules. Engineers should review each interpreter boundary deliberately. The business owner can help by asking whether the integration exposes a specific operation or a broad ability to execute whatever the model produces.
The OpenAI function-calling guide documents how model requests connect to application-executed tools. That execution boundary is the right place to enforce the contract. The model proposes inputs; the application decides whether they are acceptable for the authenticated task.
Test valid but unacceptable requests
- A valid order ID belonging to a different customer.
- A correctly formatted destination outside the approved recipient list.
- A supported status change that is illegal from the record’s current state.
- A plausible date range that includes restricted historical data.
- A model-supplied approval flag with no corresponding approval event.
- A text field containing content unsafe for its eventual rendering context.
These cases are useful because they defeat the intuition that a schema passing means the work is safe. Include ordinary mistakes as well as malicious inputs. An agent can choose the wrong identifier without any attacker being involved, and the control should catch that too.
Use structure to improve review, not bypass it
Structured fields can make an approval card clearer. They can show exact recipients, amounts and before-and-after values. They can also support deterministic checks and more useful logs. Those are substantial benefits when the application preserves the difference between a proposal and an authorized action.
Keep source references for extracted facts where possible. If a model supplies a delivery date, a reviewer may need to inspect the clause that supports it. A field value without provenance can be easy to process and difficult to trust.
The practical rule is to let schemas solve format problems and let policy controls solve authority problems. Combine them with business validation and safe output handling. That creates a stronger system than asking structured output to provide guarantees it was never designed to supply.
Sources and further reading
OpenAI: Structured model outputs. Schema-constrained output documentation; checked October 2026.
OWASP: Improper Output Handling. 2025 validation and downstream-handling risks; checked October 2026.
OpenAI: Function calling. Application-side tool execution boundary; checked October 2026.
Leave a Reply