The same JSON Schema can behave differently across models
The same prompt and the same JSON Schema produced less consistent results than expected across several . The tested models were GPT-5.5, , Gemini 3 Flash, , and Kimi K2.6. The goal was not to compare reasoning skill or rank model quality.
The goal was to see how reliably each model followed the same required output shape. Some providers followed the schema more consistently than others. Valid JSON did not always mean the answer matched the exact data contract the application expected.
Small provider differences could break later parsers when a system assumes every model will return identical structure. Production AI systems may need provider-specific validation or workarounds for .
Key points
- The same prompt and JSON Schema were sent to five s.
- Valid JSON does not guarantee the output matches the expected data contract.
- Small output differences can break downstream parsers.
- Production AI systems should add validation around .
- Cost-saving model swaps should be tested with the app’s real schema and prompts.