Why AI Answers Change When You Ask the Same Question Differently
How language models interpret prompts — and why the framing of a question often shapes the answer.
Many lawyers experimenting with artificial intelligence notice something strange:
Ask the same question twice with slightly different wording, and the answer can change.
At first glance this can make the technology seem unreliable. In reality, it reflects how large language models actually work.
Understanding why this happens can help lawyers use these tools more effectively and evaluate their outputs more thoughtfully.
AI Does Not Read Language the Way Humans Do
When humans read a sentence, we interpret meaning as a whole.
Large language models process language differently. Before analyzing a prompt, the system breaks the text into small pieces called tokens. These tokens may represent words, parts of words, or punctuation.
The model then evaluates patterns between those tokens based on relationships learned during training.
In other words, the system is not “understanding” language the way a human does. It is identifying statistical patterns between pieces of text.
Small Changes in Wording Can Shift Context
Because the model works by identifying patterns, subtle changes in phrasing can influence the context it detects.
Consider these questions:
• What are the Department of Justice rules on producing text messages?
• What DOJ guidance exists regarding preservation of text messages in litigation?
• What federal rules apply to producing text messages?
To a human reader, these questions appear very similar. But to a language model, they activate slightly different contexts.
The first question suggests agency policy.
The second implies litigation guidance.
The third invites discussion of procedural rules.
Each framing leads the model toward different information.
Why Precision Matters
Lawyers already understand that the framing of a question affects the answer. The same principle applies when interacting with AI systems.
Clear prompts that identify the relevant authority, context, and objective tend to produce more useful responses.
For example, compare:
“What are the DOJ rules on producing text messages?”
with
“What guidance has the U.S. Department of Justice issued regarding preservation and production of text messages in litigation?”
The second prompt provides a clearer framework for the system to interpret.
AI Is a Tool, Not an Authority
None of this means AI tools are unreliable. It simply means their outputs depend heavily on how questions are framed.
Used thoughtfully, these systems can be powerful research assistants.
But like any tool in legal practice, they work best when the user understands how they operate.
And often, the quality of the answer begins with the quality of the question.
¹ Footnotes from Pythia is the Delphoria Learning series exploring artificial intelligence, legal technology, and the systems shaping modern legal practice.