*This article is based on implementation knowledge and research paper data as of November 2026.
“The AI that was answering accurately at first starts to miss premises and provide misguided answers as the dialogue gets longer.”
If you use Large Language Models (LLMs) daily in your work, development, or content creation, isn’t this a phenomenon everyone has experienced at least once?
In many workplaces, when such a drop in accuracy occurs, there is a tendency to try to solve it with minor prompt adjustments (additions), thinking, “Maybe the prompt instructions were ambiguous” or “Wouldn’t it be fixed if I just added a note saying ‘Ignore previous instructions’?”
However, the paper “LLMs Get Lost in Evolving User Intent,” published by a Microsoft Research team in 2026, provides important insights for considering this issue.
The research confirmed that even for LLMs that show high performance in single-shot tasks, performance tends to drop significantly when user intent is added, modified, or shifted during a dialogue. It was demonstrated that single-shot benchmarks alone cannot sufficiently evaluate the ability to follow long-term dialogues in practical work.
It is necessary to think from the perspective of system design, including task decomposition and context management, rather than attributing the problem solely to the skill of the prompt.
1. Why AI with high performance in static evaluation loses performance in “dynamic dialogue”
Even models that show high performance in general benchmarks do not necessarily demonstrate the same stability in long-term dialogue tasks. Behind this lies a decisive difference between evaluation environments and practical work environments.
Human intent “dynamically changes” during the course of a dialogue
In real-world work, it is rare for a human to perfectly present all constraints and requirements in the first prompt. Usually, intent changes and updates gradually as the dialogue progresses, as follows:
-
Gradual addition of conditions: “Oh, by the way, assume a Windows environment this time,” or “Keep it under 2,000 characters.”
-
Change in direction of premises: “Actually, let’s proceed based on the B plan we discussed earlier, not the A plan.”
-
Revision of policy: “The previous expression is a bit too stiff, so change it to a more casual tone.”
The important point pointed out by the paper is that while LLMs are good at processing “long prompts that are complete from the start,” they are significantly vulnerable to “situations where intent dynamically changes during a dialogue.”
2. Avoiding the confusion between “Context Rot” and “Evolving Intent”
Among engineers, the phenomenon where AI breaks down as conversations get longer has increasingly been collectively called “Context Rot.” However, to take appropriate measures in practice, it is necessary to distinguish between these two phenomena.
-
Context Rot (limit of capacity/attention): A phenomenon where the number of tokens (amount of conversation) simply becomes enormous, and the model’s Attention mechanism homogenizes and loses track of important past constraints.
-
Evolving Intent (dynamic change of intent): A phenomenon where contradictions arise between “past statements” and “recent correction instructions,” causing the model to become confused about which to prioritize, rather than the length of the conversation itself.
No matter how much the context window expands to millions of tokens, as long as past logs remain in the history, interference between old and new instructions (Evolving Intent) cannot be prevented. An approach that keeps the work environment itself clean is required.
3. Practical design principles: Don’t tinker with prompts, draw “boundaries”
The standard approach to suppressing accuracy drift caused by prolonged dialogue is not to rely on adding more prompts, but to “clearly separate the boundaries of sessions, roles, and execution environments.”
Design principles that function effectively in the field can be summarized into the following three points.
① The one-task, one-session principle
Stop continuing research, design, writing, and debugging endlessly in a single chat screen. When important conditions change or confusion arises, carry over only the decisions and switch to a new session.
② Isolation into sub-processes and sub-agents
Do not have the main AI perform web browsing or scan massive raw logs itself; isolate tasks that generate large amounts of intermediate data to independent separate agents or external scripts.
③ Passing lightweight context (summary facts)
Instead of carrying over the entire past chat log as raw data, pass only the “summary facts”—which extract only the decisions and prerequisites—to the next step.
4. The dangers of fully automated agents and “defense-in-depth” design
Operating by leaving “autonomous problem solving” entirely to an AI agent is one of the highest-risk patterns.
If an autonomous loop runs while an intent misunderstanding has occurred, the AI falls into a chain of hallucinations where it “takes the next step based on wrong premises and generates even more distorted code to cover up that failure.” As a result, thousands of lines of unnecessary code are generated, or API quotas are exhausted in a short time.
The line of defense to prevent this is “defense-in-depth.”
-
First line of defense (mechanical verification): Deterministic processes such as script syntax checking, static analysis, and format inspection via regular expressions are mechanically verified on the side of lightweight scripts like Python.
-
Second line of defense (limiting autonomous loops): Set an upper limit, such as “up to 2 steps,” for self-repair by the AI during errors to stop the process before it becomes a quagmire.
-
Third line of defense (human approval gate): Always include human confirmation before destructive operations such as commits, production deployments, or deletion of important files.
“Trusting AI’s capabilities” and “leaving things entirely to AI without safety valves” are completely different things.
5. Summary: Success or failure in AI utilization is determined by “boundary design”
The most important thing in dialogue with AI, more than crafting clever prompts, is maintaining the “cleanliness of the workspace to keep the AI from getting lost”.
-
When the conversation becomes complex, leave a summary and reset the session
-
Isolate large amounts of raw data into sub-processes and receive only the conclusions
-
Set limits on autonomous loops and implement multi-layered defense with mechanical verification and human approval
It is not just about micro-techniques like prompt engineering, but about how to draw the boundaries of the entire system. By simply adopting this perspective, AI transforms into a surprisingly robust and reliable partner in practical work.
☕ Roasting ‘ordinary coffee’ in between sharpening my thoughts
036 Factory’s main business is a small, home-roasting coffee workshop where we hand-pick raw beans one by one after receiving your order and roast them.
I roast small batches of ‘ordinary coffee that you can drink every day’ without any off-flavors, to help reset your head after it has been heated up by AI implementation, programming, or writing. Please take a look at our workshop during a break from your work or thinking.
036 FACTORY Take a look at the beans in our online store
