Conclusion/Thesis
The technology for eliciting intended responses from generative AI is rapidly shifting its focus from “how to write clever prompts (instructions)” to “what kind of context (information environment) to provide to the AI.”
Prompt engineering (optimizing spells) is by no means useless, but in the field of utilizing AI agents and building systems, I believe that what truly determines the results is “context engineering (designing the environment).”
The Trigger
Recently, while trying to incorporate LLMs like ChatGPT and Claude into complex tasks and work automation, I have increasingly hit a wall.
No matter how carefully I craft the instructions (prompts), the following problems tend to occur.
-
The AI forgets what was said earlier as the conversation gets longer, or its accuracy drops (context rot phenomenon: corruption of context)
-
If too many complex instructions are given, it ends up overlooking important constraints
-
A single prompt cannot cover all prerequisite knowledge or internal company rules
I was feeling the limitations of an approach that obsesses over “what words (prompts) to use to give instructions,” and when I encountered documents and design philosophies on “Context Engineering” advocated by AI companies like Anthropic, it strongly resonated with the challenges I was facing in my own practical work and verification, which was the trigger for organizing these thoughts.
Consideration
1. The decisive difference between prompts and context
To summarize the difference between the two briefly, it is as follows.
ItemPrompt EngineeringContext EngineeringMain Focus”How to ask” (Wording/Expression techniques)”What to show” (Information environment/Prerequisite structure)ScopeA single instruction entered by the userSystem prompts, memory, RAG, and the overall picture of external toolsRole Analogy”How to give instructions” to a field worker”Organizing the desk and arranging materials” where the worker worksnewcommand{arraystretch}{1.5} %
begin{array}{l:l:l}
text{Item} & text{Prompt Engineering} & text{Context Engineering} cr
hdashline
text{Main Focus} & text{“How to ask” (Wording/Expression techniques)} & text{“What to show” (Information environment/Prerequisite structure)} cr
text{Scope} & text{A single instruction entered by the user} & text{System prompts, memory, RAG, and the overall picture of external tools} cr
text{Role Analogy} & text{“How to give instructions” to a field worker} & text{“Organizing the desk and arranging materials” where the worker works} cr
end{array}
Prompt engineering is a “skill of the writer (words)” and is very useful for one-off interactions. However, in tasks spanning multiple steps or agent systems,no matter how perfect the instructions are, if the reference materials (context) are cluttered, the AI will make mistakes.
2. The importance of designing “what not to include”
The context window of LLMs (the amount of information that can be read at once) is expanding year by year, but “just because it’s wider doesn’t mean you should stuff everything into it.”
It has been reported that if the amount of information is too large, the model overlooks important descriptions. Therefore, the essence of practicing context engineering lies in the following three controls.
(1)Separation of static information (unchanging premises)
Clearly fix the basic policies and rules of the project (configuration files and system prompts).
(2)Narrowing down dynamic information (retrieving when necessary)
Use RAG or external tools to prepare “only the information needed now” just-in-time, rather than having the AI read all the data.
(3)Organizing memory (distinguishing between short-term and long-term)
Separate the ongoing conversation history (short-term memory) from logs recorded in external files, etc. (long-term memory) to prevent context bloat.
I feel that an approach of preparing an “information environment where the AI does not get lost,” rather than trying to improve the AI’s capabilities through “clever instructions,” will dramatically increase the reproducibility and reliability of the output.
Summary
If prompt engineering is “the art of conversation with AI,” then context engineering is “system design for AI.”
In the future, the opportunities for humans to make full use of “prompt techniques” every time will decrease, and instead, the importance of “creating an environment where prerequisite information and rules are filed so that the AI can act without hesitation, and connecting it to optimal external tools” will likely increase even further.
If you are not satisfied with the behavior of the generative AI at hand, before changing the words (prompts), you might find new discoveries by reviewing “what kind of information (context) is currently lined up in front of the AI.”
