The Forefront of LLM Control Beyond Prompt Engineering

Introduction | Is LLM Response Quality Determined Solely by the “Prompt”?

When using generative AI like ChatGPT, have you ever felt the following?

• No matter how many times I change the instructions, I don’t get the expected response

• It confidently provides incorrect information

• Even with the same question, the content and format of the response are inconsistent

• Even when I have it reference internal documents, it uses outdated information

• I want to entrust tasks to an AI agent, but I’m worried about operational errors

As a countermeasure to these problems, the first thing that comes to mind is “prompt engineering.” However, in actual AI systems, there are many issues that cannot be solved just by improving prompts.

Therefore, in recent years, in addition to prompt engineering, the following approaches have been attracting attention:

• Prompt Engineering

• Context Engineering

• Harness Engineering

• Loop Engineering

All of these are design methods for drawing out the performance of generative AI and making it operate as expected, but what they design differs. In this article, we will explain the differences, specific examples, proper usage, and the history of the development of these four methods with diagrams. This content is useful not only for those studying the AI field of the IPA Information Technology Engineers Examination but also for those who are actually using generative AI in their work.

1. What are “Quality Improvement” and “Control” in LLMs?

First, let’s organize what it means to improve the quality of generative AI. LLMs (Large Language Models) generate text probabilistically based on input text, conversation history, etc. While they have extremely high language processing capabilities, their output is not always correct. Therefore, the following two concepts become important:

Quality Improvement
This involves evaluating and improving how appropriate an LLM’s response is for a given purpose. For example, it enhances accuracy, usefulness, consistency, validity of grounds, and safety.

Control / Steering
This involves controlling the behavior of the LLM and the operations it can perform according to the purpose. For example, specifying the response format, having it reference only specific internal documents, or restricting update operations to external systems fall under this category. What we want to note here is that while quality improvement and control are closely related, they do not mean the same thing.

For example, even if an LLM can generate natural text, if it sends confidential information externally, it cannot be considered appropriate as a business system. Conversely, even if permission control is perfect, if the response is inaccurate, it cannot provide sufficient value. In other words, both ingenuity to draw out the capabilities of the LLM and mechanisms to use it safely and stably are necessary.

2. Comparing the four engineering methods

First, let’s grasp the overall picture.

The way to remember them is simple.
Prompt: What to instruct
Context: What to show
Harness: What environment to execute in
Loop: When and how to repeat

For example, let’s consider a case where you assign a generative AI the task of “answering questions about internal company regulations.”
In Prompt Engineering, you instruct the response format and precautions to take when answering.
In Context Engineering, you search for the necessary internal regulations and pass them to the LLM.
In Harness Engineering, you design mechanisms such as access permission checks, source verification, and operational restrictions.
In Loop Engineering, you design processes such as re-searching if information is insufficient, re-verifying answers, and iterative processing until completion conditions are met.

These are not competing technologies.They are things used in combination within a single AI system. Also, the four are not strictly independent. For example, a prompt is part of the context, and loop processing can be included within a harness.

3. Detailed explanation of each engineering method

3-1. Prompt Engineering

Prompt Engineering is a method of designing and improving instructions to an LLM to make it easier to obtain desired responses.

It can be said to be the most familiar engineering method when using generative AI. For example, consider the following prompt.

Before improvement

Explain this system.

With this, it is unclear what should be explained and in how much detail.

Therefore, we change it as follows.

After improvement

You are an IT infrastructure technical instructor.
Please explain RDBMS transaction isolation levels for beginners. Please include the following items. Provide brief explanations for technical terms, and create a comparison table at the end.

– Purpose of isolation levels
– The four representative isolation levels
– Difference between dirty reads and phantom reads
– Concrete SQL execution examples

In this way, by clarifying the target audience, purpose, constraints, and output format, it becomes easier to bring the response closer to the objective. Representative methods include the following.

3-2. Context Engineering

Context Engineering is a method of selecting, organizing, and maintaining information passed to an LLM to make it easier to obtain responses that meet the objective.

While Prompt Engineering focuses on “how to instruct,” Context Engineering focuses on “what information to provide.”

For example, suppose you want to ask a generative AI about internal work regulations.

Please tell me the latest remote work application rules.

No matter how much you refine the prompt, if the LLM cannot refer to the latest internal regulations, you cannot expect an accurate answer. Therefore, we search for the necessary internal regulations and pass the relevant sections to the LLM. RAG (Retrieval-Augmented Generation) is often used for such searching and referencing of external information.

However, RAG and context engineering do not mean the same thing. RAG is a technology that searches for external information and uses it for generation. On the other hand, in context engineering, in addition to search results, we design the entire set of information that the LLM refers to, including conversation history, work status, memory, tool descriptions, and system instructions.

The main approaches are as follows.

The important thing is not just passing a large amount of information. There is an upper limit to the amount of information an LLM can handle at one time, and the inclusion of unnecessary information can sometimes lead to a decrease in response quality. Therefore, passing the necessary information, at the necessary time, in the appropriate amount is important. Note that external materials may contain misinformation or malicious instructions. Verifying the reliability of information sources and controlling access are also essential.

3-3. Harness Engineering

From here on, we will discuss a concept that is gaining particular attention due to the spread of AI agents.

Harness engineering is the design of tools, execution environments, constraints, and verification mechanisms to ensure that AI agents can work safely and stably. The term “harness” here refers to the execution foundation and peripheral mechanisms that support the AI.

For example, imagine a scenario where you have a generative AI write a program. If you only want the AI to write code, you could handle it by refining the prompt.

However, in actual development, that is not enough.

• Does the created program work correctly?

• Do the tests pass?

• Does it break existing functionality?

• Has it changed unauthorized files?

• Can it be recovered when a problem occurs?

To address these challenges, we prepare various mechanisms outside the LLM.

For example, consider a system that makes an AI agent generate SQL. Simply instructing it with a prompt like “do not delete data” is not a reliable safety measure. Therefore, we set the database permissions themselves to read-only or restrict the executable SQL. This is the idea of controlling execution on the system side rather than relying on the LLM’s own judgment.

In February 2026, OpenAI introduced the importance of this harness engineering along with development examples utilizing Codex. It can be said to be an important design area for evolving generative AI from a “tool that answers” to an “agent that executes work.”

3-4. Loop Engineering

Loop engineering is a design philosophy for creating mechanisms that continuously cycle through the execution, evaluation, and retrying of AI agent tasks.

For example, imagine an AI agent that writes programs.
With a standard generative AI, the following interaction would occur:

・A human instructs the AI to write code, and the AI generates it.
・The human tests it, finds an error, and asks the AI to fix it.
・The human tests it again, and if there are issues, they have the AI fix it once more.

In this scenario, the human must provide instructions for every step.
Therefore, we design a mechanism like the following:

1. AI generates code
2. Run automated tests
3. Evaluate test results
4. If it fails, fix it based on the error details
5. If it passes, complete the task after necessary human confirmation

By repeating these steps according to specific conditions, work can proceed without the human needing to issue instructions every time. However, loop engineering does not refer only to simple ‘re-execution upon error’ processes. It is discussed as a broader concept that includes the design of continuous workflows such as scheduled execution, task discovery, assigning tasks to multiple agents, progress management, and result verification.

Note that ‘Loop Engineering’ is a relatively new term that gained attention in 2026. Its definition as a technical term and its scope have not yet been standardized. Furthermore, it is not enough to simply keep the AI running indefinitely.

For example, the following stop and control conditions are important:

• Maximum number of executions

• Limits on time or API usage costs

• Objective criteria for determining completion

• Conditions for handing over to a human in case of an error

• Timing when approval for operations is required

When designing a loop, ‘when to stop’ is just as important as ‘what to repeat’.

4. History of Development | From Prompts to Autonomous AI Agents

So, why have new engineering methods appeared one after another? The background to this is the expansion of the scope of work entrusted to generative AI.

Stage 1: Refining Instructions

In the stage where generative AI was mainly used in a chat format, refining the instructions entered by the user was important. This is where prompt engineering became central.

Stage 2: Providing External Information Appropriately

As generative AI began to be used in corporate operations and specialized fields, it became necessary to handle information that the LLM had not been trained on. Therefore, broader context design, including RAG and conversation history management, became more important.

Stage 3: Preparing an environment where AI can work

As generative AI began to operate tools and execute actual processes, the design of permissions, tools, and verification environments became just as important as instructions and reference information. This is where the concept of harness engineering is gaining attention.

Stage 4: Executing tasks continuously

As AI agents began to handle multiple tasks, the operational bottleneck became the need for human intervention for every instruction. Consequently, mechanisms that continuously cycle through task discovery, execution, verification, retries, and state management have become a priority. This has led to the concept of loop engineering.

As shown, the scope of what engineers design has expanded alongside technological advancements. However, the emergence of new methods does not mean that older ones have become obsolete. For example, even with advanced AI agents, prompt design and appropriate context management remain essential.

5. Concrete Example | Improving the response quality of an internal FAQ system

Let’s organize the content covered so far using a practical example.

Suppose we are developing the following internal FAQ system.
Objective: When an employee asks a question about internal regulations, the system provides an answer while citing the supporting documents.

For example,

Please tell me the remote work application rules for this fiscal year.

is the type of question the system answers.

Step 1: Improving the prompt

First, we define the rules for generating responses.

For example, we provide instructions like the following.
This allows us to control the response format and policy.

You are an internal regulations guide.
In your response, please include the name and revision date of the referenced document.
Do not guess information not stated in the regulations, and clearly state if something is unknown.

Step 2: Improving the referenced information

Next, we search for the internal regulations necessary for the response. For example, we use RAG to retrieve documents relevant to the question. At this stage, it is necessary to design the system to prioritize the latest versions and exclude unnecessary documents. It is also important not to include documents in the search results that the user does not have permission to view.

This is the concept of context engineering.

Step 3: Set up the execution environment and verification mechanisms

Next, we will establish the following mechanisms on the system side.

• Verify user access permissions

• Confirm whether the documents serving as the basis for the answer exist

• Restrict inappropriate external operations

• Record processing results as logs

For example, if an LLM answers, “The application deadline is the end of this month,” we verify whether that basis actually exists in the reference documents. A design where the answer is withheld if the basis cannot be confirmed is also a possibility.

This is harness engineering.

Step 4: Systematize evaluation and retries

Finally, we establish a mechanism to perform re-searches or re-verifications as needed. For example, if the reference document is outdated, we attempt to search for a newer document. If there is insufficient evidence, we do not finalize the answer, but instead conduct additional research or hand it over to a human representative.

This is an example of loop design. However, we also define retry counts and termination conditions so that it does not continue searching indefinitely.

In this way, the four methods support the quality of AI systems from different angles.

6. How to use them differently in practice?

Let’s organize how to choose between the four methods we have learned so far, based on their purpose.

The important thing is not to design everything in a complex way from the start. For simple text summarization, improving the prompt alone may be sufficient. On the other hand, for AI agents that involve accessing internal company data or updating external systems, mechanisms such as context management, permission control, verification, and approval become necessary.

First, clarify the problem you want to solve, and improve from the necessary design areas.
This is the most realistic way to proceed.

7. Other types of “○○ Engineering”

We have introduced four methods so far, but in the LLM field, various other names are also used. We will organize the representative ones.

7-1. Agent Engineering

This involves designing AI agents so that they can use tools, break down tasks, and advance processing according to their objectives. It is a relatively broad concept that includes tool selection, workflows, state management, and role distribution among multiple agents. It also has overlapping parts with Harness and Loop.

7-2. Evaluation Engineering (Evals)

This is a design approach for creating test data, evaluation criteria, and evaluation processes to assess the output quality of generative AI. For example, to check the quality of an internal FAQ system, you prepare an evaluation set containing representative questions. Then, you verify the accuracy of the answers, the presence of supporting evidence, and the absence of authorization violations.

There is also a method called “LLM-as-a-Judge,” where an LLM is used to evaluate the output of another LLM. However, since the judgment of the evaluating LLM can also contain errors or biases, it is important to combine it with human verification or objective testing for critical evaluations. Note that the term “Evaluation Engineering” itself is not a unified, single standard technical term. In practice, expressions such as “Evals” and “Evaluation-Driven Development” are also used.

7-3. Tool Engineering

This is a design approach for tool inputs/outputs, descriptions, and error handling to ensure that an LLM can use external functions safely and appropriately. For example, when providing a database search function to an LLM, you clarify what arguments it accepts and what results it returns. If a tool is difficult to use, the LLM may fail to select the appropriate tool or call it with invalid arguments.

7-4. Graph Engineering

This is a design approach for modeling dependencies, branching, parallel execution, and approval paths that exist between multiple tasks or AI agents using a graph structure. An example would be a workflow that combines an investigation AI, an analysis AI, and a review AI to carry out tasks. However, as of November 2026, the classification of Graph Engineering as an independent engineering domain is not standardized. Since it overlaps with existing workflow design and agent orchestration, it is best understood as an evolutionary organization of these concepts.

8. Often Overlooked Points | “Evaluation” is Essential for Quality Improvement

Having read this far, you might be wondering, “Which method should I use for my AI system?” In doing so, what is often overlooked is the evaluation mechanism.

For example, suppose you change a prompt and the answer to one question improves. However, if the quality of answers to other questions deteriorates, the system as a whole cannot be said to have improved. Therefore, you should prepare an evaluation dataset containing representative questions and compare the results before and after the change.

Examples of evaluation metrics include the following:

The important thing is to decide on evaluation metrics in advance. Without an evaluation mechanism, it becomes difficult to judge which improvements were effective. Also, having the AI evaluate its own answers as “correct” does not constitute objective quality assurance. It is important to combine testing, rule verification, cross-referencing with actual data, and human review.

AI quality improvement is not something that ends with modifying a prompt; it is a continuous effort of repeating evaluation and improvement.

9. How should one study for the IPA Information Technology Engineers Examination?

Finally, let’s organize this from the perspective of IPA exam preparation.

9-1. The first thing to grasp is Prompt Engineering

In the Applied Information Technology Engineer Examination syllabus Ver. 7.0, “Prompt Engineering” is explicitly mentioned as an example of AI-related terminology.

Therefore, you should understand the following basic knowledge:

• The purpose of prompt engineering

• Differences between Zero-shot and Few-shot

• Reasons for specifying instructions, conditions, output formats, etc.

• LLM hallucinations and countermeasures

• Utilizing external information via RAG

9-2. Understand context design and AI agents as advanced knowledge

Context, Harness, and Loop Engineering are useful for understanding the implementation technology and operational design of AI systems. However, it cannot be concluded that these terms will appear exactly as they are on the IPA exam.

In particular, Harness Engineering and Loop Engineering are relatively new concepts. At this point, it is recommended to understand the underlying technology and design objectives rather than memorizing the terms themselves. For example, risks when granting operational authority to AI, mechanisms for retrieving necessary information with RAG, tracking via logs, and verification and feedback are also related to IT technologies other than AI.

9-3. Comprehension Check

Question 1
Which of the following is the most appropriate method for specifying output formats and response conditions to an LLM?

A. Harness Engineering
B. Prompt Engineering
C. Loop Engineering
D. Context Engineering

Correct Answer: B
The method of designing and improving instruction text for an LLM is Prompt Engineering.

Question 2
Which of the following is most directly related to the mechanism of searching internal documents and passing information relevant to a question to an LLM?

A. RAG
B. Model Quantization
C. Overclocking
D. Database Normalization

Correct Answer: A
RAG is a technology that searches for external information and uses it for generation processing. It can be used as a means of providing information in Context Engineering.

Question 3
An AI agent automatically tests a generated program and attempts to fix it if it fails. Which of the following is the most appropriate design for controlling the repetition of this process?

A. Prompt style adjustment
B. Labeling of training data
C. Loop Engineering
D. Model Distillation

Correct Answer: C
Designing a mechanism that repeats execution, verification, correction, and re-execution is a typical example of Loop Engineering.

10. Summary | In future AI utilization, the design of ‘mechanisms’ is as important as ‘instructions’

In this article, we explained four representative engineering methods for improving and controlling LLM quality. Let’s organize them one last time.

The use of generative AI is expanding from the stage of having questions answered on a chat screen to the stage of referencing external information, operating tools, and advancing complex tasks. Along with this, the things to consider to improve quality have also increased. Even if you try to improve response quality with prompts alone, you cannot solve problems such as lack of information or execution environment issues. On the other hand, building a complex mechanism does not necessarily mean that quality will improve.

It is important to clarify what you want to delegate to AI and to design mechanisms for information, environment, verification, and iteration that suit that purpose.When learning about generative AI from now on, it is desirable to acquire not only individual models and prompt techniques, but also the perspective of how to design the entire AI system.

Related Articles

Reference Materials

The technical explanations in this article are based on the following publicly available materials.

1. IPA “Applied Information Technology Engineer Examination Syllabus Ver. 7.0”
   https://www.ipa.go.jp/shiken/syllabus/gaiyou.html

2. IPA “Guidelines for the Introduction and Operation of Text-Generating AI” (July 2024)
   https://www.ipa.go.jp/jinzai/ics/core_human_resource/final_project/2024/f55m8k0000003spo-att/f55m8k0000003svn.pdf

3. Anthropic “Effective context engineering for AI agents” (September 29, 2025)
   https://www.anthropic.com/research/effective-context-engineering-for-ai-agents

4. LangChain “Context Engineering” (July 2, 2025)
   https://www.langchain.com/blog/context-engineering-for-agents

5. OpenAI “Harness engineering: leveraging Codex in an agent-first world” (February 11, 2026)
   https://openai.com/ja-JP/index/harness-engineering/

6. Addy Osmani “Loop Engineering” (June 8, 2026)
   https://addyo.substack.com/p/loop-engineering

7. OpenAI “Evaluation best practices”
   https://developers.openai.com/api/docs/guides/evaluation-best-practices

8. Anthropic “Building effective agents” (December 19, 2024)
   https://www.anthropic.com/research/building-effective-agents

Originally Appeared Here