1. Introduction — The Scope of Design Has Expanded Beyond the Model
How to build AI agents has been discussed alongside the term “…engineering.” In Japanese explanations, it is often introduced as the following three layers:
・Prompt Engineering: How to write a single request
・Context Engineering: What information to show the model
・Harness Engineering: How to build the environment (tools, testing, retries) in which the model operates
Entering 2026, a five-layer organization has emerged in the English-speaking world, adding “loops” and “graphs” to the previous layers.
・Prompt: A single request
・Context: Information shown to the model
・Harness: Management of a single execution
・Loop: Repetition of the execution itself
・Graph: How to connect processes or multiple agents
Each layer designs one level outside the previous one. They are not competing technologies, but nested ones. The outermost “Graph Engineering” is still a new term, and some say it is “just a new name for a way of building that has existed for a long time.”
This article is a column. No new measurements have been taken. I will apply what I have been testing in this magazine using small local models to this five-layer organization. The goal is to clarify how much has actually been measured and where I have yet to experiment.
To give the conclusion first, what I measured in this magazine is up to the third layer, the harness. And what worked at each layer was the ingenuity of not trusting the model’s self-reporting and verifying the results from the outside. I have not yet measured the fourth layer, loops, or the fifth layer, graphs.
2. The First and Second Layers — Prompt and Context
These two layers have already been covered in this magazine.
An example of a prompt is “having a local LLM read contract clauses.” When I asked about 16 clauses not written in the contract all at once, the small model hallucinated that they existed (27 out of 30 times). When I changed the way of asking to have it choose between “exists” or “not mentioned” for each item, the hallucinations disappeared. However, when I checked the clauses that were actually written using the same method, there were models that answered “not mentioned” for everything. The results change significantly depending on how you ask, but it was necessary to verify separately whether the changed results could be trusted.
An example of context is “RAG built with a local LLM.” It is a mechanism that searches for documents related to a question and shows them to the model together. In this article, just by forgetting one convention of the embedding model, unrelated documents were searched, and the model answered “I don’t know.” No error occurred. I could not tell whether the information shown was correct without verifying it from the outside.
However, both are about the ingenuity of “how to ask” and “what to show” the model. The model is the one that provides the answer until the end. There is no mechanism in these two layers to stop the model when it makes a mistake.
3. The Foundation — The ‘Round-trip’ of Calling Tools (Sometimes called a loop, but distinct from the 4th layer)
Before discussing the 3rd layer harness, let me clarify one term.
When we say ‘loop’ in the context of agents, it has two meanings.
・Within a single execution, the model repeatedly calls tools and receives results
・Repeating the execution itself from the outside (the 4th layer loop in the 5-layer organization)
In past articles of this magazine, I have used ‘loop’ to refer to the former. In this article, to avoid confusion, I will call the former a ’round-trip’.
A round-trip is the simplest mechanism, as described in the article ‘Giving Tools to Local LLMs’.
依頼 → モデル「calculate を呼んで」→ Pythonが実行 → 結果をモデルに返す
→ モデル「write_file を呼んで」→ Pythonが実行 → 結果をモデルに返す
→ モデル「保存しました」(道具を呼ばない)→ 終わり
In code, it is a single `for` statement. It executes while the model calls tools and returns results, and ends when it stops calling them. There are usually two conditions for ending.
・The model answered without calling a tool
・The round-trip limit (maximum number of steps) was reached
The advantage of a round-trip is that it is short to write and does not require changing the code even if the request changes. The model decides what to call and in what order. Consequently, the performance depends on the model.
In the article ‘Don’t Trust an Agent’s “I Saved It”‘, I asked three 7-8B class models to ‘Calculate 23 * 17 and save the result to result.txt’. There were failures such as answering ‘I saved it’ without calling the save tool, writing a number different from the calculation, or the tool call appearing as plain text.
I also conducted measurements at home by changing the round-trip limit (not written as an article). Even when changing the maximum number of steps from 2 to 6, success did not increase in a consistent direction (10 trials each). This is because in most cases, the model itself answered ‘I’m finished’ and stopped before reaching the limit. The limit was a safety device to stop runaway processes, not a knob to increase the success rate.
I also conducted measurements where I returned an error message to the model when a tool failed. When I forced the first save attempt to fail, none of the three models re-saved it in most cases (out of 10 trials each, 30 total across 3 models, there were only 2 successes). Returning an error does not necessarily mean the model will retry on its own.
With just a round-trip, ‘it’s over when the model says it’s over.’ The 3rd layer harness is what intervenes from the outside.
4. 3rd Layer Harness — What I Actually Measured in This Magazine
Harness is originally a term for horse tack. In software, a mechanism that operates a test subject from the outside to verify results is called a ‘test harness’. In agents, it refers to a mechanism attached to the outside of the model to pass tools, verify movements, and force retries.
In the article ‘How Much Does a Harness Increase the Success Rate of Small Models?’, I added components to the round-trip one by one and compared the success rates. The added components were the following three:
・Self-verification: When the model is about to answer, ask once, ‘Is everything complete?’
・Completion condition inspection: When the model is about to answer, check from the outside whether the file contains the necessary content, and if it is insufficient, send it back with a request to ‘Please save it’.
– Retry: If a tool returns an error, the harness executes the same call one more time.
For a request where saving fails once, a raw round-trip succeeded only 0 to 2 times out of 20. Adding a completion condition check increased this to 13 to 15 times, and adding both checks and retries brought it to 16 to 20 times (across all 3 models). On the other hand, self-verification saw success rates increase for some models and decrease for others.
What this result tells us is that there are two types of harness components.
– Components that ask the model (self-verification): Ultimately, the model makes the judgment. If it answers “Completed,” that is the end of it.
– Components that verify from the outside (completion condition checks, retries): Code outside the model makes the judgment. The contents of the file are determined regardless of the model’s answer.
The latter was what proved effective. Even if self-verification takes the form of a harness component, its content is closer to “adding one more prompt.” Like the improvements up to the second layer, it leaves the final judgment to the model. I believe that what works in a harness are components that do not blindly trust the model’s self-reporting.
However, to verify from the outside, you must be able to write in code “what constitutes completion.” You can write “result.txt contains 391,” but you cannot write “a reply that is not rude to the customer” as is. Whether a harness is effective is determined by whether you can write the completion conditions before you even consider the model.
5. The Fourth and Fifth Layers: Loops and Graphs — Things Not Yet Tried in This Magazine
From here on, I have not yet tried these in this magazine. This is just an organization of the concepts. I cannot say how the success rate will change.
Loops — Repeating the execution itself
The fourth-layer loop involves repeatedly running an execution protected by a harness from the outside. You look at the result of one execution, decide what to pass to the next, and continue until the goal is reached.
The third-layer “completion condition check” was a mechanism for sending things back within a single execution. The fourth layer can be seen as performing that send-back at the unit level of execution. Here, too, the starting point must be whether you can determine from the outside “what constitutes reaching the goal.”
Graphs — Humans decide the path in advance
The fifth-layer graph is a way of writing where humans decide the order of processing and branching as a diagram. The boxes (nodes) in the diagram are individual processes, and the arrows (edges) are where to proceed next. Conditions are attached to branching arrows.
If we turn the previous request into a graph within a single agent, it looks like this.
[計算] → [保存] → [検査] ─ 391 が入っている → [終わり]
│
└ 入っていない → [保存] に戻る(2回まで)
The difference from a round-trip is that the model does not decide “what to do next.” It is decided that calculation comes after saving, and inspection comes after saving. The model is only responsible for the work inside each box. The “completion condition check” that was effective in the third layer was essentially adding an “inspection box and an arrow to return if it fails.” It could be said that the components that worked in the harness were already in the form of a graph.
The graph referred to in the organization of the five layers is a slightly larger concept. Each box becomes an independent agent (a research agent, a writing agent, a verification agent, etc.), and the arrows represent the hand-offs between agents. It is the idea of having the work done as a whole by a combination of agents with divided roles, rather than leaving everything to one smart model.
There are two forms of tools for creating graphs. One is a library like LangGraph, where you write boxes and arrows in code. The other is a tool like Dify, where you assemble them by arranging boxes on a screen and connecting them with arrows. In Dify, this feature is called a “workflow.” In this magazine, I have covered setting up Dify locally and connecting an Ollama model in “Building a Local LLM App with Dify (Environment Setup Edition).” However, I have not yet tried building a workflow and verifying the results.
When you use a graph, you also lose something. Because you decide the path in advance, you need a different diagram for requests you haven’t planned for. This is the opposite of a round-trip, which is “the same code for any request.”
Note that “graph” here refers to a diagram of the processing flow. It is a different topic from data that connects knowledge (knowledge graphs) or GraphRAG, which uses that for search.
6. What I have tried and what I have not tried in this magazine
For each of the five layers, here is a list of what I have tried in this magazine.
・Layer 1 Prompt: Tried. The way of asking eliminated fabrications, but some models answered “not recorded” for everything.
・Layer 2 Context: Tried (RAG). If one procedure was forgotten, unrelated documents were retrieved, and no errors were returned.
・Foundation Round-trip: Tried. Increasing the limit or returning errors did not increase the number of successes.
・Layer 3 Harness: Tried. With components that verify from the outside and send back, successes went from 0-2 to 16-20 (out of 20).
・Layer 4 Loop: Not yet.
・Layer 5 Graph: Not yet (Dify is only at the stage of setting up the environment).
The further out the layer, the narrower the scope left to the model, and the more humans have to write. I believe it is not about which layer is correct, but about choosing how far out to build based on how much you can trust the model.
Small models have noticeable errors, so I quickly saw situations where relying on the model (Layer 1, Layer 2, self-verification) was not enough. With large models or cloud models, there may be more situations where inner layers are sufficient (I have not measured this). Even so, failures of the type “it said it was finished, but the file is missing” can occur regardless of the model size.
7. Summary
・Agent design has expanded outside the model: Prompt → Context → Harness → Loop → Graph. Each layer is a nesting that designs one layer outside the previous one.
・”Loop” has two meanings: round-trips within a single execution, and the repetition of the execution itself. This magazine has measured the former.
・This magazine measured up to the Layer 3 Harness. What worked was not a component that asks “Are you done?”, but a component that verifies the result from the outside and sends it back.
・Layer 4 Loop and Layer 5 Graph have not been measured yet. For graphs, there are ways to write them in code like LangGraph, and ways to assemble them on a screen like Dify. Also, there is a range from ways to fix the path of a single agent to designs that connect multiple agents.
・At any layer, the first thing to decide is “what constitutes success?”. If this is not decided, you cannot write harness inspections, loop endings, or graph branching conditions.
Reference
・Harness, Graph, and Loop Engineering ― How to Evolve From Prompts and Context (Sarthak’s Substack)
https://sarthakai.substack.com/p/harness-graph-and-loop-engineering
・The 5 Layers of AI Engineering (AI Builder Club)
https://www.aibuilderclub.com/blog/five-layers-ai-engineering
・3 Years of Graph Engineering with LangGraph (LangChain Blog)
https://www.langchain.com/blog/3-years-of-graph-engineering-with-langgraph
・Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence (arXiv)
https://arxiv.org/abs/2608.21156
