Leveraging AI through work design, not prompt engineering｜三分一技術士事務所

When using generative AI for programming and system development, one tends to think about “what kind of prompt should I write to get the correct answer on the first try?” However, as I have continued to use ChatGPT and Codex in actual development projects like UADC, XBRL GL Next, and LedgerExplorer, my way of thinking has changed significantly.

The important thing is not writing clever prompts. It is designing a “work mechanism” where even if the AI makes a mistake, it does not lead to an accident, and only correct results can be adopted as official deliverables.

In this article, I will organize how to divide tasks between humans and AI based on the problems that occurred in actual development and the mechanisms I have developed as countermeasures, such as specifications, AGENTS.md, WORK, canonical, and Official
GIT. Furthermore, I will consider the problem that has emerged beyond that: “How do we train the next generation of engineers in an era where AI handles even the grunt work?”

I also explain this in a video.

I have summarized the concepts introduced in this article in a 10-minute video “Delegation to AI—Transitioning to Management”.

In development using ChatGPT and Codex, it is not only important to write detailed instructions for the AI. It is important to create a mechanism where humans are likened to a “manager,” the web version of ChatGPT to a “supervisor,” and Codex to the “field staff,” and while dividing their respective roles, work is delegated to the AI, and humans evaluate and judge the results.

In the video, starting from this role division of “manager, supervisor, and field staff,” I introduce work rules using specifications and AGENTS.md, the management of work areas and formal promotion from task-local → canonical → Official GIT → GitHub, and the analysis and specification of legacy code using AI.

Furthermore, I touch upon the issue of how to train the next generation of engineers when AI begins to take on traditional “grunt work” such as coding, testing, and documentation.
If you want to grasp the overall picture of this article first, please watch the video.

AI Delegation—Transitioning to Management
From prompt engineering to AI delegation and management.

Approx. 10 minutes | Explaining AI development governance, specifications, AGENTS.md, Generate / Promote, legacy code analysis, and human resource development

1. Writing prompts is no longer the job

When you start using generative AI for programming, you tend to think about “what kind of prompt should I write to get the correct code on the first try?” However, the more complex the practical work becomes, the more limited the method of having a human write detailed work procedures from beginning to end becomes. It is very similar to micromanagement, where you sit next to a new hire and explain how to do the work from A to Z.

What humans should really be thinking about are the purpose and basic conditions of the work, such as “what to do,” “why do it,” “what must not be changed,” and “what constitutes a passing grade.” Therefore, what I am currently testing is a role division where humans are considered the “manager,” the web version of ChatGPT the “supervisor,” and Codex the “person in charge.”.

Humans determine the purpose, basic policy, invariant conditions, and acceptance criteria. The web version of ChatGPT develops this into specific work instructions that Codex can execute, based on the project’s basic design. Codex investigates and revises actual files, performs tests, and creates evidence such as logs and SHA-256. The results are passed back to the web version of ChatGPT to be organized and reviewed against the initial instructions. Finally, it is the human who decides whether to officially adopt the deliverables.

Figure 1 Role division between human, web version of ChatGPT, and Codex
The human (manager) determines the purpose, invariant conditions, and acceptance criteria, the web version of ChatGPT (supervisor) develops them into specific work instructions, and Codex (field staff) performs the actual work and testing. The results are organized and reviewed again by the web version of ChatGPT, and the final decision on adoption is made by the human. This cycle is repeated as necessary.

UADC and XBRL GL Next have project-specific concepts such as FSM, BSM, LHM, HMD, Semantic
Path, and Binding. Instead of a human explaining all of these every time and writing detailed instructions for Codex, I convey “what I want to do this time,” “what must not be changed,” and “what state constitutes a passing grade” to the web version of ChatGPT, which has understood these concepts through dialogues over the past few years, and have it develop them into specific work instructions.

The changes, test results, SHA-256, and Git status returned from Codex are also returned to the web version of ChatGPT to be reviewed against the initial instructions. If a problem is found at this stage, additional work instructions are created and sent back to Codex. What I feel when using it this way is that rather than just giving orders to an AI, it is the feeling of “working with a capable supervisor who understands the basic design of the work.”

2. In the AI era, specifications have become important again

What was surprising about this way of working was the importance of program specifications. Previously, there was a problem that even if detailed specifications were created, they needed to be updated every time the program was changed, and eventually, they would no longer match the implementation. It is understandable that the idea of “there is no point in creating detailed specifications if they are going to deviate from the implementation anyway” would emerge.

However, AI is adept not only at writing code but also at reading and documenting it. By comparing existing specifications with implementations, it can examine functions, inputs/outputs, internal data structures, and Semantic Paths to create draft revisions of specifications. Instead of rewriting specifications from scratch, humans can shift to the role of reviewing the specifications organized by the AI to confirm whether these specifications are truly correct.

Furthermore, this method is not limited to new development. For legacy programs that have been revised over many years, changed hands, and lost their original specifications, having an AI analyze the code can extract inputs/outputs, relationships between functions, decision conditions, and internal data structures, allowing for the reconstruction of specifications that explain the current implementation.

However, there is an important principle here: you must not assume that just because something is implemented, it is the correct specification.

Old code may contain errors, or past provisional processing may still remain. It is necessary to distinguish between the fact that ‘this is how it currently works,’ which the AI discovers from the code, and the specification of ‘how it should ideally be.’

Figure 2: Relationship between Specifications, Implementation, and Verification. Starting from basic specifications and data models, connect to LHM/HMD/Semantic Path, Binding, implementation, and testing/verification. While AI can support implementation and documentation, it is necessary to distinguish between ‘implementation as it currently exists’ and ‘specifications as they should be,’ with humans making the final decision on adoption.

In the case of UADC, we do not judge correctness simply by looking at Python programs. There is a basic data model, logical hierarchies, and Semantic
Path, as well as Bindings that define the correspondence with external interfaces, and the conversion program runs based on these. Then, the generated data is tested, and reverse conversion or regression testing is performed as needed.

Thinking of it this way, specifications are no longer static documents to be stored after completion. They become a common foundation for humans and AI, or multiple AIs, to take over the same work, and at the same time, they act as a control mechanism to prevent AI from arbitrarily changing programs based solely on common programming knowledge. I believe that in the AI era, program specifications have breathed new life as ‘living documents.’

3. Passing tests does not necessarily mean it is an official deliverable

There was an event during development with AI that left a particular impression on me. During the revision work for LedgerExplorer, the functional tests themselves passed. However, when I investigated the work content, I found that it was using the area for official reflection directly instead of the WORK area where the work should have been done.

In other words, it was a state of ‘What was created is correct. However, the way it was created is wrong.’

This was an important experience. In normal program development, we emphasize ‘whether it passed the tests.’ However, when entrusting work to AI, functional correctness alone is not enough. It is also necessary to verify the work process itself, such as ‘which files were used,’ ‘where changes were made,’ ‘what was judged as the official version,’ and ‘what path was used to move it to the public area.’

Therefore, we now clearly separate the boundaries between the work area and official deliverables. In WORK, there is a task-local area where the AI performs investigation, experimentation, and temporary script creation, and a canonical area where humans place the adopted official definitions. Only deliverables accepted in canonical are copied to the necessary locations, and after confirming consistency using SHA-256 or similar, they are reflected in the Official GIT. Afterward, they are published to GitHub through explicit commit/push.

Figure 3: AI Development Work Areas

WORK
│
├─ task-local
│   ├─ 調査・実験
│   ├─ 一時スクリプト
│   └─ テスト用成果物
│
└─ canonical
   â””─ 人間が採用した正規定義
          │
          │ COPY
          │ SHA-256照合
          │ 人間による確認
          ▼
     Official GIT
          │
          │ commit / push
          ▼
       GitHub

We codify these work rules in `AGENTS.md`. These are not abstract instructions like ‘please work safely.’ We define them as observable behavioral rules such as where it is okay to write, where it is forbidden to write, what the conditions for deletion are, where to place test scripts, and what the conditions for moving to Official GIT are.

The important thing here is not to limit the AI’s capabilities. We let the AI fully investigate, experiment, implement, and test in a safe WORK area. Instead, we clearly separate ‘creating’ from ‘adopting as official.’

4. Separate Generate (authority to create) and Promote (authority to officially adopt)

In UADC development, another interesting problem arose. To improve work efficiency, AI sometimes creates small auxiliary scripts on its own. It is rational to create ad-hoc tools to compare CSVs, read Excel files, or check large amounts of results.

The problem is that as those convenient scripts are used repeatedly, they start to be treated as if they were official programs before anyone realizes it. Therefore, we now clearly state the following principle:

Usage does not create authority.
The fact that it is being used is not a basis for legitimacy.

Even if it is used 100 times and passes all tests, that alone does not make it an official specification.

What is needed here is the separation of Generate (authority to create) and Promote (authority to officially adopt). We have the AI create the necessary tools. We have it create temporary auxiliary scripts, test data, and verification tools. However, these are placed in the task-local work area.

On the other hand, whether to include it in the canonical as an official implementation is a separate decision. Humans decide whether to adopt it by confirming whether it is necessary according to specifications, whether it overlaps with existing official tools, and whether it is something that should be maintained in the long term.

Figure 4: Generate and Promote

AI
│
│ Generate
▼
task-local
実験・補助スクリプト・一時成果物
│
│ 自動昇格しない
▼
人間によるレビュー・承認
│
│ Promote
▼
canonical
正規仕様・正規ツール

By expressing this boundary as a directory structure rather than just a note, you can distinguish between “used because it was convenient” and “adopted as an official specification.” We do not expect AI to never make mistakes.We create a structure where the official specification does not change on its own even if it makes a mistake.

I believe this is far more important in actual practice.

5. In an era where AI does the “grunt work,” who will train the leads?

Once you have established a system like this, development becomes extremely efficient. Writing code, researching existing code, debugging, testing, organizing logs, and writing specifications. AI can perform a significant portion of the work that young engineers have spent years gaining experience in over a short period of time.

This is a major step forward. However, it also creates another problem.

So, where will the “leads” of the future be trained?

Traditionally, young engineers have repeatedly performed tedious tasks such as basic coding, debugging, deciphering other people’s code, and creating specifications. By gaining experience in failing, investigating the causes, and making corrections there, they come to understand the structure of the system and eventually become able to look at other people’s deliverables and judge, “This is wrong.”

However, when AI starts to take over a significant portion of that “grunt work,” it becomes necessary to intentionally acquire the judgment experience that was previously gained naturally through work in a different way. This does not mean that basic training in writing code becomes unnecessary. In addition to that, I believe that training to evaluate deliverables created by AI, discover errors, and have the AI redo the work if necessary will become important.

I think there are four main directions for that education.

Auditing deliverables created by AI

The first is to use the code, specifications, test results, logs, etc., created by AI as teaching materials. Instead of just checking “if the program ran,” have them investigate “if it matches the specifications,” “if it has broken parts that should not be changed,” “if old definitions are mixed in,” and “if it can really be called a pass based on these test results alone.”

And what is important is not just having them guess the correct answer.

It is to have them explain the rationale, such as “why they judged it to be problem-free” or “why they judged it to be unacceptable.”

I think this will be a way to train the “eye for spotting oddities” that has been acquired through traditional debugging in a different form in the AI era.

Designing work methods, not prompts

The second is not to have them write “good instructions for AI,” but to have them design the work itself.

What is the purpose? What must not be changed? What constitutes a pass? At what condition should the work be stopped? How much operation should be allowed for the AI?

Have them define these conditions in specifications or AGENTS.md.

For example, have the junior staff themselves think of rules, such as prohibiting direct writes to anything other than WORK, requiring explicit approval for deletions, and not automatically promoting auxiliary scripts created in task-local to canonical.

This is less about prompt engineering and more of an exercise in designing a ‘workplace’ where AI can work safely.

Verifying the path to the result, not just the PASS result

The third point is training to audit not only the deliverables but also the path through which those deliverables were created.

In actual development, there have been cases where, despite functional tests passing, the changes were made directly to the area for official release rather than the WORK area where the work should have been done.

In this case, the program works correctly if you look at it in isolation. However, it is impossible to track ‘what it was based on, where the work was done, what verification it went through, and which deliverable was moved to the official version’.

Therefore, have junior staff check not only the test results but also the work location, changed files, logs, Git status, and, if necessary, evidence such as SHA-256 hashes.

Instead of ‘it’s correct because it works,’ they need to acquire a mindset of quality where ‘it is accepted because I can explain that it was created using the correct procedures’.

Re-questioning AI’s ‘next move’

The fourth point is training to re-question AI’s proposals rather than executing them as is.

AI may propose additional checks or re-testing depending on the interpretation of the provided information or instructions. However, that is not necessarily required for the current change.

Therefore, the mentor and the junior staff read the AI’s work report together and re-question it: ‘Why is this step necessary?’, ‘Is it related to the part changed this time?’, ‘Can we reuse already confirmed results?’, and ‘What would become unprovable if this work were not performed?’

The goal is not to doubt the AI.

The goal is to acquire the habit of evaluating AI proposals against design objectives and evidence.

From ‘experience in creating’ to ‘experience in creating, evaluating, and designing’

Human resource development in the AI era cannot be solved by simply being pessimistic that ‘junior staff won’t grow because the grunt work is disappearing.’ On the other hand, it is not a matter of ‘junior staff only needing to learn management from the start because AI does everything’.

The experience of understanding the underlying code and data structures will continue to be necessary. On top of that, it is necessary to intentionally incorporate into education the experience of reviewing AI-created deliverables, auditing work paths, designing rules for AI to work safely, and re-questioning AI proposals based on their rationale.

Conventionally, the flow was:

Create → Fail → Correct → Understand → Become able to evaluate

.

In the AI era, we will likely be adding a new cycle of experience:

Have AI create → Evaluate → Re-question → Improve rules → Reflect in the next AI task

to that process.

And this brings us back to the “Manager, Lead, Field” model presented at the beginning of this article.

Just because you have an excellent lead doesn’t mean the next lead will grow naturally.

Precisely because AI is becoming a “capable lead,” humans need to redesign the very method of cultivating the next lead.

I feel this is the biggest challenge right now while continuing to use AI in actual development sites.

Summary: From Prompt Engineering to “Work Design”

Having continued development using AI, this is where my thinking has changed the most. The core of AI utilization is not “what kind of prompt should I write to get AI to give the correct answer.”

AI makes mistakes. It sometimes makes mistakes in the work area. It may continue to use tools it created itself because they are convenient. However, there is no need to end the conversation by saying “AI cannot be trusted.” What is important is to create a work mechanism where, even if AI makes a mistake, it does not lead to an accident, and only the correct results can be adopted as official.

Humans decide the purpose and basic design, AI expands it into specific tasks, AI performs the actual work, and the results are reviewed at another stage. And humans decide whether to formally adopt it. This may not be a new idea unique to AI, but rather the very basics of engineering and organizational management that have been practiced for a long time.

The difference is that a new member has been added who is extremely fast and can handle a lot of work, but cannot be expected to have the same “tacit understanding” as humans. While AI requires clear work rules like a new employee, if given appropriate basic design and constraints, it can even take on the role of an excellent lead.

That is why what will become important from now on is not just the skill of writing prompts.

“In what kind of workplace should AI work, how much authority should it be given, how should its results be evaluated, and what should be adopted as official results?”—I believe it is the ability to design the entire work process.

Reference: Projects covered in this article

• XBRL GL Next
  Consideration and implementation of a standard semantic model and taxonomy for accounting transaction data.

• UADC-PoC
  Universal Adapter for Data Conversion
  PoC. The goal is to convert data from proprietary interfaces into standardized Structured CSV based on externalized conversion definitions, connecting different data representations.

• LedgerExplorer
  A visualization and exploration environment for referencing and analyzing accounting data from multiple perspectives using Structured CSV.

This article is organized based on the experience of actually using ChatGPT and Codex in these projects and going through trial and error. The draft was created by NotebookLM, revised by the web version of ChatGPT, and finalized by Sanbuichi Professional Engineer Office.

Originally Appeared Here