“AI is just probabilistically predicting the next word.”
You have likely heard this explanation of generative AI like ChatGPT at least once.
Certainly, this is an important point in explaining the basic mechanism of generative AI.
AI does not think with consciousness like a human. Based on the input text or conversation context, it calculates the probability distribution of the next token (such as a word or part of a character) to output and selects the output from there.
Thinking about it that way, one question arises.
If it is just choosing the next word probabilistically, is there any point in refining the prompts we input?
In conclusion, there is a great deal of point.
In fact, it is precisely because AI generates text probabilistically that prompt engineering is important.
This time, I will consider the reasons for that.
1. AI infers “what it should answer” from the prompt
First, let’s briefly organize how AI works.
For example, suppose you input the following two prompts.
Prompt A
“Tell me about marketing.”
Prompt B
“You are a marketing manager for a small business. Please propose five measures for a restaurant with a limited advertising budget to increase new local customers. Explain the cost, expected effect, and implementation procedure for each specifically.”
Both are asking about marketing.
However, the answers generated by the AI will be significantly different.
Why is that?
It is because the direction of the text the AI predicts changes depending on the prompt.
Prompt A contains almost no specifications regarding the target audience, purpose, prerequisites, or response format.
As a result, the AI attempts to generate a general marketing explanation from a wide range of possibilities.
On the other hand, Prompt B includes the following information:
-
Role: Marketing manager for a small business
-
Target: Restaurants with limited advertising budgets
-
Objective: Increase new local customers
-
Condition: Propose 5 measures
-
Output format: Explain costs, effects, and implementation steps
This information narrows down the direction of the AI’s response.
In other words, a prompt is not just a question.
It is information used to influence the probability distribution of the text the AI should generate.
2. Because it chooses probabilistically, the initial condition setting is crucial
This is the main point of this discussion.
AI chooses the next token to output probabilistically.
For example, let’s assume that in a certain context, there is the following probability distribution:
-
Word A: 50%
-
Word B: 30%
-
Word C: 15%
-
Others: 5%
While this is a simplified example for explanation purposes, the AI generates text by assigning probabilities to multiple candidates in this way.
So, what happens when a new condition is added here?
For example, suppose you add the instruction, “Please explain this so that even an elementary school student can understand, rather than for an expert.”
Then, the context that the AI refers to changes.
Simple expressions become more likely to be chosen than difficult technical terms.
In other words, the probability distribution of the next word to be chosen changes depending on the prompt.
The important point here is that the fact that an AI generates text probabilistically is different from the output being completely random.
The probabilities depend on the given context.
Even with the same AI, if the input information changes, the tendency of the generated text also changes.
That is precisely why there is meaning in refining the input information.
3. A prompt is a “map” for an AI
To understand the role of a prompt, let’s compare it to a map.
Suppose you ask someone to “guide me somewhere” without a destination in mind.
The person guiding you will not know where to head.
They might just guide you to a place where many people tend to go.
On the other hand,
“Please introduce three cafes where I can work quietly within walking distance from Osaka Station.”
If you ask this, the direction of the guidance becomes clear.
AI is the same.
If there are no conditions, it tends to give a general and safe answer.
If the purpose and conditions are specific, it becomes easier to generate an answer that follows them.
Prompts do not directly teach the AI the correct answer.
They indicate the direction the AI should head and the path it should follow.
4. Refining prompts also increases the reproducibility of answers
Another important benefit of prompts is that they can increase the reproducibility of answers.
For example, suppose you ask the following every time:
“Create a project proposal.”
With this, the structure, level of detail, and perspective of the answer may change each time.
Therefore, you specify conditions in advance, such as the following:
-
Write the conclusion first
-
Next, organize the current issues
-
Present three concrete solutions
-
Include expected effects and implementation costs for each measure
-
Finally, show an execution schedule
-
Avoid using technical jargon as much as possible
By doing this, it becomes easier to create project proposals that follow a consistent format every time.
Of course, even if you use the same prompt, the answer will not necessarily be identical due to the mechanisms and settings of generative AI.
However, you can stabilize the direction and structure of the answers.
This is significant not only when using AI individually, but also when introducing AI into business operations for companies and teams.
5. A good prompt is not a long prompt
Reading this far, you might think, “I should just write the most detailed, longest prompt possible.”
However, that is slightly incorrect.
What matters is not the length of the text, but whether the necessary information is conveyed to the AI appropriately.
For example, let’s compare the following two.
Bad example
You are the world’s best marketing expert, possessing all knowledge and experience. From the best perspective, please come up with wonderful, groundbreaking, and innovative marketing strategies.
Good example
Please propose strategies for a local restaurant with monthly sales of 3 million yen to increase new customers by 20% within 3 months. The additional budget is within 50,000 yen per month, and please list 5 strategies in order of low execution difficulty.
The latter makes it clearer to the AI what it should answer.
Rather than increasing grandiose role settings or abstract adjectives, it is more effective to specifically convey the purpose, constraints, target, and expected results.
Refining prompts is not about memorizing special words like incantations.
It is about organizing what you are looking for into a form that the other party can understand.
6. Refining instructions to AI is also about organizing human thinking
Actually, the biggest benefit of thinking about prompts is not just that the AI’s answers improve.
It also lies in organizing your own thoughts.
For example, suppose you are about to ask an AI to ‘come up with a new business idea’.
At that moment, you pause and think for a bit.
-
Who is the business for?
-
What kind of problem do you want to solve?
-
What kind of market are you envisioning?
-
What is the available budget?
-
What defines success?
Thinking about these things helps clarify what you truly want to achieve.
In other words, to write good prompts, you need to understand your own goals and challenges.
The act of giving instructions to an AI is also the act of articulating your own thoughts.
I believe this is a skill that will become increasingly important in the age of AI adoption.
7. The difference between those who can master AI and those who cannot
From now on, the era where a significant gap is created simply by whether or not you can use AI will likely come to an end.
Once many people are able to use generative AI like ChatGPT, the mere ability to input questions will no longer be a special skill.
The difference will then emerge in what you ask of the AI, what information you provide, and how you evaluate and improve the returned answers.
Even when using the same AI,
“Think of some good ideas”
for someone who inputs this,
“Compare and evaluate highly feasible ideas within this market, this customer base, this budget, and these constraints”
the results obtained will differ for someone who inputs this.
Furthermore, it is important not to take the AI’s answers at face value, but to verify the facts and ask additional questions as needed.
Writing a prompt is not a one-time task.
Look at the answer, add missing information, adjust the conditions, and interact repeatedly.
Through this repetition, you will be able to draw out the AI’s capabilities more effectively.
Summary: Precisely because AI chooses words based on probability, humans must provide direction.
AI generates text by probabilistically predicting the next token.
However, that does not mean it is just stringing words together randomly.
It calculates what it should output next based on the input prompt and the context of the conversation so far.
That is precisely why the text generated by the AI changes when the prompt changes.
To summarize the key points of this article:
-
Prompts create the context for the AI to generate its response.
-
When the context changes, the probability distribution of the next token to be selected also changes.
-
By clearly communicating your goals and conditions, you can narrow down the direction of the response.
-
By specifying output formats and constraints, you can increase the reproducibility of the response.
-
Thinking about good prompts also leads to organizing your own thoughts and objectives.
Ultimately, refining prompts is not about casting some special magic on the AI.
It is a communication technique to draw out the vast knowledge and language generation capabilities that the AI possesses to suit your own purposes.
If the AI chooses the next word probabilistically, then humans should design the context that influences those probabilities.
This is the reason why prompt engineering is necessary and effective.
The first step to mastering AI is not learning difficult techniques.
It is about being clear about “what you want it to do,” “why you need it,” and “what kind of result you want.”
I believe that those small efforts will significantly change the results you get from AI.
Written by Michael
If you found this article helpful, please let me know by clicking the “Like” button✨
I am also waiting for your thoughts and requests for more information in the comments.
Actual implementation examples and detailed service information are summarized here.
Feel free to reach out if you would like to consult with us.
▶︎ https://innate-ss.co.jp/
