When using an AI chatbot like ChatGPT, you start with a prompt, and the quality of the output largely depends on how you write the prompt—that is where the problem begins. Multiple users have found themselves spending a lot of time fine-tuning their prompts to achieve the most polished output from AI.
We are probably witnessing the peak of AI era with companies dropping new AI models almost every week. These models use an entirely different approach from their previous iteration and the prompt skills you may have mastered after hundreds of hours of practice are no longer relevant.
For example, in the early days of Stable Diffusion, you had to write “HD, f/01 shutter, high exposure, high quality, Zeiss lens, etc.” Sounds alien? That’s because now, the same or even better output can be achieved by typing “beautiful oil painting of an oak tree” into DALLE-3.
Due to this, crafting the “perfect” prompt has become a headache for users. A Reddit user mentioned that he has probably written a thousand pages in an effort to create a perfect prompt only a few times.
“Some days, everything just works, and the model produces amazing results. On other days, I’m forced to backtrack or even start from scratch. And, of course, there is endless testing. The obsession with crafting that perfect prompt can become all-consuming. I even find myself thinking about prompts in my dreams,” he added, further explaining how he is going through the prompt-writing burnout.
There is No Perfect Prompt
Two ML engineers from VMWare, Rick Battle and Teja Gollapur (now working at Meta), released a research paper on how different prompt-engineering strategies affect an LLM’s ability to solve grade-school maths questions.
They tested three different open-source language models with 60 different prompt combinations each. Specifically, they optimised the system message part of the prompt, which is automatically included in each query before the grade-school math question is posed. What they found was a surprising lack of consistency. Even chain-of-thought prompting sometimes helped and other times hurt performance.
A Reddit user mentioned that there is no universal “perfect prompt”, so spending time optimising prompts for every different architecture and model family does not make sense.
Like so many other skills, in prompt engineering too, you can achieve 80-90% of the results with just about 20-30% of the knowledge. Trying to hyperoptimise the last 5% is often simply not worth it because that’s where 60-70% of the time goes waste.
The fact is, new models with new architectures are unveiled all the time and none of them have the same knowledge base. These also respond differently to the same “perfect prompt”. “Like coding, you could spend 700 hours hyperoptimising our PHP server in 2000, but as the technology progresses, it’ll render it obsolete anyway,” he added.
This is why there are discussions going on about the future and relevance of prompt engineers. Some believe that a prompt engineer is just someone who throws a bunch of stuff against the wall to see what sticks. Besides, each AI works differently and the worst part is that different sizes of the models work differently.
Another Reddit user said that a prompt engineer is not someone who can say, “I’ve formed a strategy for prompts that always works and here are some basic rules”. As the AI will be slightly different each time, even between version releases.
“I guess it bothers me because, really, it’s just attempting to find the *random* little thing that will get the AI to do what you want. How can you call it engineering when it’s non-deterministic? Engineering is supposed to be rigorous. People who say, ‘I’m a prompt engineer’, think they have found some secret sauce, that they know how to talk to any AI and get it to do what they want – that’s just not true. It’s much more random than that. You are literally just throwing random keywords at the AI until it works. That’s not engineering,” he added.
This results in AI Fatigue
Things are taking a different turn. When AI arrived, users were relieved that AI would help them deal with burnout. However, according to a recent study from Quantum Workspace, employees who use AI are more likely to encounter burnout than those who don’t.
The report was based on one of the largest databases on employee experience in the US, including over 700,000 voices across more than 8,000 organisations. This might be because of the prompt-writing burnout as employees were likely spending more time refining prompts to achieve the desired results.
The solution is to use the right tools at the right time and for the right purpose. People abused OpenAI o1 models for their poor coding skills, but they missed the key part – those models were not supposed to do so. Similarly, being solely dependent on AI tools will obviously deteriorate your work, and as time goes by, you might lose the ability to judge the output given by an LLM.
A good escape from prompt fatigue could be by using AI tools to create prompts. You can also use techniques like reflection for better results but we suggest the time is yet to come when you will be able to get the desired output in one shot.
