the new reality
MidJourney
A new career is emerging with the spread of generative AI applications like ChatGPT: prompt engineering, the art (not science) of crafting effective instructions for AI models.
“In ten years, half of the world’s jobs will be in prompt engineering,” declared Robin Li, cofounder and CEO of Chinese AI giant, Baidu. “And those who cannot write prompts will be obsolete.”
That may be a bit of big tech hyperbole, but there’s no doubt that prompt engineers will become the wizards of the AI world, coaxing and guiding AI models into generating content that is not only relevant but also coherent and consistent with the desired output.
So, what exactly is prompt engineering?
Getting generative AI to do what you want is no easy task, as anyone who has tried image generation systems like Dalle-E or MidJourney or language models like ChatGPT. While successful creations are dazzling, an untrained user’s results are likely to be deeply flawed or, with ChatGPT, even wrong. The same is true for AI code-writing generators.
“In ten years, half of the world’s jobs will be in prompt engineering,” declared Robin Li, cofounder and CEO of Chinese AI giant, Baidu. “And those who cannot write prompts will be obsolete.”
That’s because generative AI models respond to natural language and natural language is notoriously imprecise. The same sentence can have different meanings depending on the context, making it difficult for the AI model to understand what the user wants it to generate. And natural language prompts may not provide enough context for the AI to understand the user’s intent fully. This can lead to the AI generating responses that are not relevant to the user’s needs or expectations.
Also, generative AI models are typically trained on large amounts of text data, but the training data may not contain examples that match the specific intent of the user’s prompt. This can limit the AI’s ability to generate responses that accurately reflect the user’s needs.
And, finally, generative AI models are trained to generate responses based on patterns in the training data, so they may not be able to generate responses that are truly creative or innovative.
So, talking to a generative AI model is a bit like speaking to an idiot savant – you need to understand what they respond to if you hope to get the results you want.
Already, prompt engineering experts are popping up all over the place, startups are offering prompt engineering services and companies are starting to list ‘prompt engineer’ as a job title. Trainers and educators are fanning out to help industries train workers on how best to use the generative AI, and video lecture sites like Udemy are already offering many courses on formulating effective prompts. Whether or not the importance of the job grows to meet Robin Li’s expectations, it is likely here to stay for a while.
Meanwhile, with the increasing sophistication of AI algorithms, it is becoming more feasible for AI systems to take over the role of coding, leaving software engineers to focus on higher-level tasks such as formulating intent and logical sequences to guide the code generator. This shift will likely require software engineers to have a more profound understanding of AI algorithms and how they operate. As a result, they will be able to craft prompts that can direct the AI to generate code that meets the desired specifications accurately.
The role of software engineers will evolve into one of guiding and overseeing the AI’s work, providing input and feedback, and ensuring that the generated code meets the project’s requirements.
Prompt engineering will be critical in using automated code generators as prompts must be carefully crafted to accurately capture the intent of the desired code. Additionally, prompt engineering can help ensure that the generated code aligns with industry best practices, standards, and guidelines.
Already, we are seeing the compilation of prompt libraries, like libraries of pre-written code or software components that can be reused in different programs or applications. Just as a code library contains components designed to be reusable, allowing developers to save time rather than creating new code from scratch, prompt libraries will do the same.
There are also prompt specializations emerging for code generation, output testing, text generation, and art generation. Ultimately, prompt engineering is all about knowing what to communicate to an AI model to produce the desired output, empowering users to optimize communication for accurate outputs.
There is a flood of startups and new tools to help engineer prompts, including PromptPerfect and PromptingGuide. Online schools are beginning to offer courses in prompting and PromptBase is an online marketplace for buying and selling high-quality prompts. The list goes on and on and will undoubtedly grow.
Startup Anthropic, whose mission is to create reliable, interpretable, and steerable AI systems, is offering a salary of up to $335,000 for an experienced prompt engineer. The position involves figuring out the best methods to prompt AI for various tasks, documenting these methods, building a library of tools, and creating tutorials for others to learn prompt engineering.
Ultimately, prompt engineering can help improve the usability and reliability of automated code generators, making them more accessible to users who may not have a strong background in programming. This emerging class of skilled human operators who know how to interact with AI models effectively will bridge the gap between the human and the AI worlds.
