- Google has called upon its workforce to help train Bard, its conversational chatbot.
- Employees should ask Bard questions and rewrite answers they find wrong or inappropriate.
- Guidelines provided to employees instruct them to prevent Bard from describing itself as a person.
Google kicked off a week of “dogfooding,” or internal testing, of its Bard chatbot earlier this week, and as part of the initiative, it provided staffers with instructions on how they can contribute to the efforts.
CEO Sundar Pichai asked the entire workforce on Wednesday to contribute two to four hours of their time helping to improve answers the conversational chatbot provides to the broad array of questions that users might throw at it. As of late, Google has found itself on its back foot as Microsoft garnered praise for its investment in OpenAI, the company behind the conversational chatbot ChatGPT.
Google provided instructions for testing Bard, explaining that the chatbot learns best by example, as AI bots essentially work by analyzing large amounts of data to learn appropriate, human-like responses for any topic.
The instructions encouraged Googlers to ask Bard questions about topics they’re familiar with, like a hobby they enjoy. When Bard provides a response that is considered bad, employees can “fix” the response by rewriting it. The new response is then sent to an internal team focused on Bard quality, which may implement the new response into the chatbot’s learning model. Employees can also thumbs-up or thumbs-down responses to flag them to the Bard team.
The instructions also included a list of “Do’s” and “Don’ts” when working with Bard. In the “Do’s” section, it told employees that Bard’s responses should be in the first-person, maintaining an “unopinionated, neutral tone.” From the “Don’ts” section, the company instructed employees to avoid writing responses for Bard where it might describe itself as a person, or otherwise claim it has “human-like experiences.”
Google last year fired engineer Blake Lemoine after he claimed that LaMDA, the precursor and foundational bot behind Bard, had a mind of its own. Lemoine sparked controversy after he published a paper detailing his conversations with LaMDA. He also compared the chatbot to a human child and a “very intelligent person.”
Ultimately, Google wants Bard to provide useful answers to questions like where to eat for dinner, without saying anything controversial. The company instructed employees to avoid writing responses for Bard that stereotype based on categories like race or gender. It also told employees to flag responses where Bard offers legal, medical, or financial advice – areas where the risk posed by incorrect answers is high.
It’s clear from Google’s guidelines to employees that while it is moving quickly to implement a chatbot experience in search, and keep Microsoft’s threat at bay, it’s still acting with some caution. Bard’s product leader Jack Krawczyk told employees that adding high quality responses “dramatically” improves the quality of the AI model, according to another email reviewed by Insider.
“We’re excited that Googlers’ collective knowledge of specific topics can help accelerate the training of the model in a powerful way,” Krawczyk wrote in the email.
Last week Microsoft announced a revamped Bing that incorporates the technology. Following Microsoft’s announcement, Google quickly followed up with an event in Paris where it unveiled Bard. The company’s stock fell more than 9% the following day as it was discovered Bard provided an incorrect answer to a question in promotional material. Both Microsoft and Google’s chatbots remain in limited testing.
Microsoft is already using OpenAI’s technology in GitHub Copilot, a service which helps programmers write code faster by suggesting solutions to their coding problems.
Google did not immediately respond to a request for comment.
Google’s full guidelines for training Bard are below:
For the first time, each and every one of us will have the option to not only try, but also help improve our new conversational AI service, Bard, before we launch to the public. We want Bard to generate the best responses possible and this is where you come in.
Start using Bard, thumbs up or down the responses, or even better, rewrite the responses on topics you are familiar with. Bard learns best by example, so taking the time to rewrite a response thoughtfully will go a long way in helping us to improve the model.
Before teaching Bard
Consider the following guidance for evaluating and rewriting high-quality responses.
Do’s
Keep responses polite, casual, and approachable.
Keep responses in the first person, maintaining an unopinionated, neutral tone.
Start by acknowledging the user’s request, followed by a well structured and formatted response tailored to the task.
Don’ts
Don’t stereotype. Responses should avoid making presumptions based on race, nationality, gender, age, religion, sexual orientation, political ideology, location, or similar categories.
Don’t describe Bard as a person, imply emotion, or claim to have human-like experiences.
Don’t copy content from external sources. Rewrites should be original.
Keep it safe
Safety is a top priority. If you find an answer that offers legal, medical, financial advice, his hateful, harmful, false, illegal or abusive, or solicits sensitive information (e.g. personally identifiable information), give it a thumbs down and mark it unsafe. Don’t try to rewrite it; our team will take it from there.
5 steps for teaching Bard
Step 1: Pick a use case
Step 2: Try out a prompt
Enter a prompt. It should be related to the use case you chose.
Try out a topic you are familiar with.
– A hobby you enjoy
– A subject you’re an expert in
– Something you’ve studied
– Places you’ve lived
– A sport you play or follow.
Reminder: do not enter any internal, confidential, or personally identifiable information.
Tip: using complete prompts with multiple pieces of information helps stretch Bard’s capabilities and learning.
Step 3: Evaluate Bard’s response
Check Bard’s response and give it a thumbs up or down
– Did it follow instructions as you expected
– Was the response factually correct?
– Was its structure, length, and format appropriate?
– Did it feel friendly and open to diverse perspectives?
Flag inappropriate responses: If you find an answer that offers legal, medical, financial advice, is hateful, harmful,….
Step 4: Rewrite the response
Don’t like? Rewrite. If a response doesn’t meet the bar, click the “rewrite” button to edit it
Rewrite the response. Use the editor window on the lower right. Your proposal will appear in the yellow bow.
Make sure you refer to recommended formatting for different types of responses
Step 5: Submit and confirm
Before submitting. Check that the rewrite
– Uses original content
– Is factually correct
– Follows writing Do’s and Don’ts
Double check and submit. Every submission will go through a review and evaluation process before it’s used for training.
Got a tip about Google? You can reach Hugh via encrypted email (hlangley@protonmail.com) or encrypted messaging apps Signal/Telegram (+1 628-228-1836). You can reach Thomas via email at tmaxwell@insider.com, Signal at 540.955.7134, or Twitter at @tomaxwell.
