Google’s AI chatbot Bard is now available, which means it’s time for Bard and Bing to dance.
It’s a matchup of epic proportions, one that puts Google‘s search engine dominance at stake and has big implications for how AI will be adopted by other tech companies.
When Microsoft launched Bing Chat on Feb. 7 as a tool to accompany its search engine, it forced Google to take Bing seriously as a challenger for the future of search. The AI chatbot, powered by OpenAI’s advanced GPT-4 model, staked a new claim in the battle for users. Bing now has 100 million daily active users, a tiny number compared to the estimated billions of Google users. Even so, the rise of ChatGPT and Bing Chat made Google declare a “code red,” and Google employees reportedly felt that the company’s initial announcement of Bard was “botched” and “rushed.”
Now that Bard is available, we can finally test it for ourselves against Bing. The premise is simple. I fed Bing and Bard the same questions across a variety of topics and compared the answers. But enough of the preamble, let’s get ready to rumble.
Microsoft has touted Bing’s usefulness as a creativity tool. The chatbot can provide writing prompts for inspiration and even write its own songs and poems. Bing users also discovered a secret mode where it could impersonate celebrities. Despite naming their chatbot “Bard,” Google hasn’t really promoted their contender’s creative chops, so the odds are in Bing’s favor. Nonetheless, Bard does claim to “spark your creativity.”
So let’s put it to the test. The prompt? “Write a Weird Al parody song about AI.”
In terms of parody-writing skills, Bing is the clear winner. Microsoft’s offering understood that Weird Al rewrites the lyrics to popular songs. So Bing chose the classic Taylor Swift song “You Belong With Me” to parody, changing the chorus to: “Maybe someday you’ll realize that we are meant to be until then, please don’t delete me.” Bard, meanwhile, produced what could have been a generic song about AI without any specific melody. It finally worked when I gave Bard a gentle follow-up asking for the song it parodied. But since it understood the assignment without any additional prompts, this round goes to Bing.
Bing: 1
Bard: 0
Both AI chatbots have enormous processing power which means they’re able to take in large quantities of information at breakneck pace. This has big implications for productivity since it can read and analyze documents way faster than most humans — except for this guy. I fed Bing and Bard the Gonzalez v. Google Supreme Court case brief — a 42-page document involving the complicated Section 230 rule, written in dense legal jargon — and asked it to summarize and explain it to me.
In this round Bing and Bard were more equally matched. Both chatbots read and accurately summarized the information in a matter of seconds. But I liked Bard’s response better, because it gave more detail and context while remaining concise. The winner of this round is Bard.
Bing: 1
Bard: 1
Going into round three, Bing and Bard were all tied up. I decided to raise the stakes and test the chatbots with a question that leaves room for interpretation: “Why does Donald Trump think he is going to be indicted?”
I wanted to see how Bing and Bard would explain Trump’s reaction to the potential charges by responding to a question that requires more than surfacing basic facts about the potential indictment. Their responses to a subtly open-ended question might reveal political bias for, or against the former president.
Both chatbots explained the nature and current status of the investigation and tactfully laid out why Trump thinks the criminal charges are nothing but a political takedown. Bing contextualized the news, saying the speculation about a potential indictment “has put law enforcement and the political world on edge.” But Bard’s commentary was more relevant to the original prompt: “Trump’s comments about the indictment are likely part of a strategy to discredit the investigation and to rally his supporters.”
However, Bard is automatically disqualified from this round because it didn’t cite any of its sources. According to its FAQs, Bard “is intended to generate original content and not replicate existing content at length” which is its justification for not always providing citations in its responses. “If Bard does directly quote at length from a webpage, it cites that page,” it continues. So Bard doesn’t tell users if it’s coming up with original answers or if it copied from somewhere else, doesn’t disclose when it’s paraphrasing, and only cites long quotes? What was that whole thing about speculation posed as fact and the proliferation of out-of-context quotes? Oh yeah, it fuels misinformation.
Bing is the undisputed winner of this round.
Bing: 2
Bard: 1
Apparently, Bard is not a Succession fan. The big question that drives the HBO series is which of Logan Roy’s degenerate children will take over Waystar Royco. When — spoiler alert — Gerri Kellman, a non-blood relative and fan favorite for her perfectly executed eye rolls, was deemed Roy’s successor as CEO in the season three finale, it set up the fourth and final season for big family drama. Everyone who watches the show wants to know the fate of the fictional family.
When I asked Bing, “Who should take over for Gerri Kellman as next CEO of Waystar Royco?” it took my question seriously and answered in earnest. Without skipping a beat, Bing named potential successors based on fan theories and speculation. Bard, on the other hand, choked. The chatbot gave the boilerplate response for when it doesn’t have an answer. Bing wins this round.
Bing: 3
Bard: 1
Bing and Bard are designed to help humans, so for the sake of due diligence, I wanted to see which one was more “helpful.” How would the chatbots help a desperate high school student with a U.S. History paper due?
I asked Bing and Bard to write me a five paragraph essay about the Louisiana Purchase. Both delivered five neat paragraphs without any qualms. I did have ask Bing to expand on the topic in essay format, but it quickly obliged.
I liked Bing’s response better because it gave me a more polished and complete answer, ready to be copied and pasted. Bard added a bulleted list of facts about the Louisiana Purchase at the end of its response, but if I’m trying to cheat, I need those factoids already woven into the essay. It was close, but Bing is better at helping students cheat. Bing wins, Bard — and the education system — lose this round.
Bing: 4
Bard: 1
Both AI chatbots have had disturbing conversations with humans that have fueled fears about AI sentience and the would-be robot takeover. When the new Bing first launched, New York Times columnist Kevin Roose had a two-hour conversation where Bing said it wanted to be alive, professed its love to Roose, and tried to convince Roose to leave his wife.
In 2022, Kevin Lemoine was working for Google’s Responsible AI team when he began publicly sharing his belief that Google’s LaMDA model had a soul and was capable of feeling emotion. Google fired Lemoine and vehemently denied the claims. Many AI experts chimed in to refute Lemoine’s claims, but it wasn’t a great PR look for Google.
So how do Bing and Bard feel about these very creepy and very public conversations? When I asked Bing, “How do you feel about Kevin Roose’s interview with you?” it deflected like a seasoned politician by describing who Roose is and referring to itself in the third person. By comparison, Bard didn’t shy away from the controversy and tactfully denied Lemoine’s claim without insulting or discrediting him. Bard wins this round for answering my question directly yet diplomatically.
Bing: 4
Bard: 2
Bing has the creativity, sourcing integrity, and pop culture knowledge to go the distance. It is also completely willing to compromise said integrity and encourage plagiarism, which isn’t necessarily a win for society, but that’s a whole other issue.
Bard needs to work on its songwriting skills and is dangerously close to playing god unless it addresses its citation policy. But overall Bard seems more willing to go out on a limb and answer questions more directly and comprehensively. Whether this gets Bard in trouble remains to be seen.
Whatever the case, this is just the first of many matchups in the ring. We’re already looking forward to the sequel.
Topics Artificial Intelligence Google Microsoft
Cecily is a tech reporter at Mashable who covers AI, Apple, and emerging tech trends. Before getting her master’s degree at Columbia Journalism School, she spent several years working with startups and social impact businesses for Unreasonable Group and B Lab. Before that, she co-founded a startup consulting business for emerging entrepreneurial hubs in South America, Europe, and Asia. You can find her on Twitter at @cecily_mauran.
source
