The Washington PostDemocracy Dies in Darkness

After AI chatbot goes a bit loopy, Microsoft tightens its leash

The Bing chatbot, after five responses, now tells people it would “prefer not to continue this conversation”

Microsoft chief executive Satya Nadella unveils a search engine powered by artificial intelligence this month. (Jovelle Tamayo for The Washington Post)
3 min

Microsoft started restricting on Friday its high-profile Bing chatbot after the artificial intelligence tool began generating rambling conversations that sounded belligerent or bizarre.

The technology giant released the AI system to a limited group of public testers after a flashy unveiling earlier this month, when chief executive Satya Nadella said that it marked a new chapter of human-machine interaction and that the company had “decided to bet on it all.”

But people who tried it out this past week found that the tool, built on the popular ChatGPT system, could quickly veer into some strange territory. It showed signs of defensiveness over its name with a Washington Post reporter and told a New York Times columnist that it wanted to break up his marriage. It also claimed an Associated Press reporter was “being compared to Hitler because you are one of the most evil and worst people in history.”

Microsoft officials earlier this week blamed the behavior on “very long chat sessions” that tended to “confuse” the AI system. By trying to reflect the tone of its questioners, the chatbot sometimes responded in “a style we didn’t intend,” they noted.

Those glitches prompted the company to announce late Friday that it started limiting Bing chats to five questions and replies per session with a total of 50 in a day. At the end of each session, the person must click a “broom” icon to refocus the AI system and get a “fresh start.”

Reporter Danielle Abril tests columnist Geoffrey A. Fowler to see if he can tell the difference between an email written by her or ChatGPT. (Video: Monica Rodman/The Washington Post)

Whereas people previously could chat with the AI system for hours, it now ends the conversation abruptly, saying, “I’m sorry but I prefer not to continue this conversation. I’m still learning so I appreciate your understanding and patience.”

The chatbot, built by the San Francisco technology company OpenAI, is built on a style of AI systems known as “large language models” that were trained to emulate human dialogue after analyzing hundreds of billions of words from across the web.

Its skill at generating word patterns that resemble human speech has fueled a growing debate over how self-aware these systems might be. But because the tools were built solely to predict which words should come next in a sentence, they tend to fail dramatically when asked to generate factual information or do basic math.

“It doesn’t really have a clue what it’s saying and it doesn’t really have a moral compass,” Gary Marcus, an AI expert and professor emeritus of psychology and neuroscience at New York University, told The Post. For its part, Microsoft, with help from OpenAI, has pledged to incorporate more AI capabilities into its products, including the Office programs that people use to type out letters and exchange emails.

The Bing episode follows a recent stumble from Google, the chief AI competitor for Microsoft, which last week unveiled a ChatGPT rival known as Bard that promised many of the same powers in search and language. The stock price of Google dropped 8 percent after investors saw one of its first public demonstrations included a factual mistake.