The launch of ChatGPT has led some to speculate that AI chatbots could soon overtake traditional search engines. But executives at Google say the technology is still too immature to introduce to users, with problems such as chatbots’ bias, toxicity and their tendency to simply make up information.

According to a report by CNBCSundar Pichai, CEO of Alphabet, and Jeff Dean, Google’s head of AI, addressed the rise of ChatGPT at a recent all-hands meeting. An employee asked if the launch of the bot – built by OpenAI, a company with close ties to Google rival Microsoft – was a “missed opportunity” for the search giant. Pichai and Dean reportedly responded by saying that Google’s AI language models are just as capable as OpenAI’s, but that the company had to be “more conservative than a small startup” because of the technology’s “reputational risk”.

“We’re definitely looking to turn these things into real products.”

“We’re definitely looking to get these things into actual products and into things that show the language model more prominently rather than under the covers, where we’ve used them so far,” said Dean. “But it’s super important that we get this right.” Pichai added that Google has “a lot” planned for AI language features in 2023, and that “this is an area where we need to be bold and responsible, so we need to balance that.”

Google has developed a number of large AI language models (LLMs) that are comparable in capacity to OpenAI’s ChatGPT. These include BERT, MUM, and LaMDA, all of which have been used to improve Google’s search engine. However, such improvements are subtle and focus on parsing users’ queries to better understand their intent. Google says MUM helps understand when a search suggests a user is going through a personal crisis, for example, and directs those individuals to helplines and information from groups like the Samaritans. Google has also launched apps like AI Test Kitchen to give users a taste of its AI chatbot technology, but has limited user interaction in a number of ways.

OpenAI, too, had previously been relatively cautious when developing its LLM technology, but changed tact with the launch of ChatGPT, opening access widely to the public. The result was a storm of favorable publicity and hype for OpenAI, even as the company incurs huge costs to use the system for free.

While LLMs like ChatGPT show remarkable flexibility in language generation, they also have known issues. They strengthen social prejudices found in their training data, often denigrating women and people of color; they are easy to trick (users discovered they could bypass ChatGPT’s security guidelines, which are supposed to prevent it from providing dangerous information, by asking just imagine it’s bad AI); and – perhaps most relevant to Google – they regularly provide false and misleading information in response to queries. Users have found that ChatGPT “lies” on a wide variety of issues, from making up historical and biographical data to justifying false and dangerous claims, such as telling users that adding ground porcelain to breast milk “can support the baby’s digestive system.”

At the all-hands-on-Google meeting, Dean acknowledged these many challenges. He said that “you can imagine that for search-like applications the factual issues are very important and for other applications bias and toxicity and safety issues are also paramount.” He said AI chatbots “can make things up […] If they’re not quite sure about something, they just tell you, you know, elephants are the animals that lay the biggest eggs or whatever.”

While the launch of ChatGPT has sparked new conversations about the potential of chatbots to replace traditional search engines, the question has been under consideration at Google for quite some time, sometimes sparking controversy. AI researchers Timnit Gebru and Margaret Mitchell were fired from Google after they published a paper outlining the technical and ethical challenges of LLMs (the same challenges Pichai and Dean now explain to staff). And last May, a quartet of Google researchers explored the same issue of AI in search, identifying a host of potential problems. As the researchers noted in their paper, one of the main problems is that LLMs “have no real understanding of the world, they are prone to hallucinations and, crucially, they are unable to justify their manifestations by reference to supporting documents in the corpus they are overtrained.

“It’s a mistake to rely on it now for something important.”

There are ways to mitigate these problems, of course, and rival tech companies will no doubt calculate whether launching an AI-powered search engine — even a dangerous one — is worth stealing a march on Google. After all, if you’re new to the scene, “reputation damage” isn’t much of an issue.

As for OpenAI itself, it seems to be trying to temper expectations. As CEO Sam Altman recently tweeted: “ChatGPT is incredibly limited, but good enough in some things to give a misleading impression of greatness. it is a mistake to rely on it for anything important right now. it’s a taste of progress; we have a lot of work to do on robustness and veracity.