The smarter the artificial intelligence, the more it likes to hide its ignorance?

An artificial intelligence (AI) research team at the Polytechnic University of Valencia, Spain, found that as large language models (LLMs) become larger and more sophisticated, they tend to be less likely to admit to users that they don't know the answer.

In the study, published in the journal Nature , the team tested the latest versions of three of the most popular AI chatbots on responsiveness, accuracy, and the ability of users to spot incorrect answers.

As LLMs become larger and more sophisticated, they tend to become less open. (Illustration).

To test the accuracy of the three most popular LLMs, BLOOM, LLaMA, and GPT , the team posed thousands of questions and compared the responses received with responses from previous versions to the same questions.

They also vary in subject matter, including math, science, word puzzles, and geography, as well as the ability to create text or perform actions like sorting lists.

The study results revealed some notable trends.

The overall accuracy of chatbots improves with each new version, but still drops when faced with more difficult questions.

Surprisingly, as LLMs become larger and more sophisticated, they tend to be less open about their ability to answer correctly.

In previous versions, most LLMs would be upfront with users when they couldn't find an answer or needed more information.

In contrast, newer versions tend to guess more, resulting in more answers overall, including both correct and incorrect.

More worryingly, the study found that all LLMs still occasionally gave incorrect answers even to easy questions, suggesting that their reliability remains an issue that needs improvement.

These findings highlight a paradox in the evolution of AI: while models are becoming more powerful, they may also become less transparent about their limitations.

This poses new challenges in the use and trust of AI systems, requiring users to be more cautious and developers to focus on improving not only the accuracy but also the 'self-awareness' of models.