Microsoft’s recent public demo of Bing’s AI-powered upgrade included several factual errors, exposing the risks that the company and its competitors face while implementing new technologies into search engines. During the demo, held at Microsoft headquarters, the company showcased how integrating artificial intelligence features from ChatGPT would improve Bing’s search results with more conversational and complex answers.
The demo included a list of pros and cons for various products, an itinerary for a trip to Mexico City, and the ability to compare corporate earnings quickly. This event highlighted the potential for errors when incorporating new technology into search engines and the need for continued development and testing to ensure accurate results.
An analysis of a demo showcasing the capabilities of an AI system has revealed some inaccuracies. The system, designed to provide recommendations to users, failed to differentiate between different types of vacuums and even provided made-up information about certain products. Independent AI researcher Dmitri Brereton also found that the system missed relevant details or fabricated information about the bars it referenced in Mexico City.
Furthermore, the system inaccurately stated the operating margin of Gap and compared it to a set of Lululemon results that were not factually correct. These findings highlight the importance of thoroughly testing and refining AI systems to ensure their accuracy and reliability.
Microsoft has responded to a report about their AI-powered writing tool and acknowledged that there is room for improvement.
The company said, “We have reviewed the report’s findings and are using this feedback to enhance the user experience. We understand that the system is still in its preview phase and mistakes may occur, which is why we value feedback to help improve the models.” Microsoft is committed to learning from its mistakes and developing better user solutions.
Following the launch of Bing’s new AI-powered tool, users have interacted with it and shared their feedback, allowing the model to learn and make many improvements. However, some users have pointed out apparent mistakes in the tool’s results.
This discovery comes after Google’s public demo of a similar AI-powered tool had an error, which caused a loss of $100 billion in value for the company’s shares. With more companies racing to deploy AI chatbots, experts warn that generative AI systems are notoriously unreliable and may produce inaccurate results. As Laura Edelson, a computer scientist and misinformation researcher at New York University, said, “there’s a big difference between an AI sounding authoritative and producing accurate results.”
Bing’s AI feature has come under scrutiny this week as CNN conducted a series of tests that revealed inaccuracies. When asked about Meta’s fourth-quarter results, Bing responded with bullet points that incorrectly stated the figures. For example, Bing claimed that Meta’s revenue was $34.12 billion, while the actual amount was $32.17 billion. Additionally, Bing stated that revenue had increased from the prior year, but it had decreased.
In another search, CNN asked Bing for the pros and cons of the best baby cribs. Bing listed several cribs and their pros and cons, citing a Healthline article. However, Bing attributed the information to an article that was not there. For instance, Bing claimed that one crib had a “water-resistant mattress pad,” but that information was not in the article. These inaccuracies have raised concerns about Bing’s accuracy and reliability.
Executives from Microsoft and Google have acknowledged that there may be some potential issues with their new AI tools.
In a recent statement, Microsoft’s Vice President and Consumer Chief Marketing Officer, Yusuf Mehdi, admitted that their AI tool might not be able to answer every question all the time. He also acknowledged that mistakes might happen, so they have included a quick feedback button on every search page. This feature will allow users to provide feedback, which will help the AI tool learn and improve.
