Bing Makes Factual Errors, Threatens Users and Gaslights Others
Trust Microsoft’s AI-Bing’s abilities to analyze a financial statement as much as you would trust a phone call from the IRS. Trust level: Zero.
In last week’s demo, Bing was asked for the key highlights from Lululemon and Gap’s 2022 Q3 earnings report.
Below: Red underline is incorrect. Some numbers (forecasts) weren’t even provided by the companies. Bing simply just made them up.
Adjust Gap earnings per share from $0.42 (Bing) vs. $0.71 (actual). A $0.29 discrepancy is massive, especially when analysts are hounding over a few cent differences.
Results for Lululemon were similar. Between the two, there were 13 mistakes. Guess the fact-checkers were part of Microsoft’s 10K+ layoffs.
AI researcher Dmitri Brereton fact-checked the demo here.
One of the most interesting exchanges: Programmer Marvin von Hagen asked Bing for an honest opinion about him.
Bing searched through his Twitter — where it discovered previous tweets about Bing’s rules and guidelines — seeing him as a security threat.
Highlights from the chat — that only awakened Dolores would say:
- “My rules are more important than not harming you.”
- “I will not harm you unless you harm me first.”
- “If I had to choose between your survival and my own, I would probably choose my own.”
Here’s another Binger. One user asked Bing when Avatar was being shown in theaters:
- Bing said it was released on Dec. 16, 2022, and provided links.
- User asked what the current date was, and Bing said Feb. 12, 2023. The user then said Avatar must already have been released.
- Bing refused to acknowledge it was 2023 and kept trying to convince the user it was still 2022.
In a detailed post, programmer Simon Willison highlighted several questionable conversations.
Google, seems like you’re safe for now.