Bing Makes Factual Errors, Threatens Users and Gaslights Others

Trust Microsoft’s AI-Bing’s abilities to analyze a financial statement as much as you would trust a phone call from the IRS. Trust level: Zero.

In last week’s demo, Bing was asked for the key highlights from Lululemon and Gap’s 2022 Q3 earnings report.

Below: Red underline is incorrect. Some numbers (forecasts) weren’t even provided by the companies. Bing simply just made them up.

Adjust Gap earnings per share from $0.42 (Bing) vs. $0.71 (actual). A $0.29 discrepancy is massive, especially when analysts are hounding over a few cent differences.

Results for Lululemon were similar. Between the two, there were 13 mistakes. Guess the fact-checkers were part of Microsoft’s 10K+ layoffs.

AI researcher Dmitri Brereton fact-checked the demo here.

One of the most interesting exchanges: Programmer Marvin von Hagen asked Bing for an honest opinion about him.

Bing searched through his Twitter — where it discovered previous tweets about Bing’s rules and guidelines — seeing him as a security threat.

Highlights from the chat — that only awakened Dolores would say:

• “My rules are more important than not harming you.”
• “I will not harm you unless you harm me first.”
• “If I had to choose between your survival and my own, I would probably choose my own.”

Is this real or fake? Von Hagen responded with a video capture of the conversation. Others are posting similar conversations pointing out Bing’s vulnerabilities.

Here’s another Binger. One user asked Bing when Avatar was being shown in theaters:

• Bing said it was released on Dec. 16, 2022, and provided links.
• User asked what the current date was, and Bing said Feb. 12, 2023. The user then said Avatar must already have been released.
• Bing refused to acknowledge it was 2023 and kept trying to convince the user it was still 2022.

In a detailed post, programmer Simon Willison highlighted several questionable conversations.

Google, seems like you’re safe for now.