Bing Makes Factual Errors, Threatens Users and Gaslights Others – The Average Joe

    Bing Makes Factual Errors, Threatens Users and Gaslights Others

    Victor Lei — Head of Research

    February 16, 2023

    February 16, 2023

    Trust Microsoft’s AI-Bing’s abilities to analyze a financial statement as much as you would trust a phone call from the IRS. Trust level: Zero.

    In last week’s demo, Bing was asked for the key highlights from Lululemon and Gap’s 2022 Q3 earnings report.

    Below: Red underline is incorrect. Some numbers (forecasts) weren’t even provided by the companies. Bing simply just made them up.

    Adjust Gap earnings per share from $0.42 (Bing) vs. $0.71 (actual). A $0.29 discrepancy is massive, especially when analysts are hounding over a few cent differences.

    Results for Lululemon were similar. Between the two, there were 13 mistakes. Guess the fact-checkers were part of Microsoft’s 10K+ layoffs.

    AI researcher Dmitri Brereton fact-checked the demo here.

    One of the most interesting exchanges: Programmer Marvin von Hagen asked Bing for an honest opinion about him.

    Bing searched through his Twitter — where it discovered previous tweets about Bing’s rules and guidelines — seeing him as a security threat.

    Highlights from the chat — that only awakened Dolores would say:

    • “My rules are more important than not harming you.”
    • “I will not harm you unless you harm me first.”
    • “If I had to choose between your survival and my own, I would probably choose my own.”

    Is this real or fake? Von Hagen responded with a video capture of the conversation. Others are posting similar conversations pointing out Bing’s vulnerabilities.

    Here’s another Binger. One user asked Bing when Avatar was being shown in theaters:

    • Bing said it was released on Dec. 16, 2022, and provided links.
    • User asked what the current date was, and Bing said Feb. 12, 2023. The user then said Avatar must already have been released.
    • Bing refused to acknowledge it was 2023 and kept trying to convince the user it was still 2022.

    In a detailed post, programmer Simon Willison highlighted several questionable conversations.

    Google, seems like you’re safe for now.

    Recent Posts