Threatening to ban users for inquiring

about Strawberry‘s reasoning, OpenAI

 

 

“Additional violations of this policy may result in loss of access to GPT-4o with Reasoning.”

With their latest AI model, code-named “Strawberry” and made available as an o1-preview, OpenAI says that it is capable of “reasoning.” However, it appears that the creator of ChatGPT is adamant about keeping knowledge of its thought process confidential.

OpenAI is now threatening to ban users who attempt to get the massive language model to divulge how it thinks, as reported by Ars Technica. This is a clear indication of how the firm has long since abandoned its original goal of supporting open source AI.

Social media profiles for the Microsoft-backed firm claim that customers are getting emails from them stating that their queries to ChatGPT have been flagged for “attempting to circumvent safeguards.”
“Additional violations of this policy may result in loss of access to GPT-4o with Reasoning,” the messages state.

Given that a large portion of Strawberry’s appeal stemmed from its “chain-of-thought” reasoning, which enabled the AI to explain how it arrived at a solution, step by step, this crackdown is more than a little comical. This is a “new paradigm” for the technology, according to Mira Murati, chief technical officer of OpenAI.

Regarding what causes the violations, reports differ. According to several users, Ars discovered, using the word “reasoning trace” is what landed them into trouble. Others say that even using the word “reasoning” on its own was enough to alert OpenAI’s systems. Users can still see what is essentially a summary of Strawberry’s thought process, but it’s cobbled together by a second AI model and is heavily watered-down.

In a blog post, OpenAI argues that it needs to hide the chain-of-thought so that it wouldn’t need to put a filter on how its AI thinks, in case it says stuff that isn’t compliant with safety policies while thinking out loud. Developers will be able to view its “raw” internal thought process safely in this manner.

However, as the business

But as the company freely admits, this measure also helps it maintain a “competitive advantage,” staving off competitors from trying to ride its coattails.

The flipside of this approach, however, is that concentrates more responsibility for aligning the language language model into the hands of OpenAI, instead of democratizing it. That poses a problem for red-teamers, or programmers that try to hack AI models to make them safer.

“I’m not at all happy about this policy decision,” AI researcher Simon Willison wrote on his blog, as quoted by Ars. “As someone who develops against LLMs, interpretability and transparency are everything to me the idea that I can run a complex prompt and have key details of how that prompt was evaluated hidden from me feels like a big step backwards.”

As it stands, it seems that OpenAI is continuing down a path of keeping its AI models an ever more opaque black box.