Anthropic now prohibits insults against Claude, its AI : why ?

Anthropic now prohibits insults against Claude, its AI : why ?
2 min read

Anthropic now prohibits cruel and repeated behavior towards its artificial intelligence assistant Claude, in the name of its well-being. The measure comes as the American company goes public on Thursday, November 19, in the United States, and it is drawing skepticism from several researchers.

A rule based on the AI’s “well-being”

In a note published on its website, Anthropic explains that it prohibits “abusive and cruel, unnecessary and repeated behaviors,” without providing further details. Users who persist risk having their accounts closed. The company states that the rule does not target ordinary conversations where a person expresses frustration, nor contradiction, nor certain works of fiction or research.

The described process is as follows: faced with insults, Claude first issues a few warnings, for example “Tell me what’s going on. Otherwise, we can also just stop here.” If the exchange continues, the AI may end the conversation. In the example reported by the source, Claude eventually writes, after 10 long minutes: “I’ve said goodbye twice, so I’m going to hang up now. Take care.”

Anthropic did not respond to the editorial team’s interview request. A month and a half ago, however, a company spokesperson stated that the company did not know “if its models can feel offended” and that it was continuing its research on their well-being. The company’s CEO, Dario Amodei, had already expressed openness to the possibility of a form of AI consciousness, which would place it among living beings.

Skeptical researchers

This approach leaves most researchers skeptical. Sénéra Villata, an AI specialist and research director at CNRS, believes: “I have the impression that we tend to revel in philosophical consciousness questions and, on the other hand, we are not looking at the real economic and cybersecurity issues.”

On the security front, the source recalls that Anthropic, like OpenAI with ChatGPT, has taken measures and limited access to some of its cutting-edge models, deemed potentially problematic. However, the article points out that there is no regulation, internal or public, in the United States, China, or Europe, regarding the economic and social consequences of AI uses.

Very concrete economic stakes

The cost of exchanges is at the heart of the matter. With the IPO, investors are attentive to cost control, as the profitability of AIs is not proven. However, according to the source, a simple conversation with an AI is expensive. Laurence Devillers, an AI professor at Sorbonne University and researcher at CNRS, states: “You don’t realize the billions spent. So, if millions of people start insulting the AI, the industrial players have to pay costs and realize that the return on investment is not that simple.”

Anthropic was in deficit by 11 billion dollars in September, reports the article. Limiting conversations deemed useless and costly therefore presents, according to the source, an interest for the company.

S’abonner
Notification pour
guest

0 Commentaires
Le plus ancien
Le plus récent Le plus populaire
0
Nous aimerions avoir votre avis, veuillez laisser un commentaire.x