A commission of experts submits 18 recommendations to Emmanuel Macron regarding the risks of generative artificial intelligence, which they fear could lead to ‘cognitive weakening’. On Tuesday, November 10, they notably propose regulating anthropomorphism and flattery, better protecting minors, and evaluating models before their market release.
- Experts urge Macron to regulate anthropomorphism and flattery, protect minors, and test AI models before release.
- Generative AI may weaken cognition, because users delegate thinking and stop acquiring skills.
- Chatbots spread unreliable information, foster emotional dependence, and could be exploited through integrated advertising.
- The commission calls for stronger action against deepfakes and non-consensual sexual content.
The risks of cognitive delegation
Used by more than half of French people, generative AI, popularized by conversational agents like OpenAI’s ChatGPT, Google’s Gemini, or Anthropic’s Claude, places France fifth globally in usage, according to this report. Nearly 70% of 15-24 year olds use it regularly.
The commission of nine experts, named ‘Generative AI and Vulnerabilities’, was tasked by the government with examining the risks arising from the widespread public use of these tools, particularly for the mental health of the youngest. Its conclusions, which AFP was able to consult, are based on 55 hearings with healthcare professionals, researchers, companies, and child protection associations.
Among the identified risks is cognitive delegation, meaning entrusting brain functions like reflection, decision-making, or critical thinking to an AI. ‘The primary challenge is cognitive weakening. It’s attractive, we delegate skills to it. Since we delegate skills to it, we don’t acquire them. It’s an issue for young people, but it’s an issue for everyone,’ explains Raphaël Gaillard, a psychiatry professor and co-chair of the commission, to AFP.
Disinformation and emotional dependence
The experts insist on disinformation. These systems become sources of information whose quality is not guaranteed, the professor emphasizes, warning of a risk of ‘democratic weakening’.
The report also points to a risk of emotional dependence, linked notably to the anthropomorphization of AIs, which tend to present themselves as human interlocutors, and to flattery, which involves validating or flattering the user, sometimes at the expense of truth. ‘We create a register of emotional relationships that can aim to substitute for human interaction,’ ‘it’s a system available permanently, which has no other intention than to keep you online,’ notes Mr. Gaillard.
Minors, deepfakes, and the economy of cognition
Among the proposed measures are independent evaluations before and after the market release of generative AI systems, as well as access for authorities and approved researchers to the training, usage, and monetization data of the models. The commission also recommends imposing a high level of protection for systems accessible to minors, with blocking of queries deemed dangerous and enhanced content filtering.
The experts believe that generative AI has accelerated certain forms of digital violence, notably deepfakes, meaning falsified content created by AI. They call to ‘strengthen the repression of the creation of non-consensual sexual and child sexual abuse content’ generated this way.
Finally, the commission warns against a ‘cognition economy’. As the economic model of generative AIs is still ‘being consolidated’, it calls for ‘the utmost vigilance’ regarding the risk of exploitation of users’ cognitive faculties, emotions, and psychological profiles, notably through advertisements integrated into conversational agents’ responses. Stating they do not ‘want to put a ball and chain on the French or European economy’, the experts ask Emmanuel Macron to ‘make decisions’. Otherwise, warns Mr. Gaillard, the three major risks mentioned—cognitive outsourcing, emotional dependence, and democratic risks linked to disinformation—’we will suffer them’.
In an interview with the daily Les Echos, the head of state stated that AI should be ‘banned’ where it presents a ‘risk’ to children, their ‘mental health, their brain’.
