Facing the Flood of Papers, a One-Year Study Explores Learning from Demonstration

Facing the Flood of Papers, a One-Year Study Explores Learning from Demonstration
3 min read

Faced with the explosion of publications in robotics, six researchers attempted to read a large portion of the work published in 2024 in a single subfield, learning from demonstration (LfD). According to their study, only about 20% of the papers would be major contributions, with the rest mostly proposing incremental improvements to existing methods or new application domains.

In this method, robots learn to perform tasks by being guided by human experts. The authors, Aude Billard, Renaud Detry, Nadia Figueroa, Maximilian Foriest, Dongheui Lee and Kunpeng Yao, point out that researchers are expected to read new publications in their field to stay current, a task that quickly becomes unfeasible. For example, IEEE published 46,968 articles on ‘robotics’ or ‘automation’ in 2024 alone, while IEEE publications represent only a portion of the research available online.

A Continuous Growth in Submissions

According to the study, the number of submissions to robotics journals has increased every year for a decade, with a clear acceleration: +26% in 2023 and +31% in 2024. The authors cite several probable causes, including growing interest from the public and private sectors, as well as the availability of AI tools that help write papers and code. The number of published articles follows this trend, despite editorial boards’ efforts to curb the increase by reducing acceptance rates.

Conferences are experiencing the same evolution. The ICRA conference thus doubled the number of papers published in ten years, reaching about 1,800 in 2024. At the same time, pressure from the community to publish quickly has reduced the time between paper submission and publication by 50%.

Automated Tools with Clear Limits

The authors evaluated the task in two ways: a human reading of all the papers, and the use of scripts and large language models (LLMs, systems capable of processing and generating text). According to them, these tools provide a general quantitative assessment of acceptable fidelity. However, they fail to assess the real importance of research: they do not spot a paper that rehashes already solved work, nor the exaggerated claims of an abstract compared to the actual contribution.

Among more than 300 papers analyzed, the authors note that the most notable contributions do not necessarily correspond to the number of downloads or citations. Yet, these ‘gems’ are essential to prevent new work from going unnoticed and to limit research duplication.

The study makes several recommendations. The authors notably propose developing a search engine that ranks papers based on evaluations and the peer review status from journal and conference committees, instead of Google Scholar or IEEEXplore, which place all publications on an equal footing. They also suggest considering a blind publication model, where the names of authors and their institutions would be secondary. Finally, they recommend using LLMs for what they do well, namely summarizing work and gathering precise quantitative information, without considering them capable of assessing the real novelty of a paper. The authors note, however, that if these tools ever succeeded in doing so, it could have repercussions on the human ability to provide this expertise.

The phenomenon is not limited to IEEE: according to the study, the IJRR journals, as well as the RSS and CoRL conferences, follow the same trend.

S’abonner
Notification pour
guest

0 Commentaires
Le plus ancien
Le plus récent Le plus populaire
0
Nous aimerions avoir votre avis, veuillez laisser un commentaire.x