Researchers affiliated with the University of Texas present UNCAP, a cooperation framework for autonomous vehicles in which cars describe to each other what they observe, in natural language. According to them, this system teaches vehicles to choose which neighbors to listen to and to assign an appropriate degree of trust to their messages.
- UNCAP permet aux véhicules autonomes de décrire en langage naturel ce qu’ils observent.
- Les voisins signalent ainsi des dangers cachés, que le véhicule ne peut pas voir lui-même.
- Le système choisit les voisins pertinents et pondère la fiabilité de leurs messages.
- En simulation, il réduit la bande passante de 63 % et améliore le score de conduite de 31 %, mais reste à tester en conditions réelles.
The principle is based on a simple situation. A truck parked at an intersection obscures a car approaching from the right. The neighboring vehicle, however, sees it and could transmit a message like this: “Vehicle merging from the right, about 15 meters away, moving fast.” The car thus receives information that its own sensors could not observe.
Speaking rather than sharing raw data
Sharing raw data, such as camera images, LiDAR point clouds, or radar measurements, is the classic approach. The researchers note that it becomes difficult to deploy on a large scale: these streams consume significant bandwidth, require a lot of computation, and assume that all vehicles have comparable hardware. Yet traffic brings together very different manufacturers, sensor configurations, and software.
Natural language offers an alternative. A message such as “pedestrian crossing behind the bus” or “motorcycle approaching from the left, at high speed” is very short, hardware-independent, and understandable by vision-language models, i.e., models capable of processing both images and text. However, it remains to decide which messages to listen to, and what to do when two vehicles contradict each other.
Four steps
UNCAP works without specific training, described as “zero-shot,” and relies on existing models to which it adds a communication strategy. The first step is to identify neighbors: each vehicle periodically broadcasts a signal of a few bytes containing its position and direction. The second step filters relevant vehicles, those that are close enough and likely to influence its trajectory. According to the researchers, this selection drastically reduces unnecessary exchanges and allows the system to scale.
The selected vehicles then describe their observations, accompanied by “calibrated” confidence estimates using conformal prediction, a lightweight method that matches confidence levels to the actual reliability of the information. Each vehicle then measures, using mutual information, to what extent each message reduces its uncertainty about the scene. Finally, the description thus constructed is passed to a vision-language model, which proposes a decision, for example merge, wait, or yield, with an associated confidence score.
Simulation results
Tests were conducted in the CARLA driving simulator, with the OPV2V benchmark, which includes highway merges and urban intersections. Compared to vehicles without communication, and compared to previous approaches that broadcast every observation to all vehicles, the researchers report consistent improvement. They cite a 63% reduction in communication bandwidth, a 31% increase in driving score, a 61% decrease in decision uncertainty, and safety margins four times larger during near-collisions.
The most unexpected result concerns raw images, which performed worse than language. The researchers suggest that vision-language models struggle to reason about multiple camera streams taken from different angles, and that translation into concise descriptions acts as an information filtering step. The gains are observed with several models, GPT-4o, GPT-4.1, GPT-4o-mini and GPT-5, which, according to them, is due to the communication strategy rather than a specific model.
Limitations
The experiments remain limited to simulation. Sensor noise, unreliable wireless communications, network delays, and packet loss remain to be studied. Furthermore, a query to a vision-language model takes about a second, so decisions are made at key moments and not continuously. The authors believe their approach goes beyond autonomous driving, for teams of robots, drones, or software agents. Their paper, published on arXiv, has been selected for the best paper award at the AAMAS 2026 conference.
