This super-realistic virtual world is a driving school for self-driving AI

The problem is that for an AI to learn to handle the chaos of real roads, it has to be exposed to the full range of events that it might encounter. That’s why driverless-car firms have spent the last decade driving millions of miles on streets around the world. A few, like Cruise and Waymo, have begun testing vehicles without human drivers in a handful of quiet urban environments in the US. But progress is still slow. “Why haven’t we seen an expansion of these small pilots? Why aren’t those vehicles everywhere?” asks Urtasun.

Urtasun makes bold claims for the head of a company that not only hasn’t road-tested its tech, but doesn’t even have any vehicles. But by avoiding most of the costs of testing the software on real streets, she hopes to build an AI driver more quickly and cheaply than her competitors, giving the whole industry a much-needed boost. 

Virtual drivers

Waabi is not the first company to develop realistic virtual worlds to test self-driving software. In the last few years, simulation has become a mainstay for driverless-car firms. But the question is whether simulation alone will be enough to help the industry overcome the final technical barriers that have stopped it from becoming a viable proposition. “No one has yet built the Matrix for self-driving cars,” says Jesse Levinson, cofounder and CTO of Zoox, an autonomous-vehicle startup bought by Amazon in 2020.

In fact, nearly all autonomous-vehicle companies now use simulation in some form. It speeds up testing, exposing the AI to a wider range of scenarios than it would see on real roads, and it cuts costs. But most companies combine simulation with real-world testing, typically looping back and forth between real and virtual roads. 

Waabi World takes the use of simulation to another level. The world itself is generated and controlled by AI, which acts as both driving instructor and stage manager—identifying the AI driver’s weaknesses and then rearranging the virtual environment to test them. Waabi World teaches multiple AI drivers different abilities at the same time before combining them into a single skill set. It all happens nonstop and without human input, says Urtasun. 

Rare events

Driverless-car companies use simulation to help them test how the neural networks controlling the vehicles handle rare events—a bike courier cutting in front, a truck the color of the sky blocking the way, or a chicken crossing the road—and then tweak them accordingly.

“When you have an event that happens rarely, it takes thousands of road miles to test it properly,” says Sid Gandhi, who works on simulation at Cruise, a company that’s begun testing fully autonomous vehicles on a limited number of roads in San Francisco. That’s because rare—or long-tail—events might happen only one time in a thousand. “As we work on solving the long tail, we’ll rely less and less on real-world testing,” he says.