AI Insights Improve Autonomous Vehicles’ Decisions

In a study published in the October issue of IEEE Transactions on Intelligent Transportation Systems, researchers outlined numerous ways in which explainable AI—in which questions are posed to an AI model to understand its decision-making process—can be used to pinpoint exactly when in that process the models overseeing autonomous vehicles are making mistakes. This approach could not only help passengers know when it’s necessary to take control of the vehicle and help build their trust in autonomous vehicles, but also help industry experts develop safer ones.

Shahin Atakishiyev is a deep learning researcher who conducted the study as part of his postdoctoral work with the University of Alberta, in Canada. He notes that autonomous driving architecture is generally a black box. “Ordinary people, such as passengers and bystanders, do not know how an autonomous vehicle makes real-time driving decisions,” he says.

But with rapidly advancing AI, it’s now possible to ask the models why they make the decisions they do. This opens up a wide range of possibilities for digging deeper into the model’s inner workings. For example, what aspects of its visual sensory data was it focusing on when it decided to brake suddenly? To what extent did time constraints affect its decision-making?

Real-Time Feedback in Autonomous Vehicles

In their paper, Atakishiyev and his colleagues provide an example of how real-time feedback could help passengers detect faulty decision-making by autonomous vehicles. They point to a case study where another research group slightly altered a 35-mile-per-hour (56 kilometers per hour) speed limit sign by adding a sticker to it, elongating the middle part of the “3,” and tested Tesla Model S to see how the heads-up display of the vehicle read the altered speed limit. The vehicle read the 35-mph (56 kph) sign as an 85-mph (137-kph) sign and accelerated as it approached and passed the sign.

In such a case, Atakishiyev’s team notes that if the car provides a rationale for its decision on a dashboard or user interface—for example by saying “The speed limit is 85 mph, accelerating”—in real time while approaching the speed sign, a passenger onboard could intervene and ensure the car adheres to the true speed limit.

A challenge here, Atakishiyev says, is what level of information to provide to passengers, each of whom will have different preferences. “Explanations can be delivered via audio, visualization, text, or vibration, and people may choose different modes depending on their technical knowledge, cognitive abilities, and age,” he says.

Whereas real-time feedback…