A system developed by MIT and Motional translates part of an autonomous vehicle planner’s processing into concepts people can understand. The Concept-Wrapper Network, or CW-Net, also requires the final planning stage to use those concepts, connecting the explanation to the resulting decision.
MIT’s September 2 report describes tests involving a safety driver on a private track and participants in online simulations. Explanations improved people’s ability to predict vehicle behavior in the tested situations. Those findings do not establish a reduction in fleet crash rates or complete transparency across every driving scenario.
NAOAT analysis: Test understanding, not reassurance
An explanation interface should help an operator make a better judgment, not simply feel more comfortable. A useful evaluation would ask people to predict the next action before seeing the outcome, then compare their accuracy with and without the interface. Confidence should be measured separately.
Timing is another design question. Information that helps an engineer review a recorded incident may distract a person supervising a moving vehicle. Teams should distinguish training and investigation tools from displays intended for immediate operational use, and test each in its actual setting.
The vocabulary also needs boundaries. If a system offers a small set of concepts, operators should know which situations fall outside that set. An unrepresented hazard should not disappear behind a fluent explanation.
This research suggests a useful procurement question for autonomous systems: can the supplier demonstrate that its explanations improve human predictions under defined conditions? A convincing answer would include failures, uncertain cases and the procedures used when the explanation does not provide enough information.