Research exploring adversarial machine learning, or the ability to fool machine learning systems, is on display at the Science Museum in London as part of “Driverless: Who is in Control?” This free exhibit includes a modified stop sign developed by a team of researchers to fool driverless cars into misidentifying it and asks “can self-driving cars see the world as well as you can?”
By placing stickers or posters on a stop sign, the team was able to fool a machine learning system into misidentifying the stop sign as, say, a speed limit sign, or failing to identify it as a traffic sign altogether. The team, which included Ivan Evtimov, Earlence Fernandes, and Co-Director Tadayoshi Kohno from the UW Tech Policy Lab as well as researchers from Samsung Research North America, Stanford University, Stony Brook University, University of California at Berkeley, and University of Michigan, was one of the first to fool a machine learning system by making real world alterations to objects. The stickers or posters added to the signs are designed to mimic vandalism or street art, making it difficult for a casual observer to identify the risk they could pose. To someone in the driver’s seat, the sign would still look like a stop sign. But for the driverless car, it might be identified as something else entirely.
“Our goal is to identify security problems in self-driving car systems to make them safer. As self-driving cars ‘see’ in a completely different way compared to humans, there could be weaknesses in how they recognize objects that hackers could exploit,” says Earlence Fernandes, a Tech Policy Lab alumnus and professor of computer science at the University of Wisconsin-Madison.
The exhibit, which runs through October 2020, looks at the use of artificial intelligent technologies and asks “How many of these smart machines already exist? And how much control are we willing to transfer to them?”
A paper on image classification was published at Computer Vision and Pattern Recognition (CVPR 2018) and a paper on object detectors at the 12th USENIX Workshop on Offensive Technologies (WOOT 2018). Learn more about this research on our website and on the project website looking at road sign classifiers and object detectors.