Researchers Hacked A Self-Driving Car By Putting Stickers On Street Signs

Submitted by: skypirate 4 months ago in News & Politics Tech

A few stickers and a STOP sign turns into a 45MPH sign. Looks like we got a ways to go before self driving cars are a norm.

An excerpt from AutoBlog: Researchers at University of Washington, University of Michigan, Stony Brook University, and UC Berkeley have figured out how to hack self-driving cars by putting stickers on street signs. Starting by analyzing the algorithm the vision system uses to classify images, they used a number of different attacks to manipulate signs in order to trick machine learning models into misreading them. For instance, they printed up stickers to trick the vision system an autonomous car would use into reading a stop sign as a 45-mile-per-hour sign, the consequences of which would obviously be very bad in the real world.

In the paper, "
Robust Physical-World Attacks on Machine Learning Models," the authors demonstrated four different ways they were able to disrupt the way machines read and classify these signs using only a color printer and a camera. The most troubling part about these experiments is that they all appear very subtle to the human eye, camouflaged as graffiti, art, or incorporated into the sign's imagery.
There are 19 comments:
Male 942
I don't think we will have practical self driving cars until we have achieved true AI.  You're just not going to get it to work safely without a human-level of reason and decisions making.  It's not just about recognizing street signs.  And even when we have the AI for it, we're still going to run into problems.  For example, if there is an inevitable accident due to unforeseeable circumstances, how will it decide if it should, say, run over a biker or drive itself (and its occupant) off a cliff?  There's also no one to blame if a human is not in control of the vehicle.  I just think it's a LOT further away than most people think.
0
Reply
Male 1,604
cjeffblanchr 
We will have a fully working general AI in 2022. 
Human cybernetic interfaces will be viable(though expensive) in 2024. 
Widespread integration of neural weaves in the general population will happen by 2031, essentially rendering general AI useless and relegating it to simple tasks like traffic control and infrastructure management. 
Though early gAI will have been sent to Mars, Titan, Europa, Venus and Jupiter in various rovers/satellites for research starting in December 2022 with the first being sent to Titan in the hopes of determining viability for human colonization and the final being sent Jupiter in March of 2029 with the mission of analyzing samples for automated mining of hydrogen3 from the upper atmosphere.
Once Human neural interfacing is perfected in 2030 and a non invasive method of introducing the mesh into the human brain (specifically injecting nanomachines into the bloodstream and crossing the blood-brain barrier to construct the mesh from the inside) the ability to drive without physical controls (a steering wheel or pedals) and the slow reaction time of the brain to the hands/feet will nearly eliminate traffic fatalities and lower property damage and personal injury to the point that self-driving cars will be useless.
So, no self-driving cars. 
and unfortunately no flying cars. 
0
Reply
Male 3,758
Here is the thing about self-driving cars.  As soon as it is deemed safer than human drivers, and that is coming fast, insurance companies will force the issue.  If you want have a manually driving car you will have to pay a premium.  The only exception I can see will be for vintage\classic cars, which hold more value to the driver.
0
Reply
Male 601
Just one of many minor stumbling blocks I'm sure, with each discovery like this the technology will become even better. Autonomous cars will be here pretty soon I reckon, the sooner the better in my opinion. 
0
Reply
Male 1,973
As has already been pointed out, what the hell? They weren't smart enough to program these things to recognize something as important as a stop sign by shape? The printing on it should be extremely secondary to the shape! Especially considering how many stop signs I've seen with various stickers that people have pasted to them.
0
Reply
Male 1,820
oops
0
Reply
Male 5,412
Definitely a problem, but nothing insurmountable. It wouldn't surprise me if autonomous driving cars wind up using maps tied to GPS, with notes on where Stop signs are located. If the sensors see a stop sign where there shouldn't be one, or don't see one where they should, an alarm could sound and the car could go into a safe mode with hazards flashing and a pullover to the side of the road. I'm thinking a safe mode has to be part of the design already, to handle cases where the navigation fails for whatever reason.

Also, it should be possible to improve the sensor designs to more reliably detect Stop signs by their shape. One of the more intelligent aspects of modern road sign design is that the octagonal sign is strictly reserved for Stop signs. If it isn't octagonal, it isn't a Stop sign; and if it isn't a Stop sign, it isn't octagonal. That was done so that a Stop sign covered in mud or wind-blown snow is still recognizable as a Stop sign to humans--but that good design feature ought to be leveragable by sensors.
1
Reply
Male 1,553
squrlz4ever I'm surprised the unique shape of the stop sign is not the overriding factor in determining the action to take.  GPS maps are great as long as there is reception.  The visual recognition aspect needs to be robust enough to be reliable.
1
Reply
Male 3,758
squrlz4ever Great minds think alike (see below) :)
1
Reply
Male 5,412
kalron27 I saw! Bingo. As usual, I've blathered on for a few hundred words where you managed to cover the territory in 23.  >.<  I can't help myself!

Also, I'll add this. I think these researchers did a great service to the development of autonomous cars. They identified a serious design flaw, which can now be addressed and that will make the design that much safer.
0
Reply
Male 1,820
squrlz4ever Ya like we have eliminated computer viruses.
0
Reply
Male 5,412
casaledana You seriously think self-driving cars aren't on the horizon? People around here are such sticks in the mud.

Five years ago, I posted a video on here of quadcopters at the University of Pennsylvania and predicted that in the near future, lots of people would fly them for fun and that they would be used for filmmaking and security. I remember very well the first comment in reply to that prediction of mine. It was in all caps: "YEAH. LIKE THAT'S GOING TO HAPPEN."
0
Reply
Male 715
squrlz4ever masd self driving cars arent going to happen. It will cause gridlock in cities. Pedestrians will know you walk across the road the car will stop. So why wait? Atm wr make judgement calls depending on traffic, speed of car and general experience in the city we are in. Humans change behaviour dependent on our surroundings, in cities especially people adapt.
0
Reply
Male 4,362
jayme21 Yep, that's just one, there are a billion of problems to be addressed. Not that we wont see self driving cars but on a large scale? Not in quite a long time.
0
Reply
Male 3,758
Eliminate this with further technology.  GPS can already tell speed limits in areas, expand that to know where stops are, no signs needed.
0
Reply
Male 6,562
yep, and this is why a self driving car might be a ways off.  It pretty much needs a closed system, or true ai.
0
Reply