It’s quite ironic that a system designed to be bots is being beaten by bots faster than humans can get through it. But that appears to be exactly what’s happened with CAPTCHA. You know, all those tests you have to do to prove you’re not a robot on a website?
Well, robots are better at proving it than we are now, according to a new study. The study put its AI against a thousand real humans on different CAPTCHA systems on over a hundred popular websites. We lost.
What’s CAPTCHA?
CAPTCHA, for those who didn’t know, stands for “Completely Automated Public Turing test to tell Computers and Humans Apart”. It’s a pretty self-explanatory title. It’s a test to see if the person interacting with the website is human or not.
If you’re reading this, you’ve almost certainly experienced a CAPTCHA on a form somewhere during your time on the web. They exist pretty much everywhere. Even Google pops one up occasionally if it thinks you’re searching for too many things too quickly.
Sometimes, you just have to check a box that says you’re not a robot. Sometimes, you might need to decipher a code, and at other times you might be asked to identify all of the squares containing traffic lights. Or all of the photographs with a pedestrian crossing. Yeah, those. The really annoying ones you end up having to do about fifteen times before it finally lets you in.
However, with the advancements in AI over the last little while, bots are now better at beating these tests. And they’re better at beating them than we are. They’re beating us on both accuracy and overall speed – sometimes by a massive margin.
The researchers tested their system on over 100 popular websites. They also tested a thousand humans of varying ages, genders, education and locations on the same websites. While the humans, as a group, appear to cover the same range of websites, each individual human reviewed no more than ten websites.
But, why?
The point of the tests, was to essentially show that the more than twenty-year-old technology is no longer enough to keep your websites free from the bots they’re designed to protect you from. The researchers are calling for better methods to determine which users on a website are actually human.
Bots have been causing havoc on the web for as long as the web has been a thing. Back in the day, most of them were friendly and helpful. Search engine crawlers and the like. These days, there are probably more malicious ones than benign ones. And wherever possible, you want to keep them out.
Hence, the CAPTCHA tests.
The tests are designed specifically to be easy to solve by humans but difficult for a bot to solve. Most bots up until lately have been pretty dumb only doing what they’re told and they’re often coded in such a way that the first time they see any data they don’t understand, they just give up.
So, the tests beat those easily. Things have changed now, in these latest tests.
So, how did we do?
First up, accuracy. The humans in the tests beat the CAPTCHA 50-84% of the time, depending on the specific test. The AI bots, on the other hand, beat the tests 85-100% of the time across their range.
When it comes to text-based CAPTCHA tests, humans beat the test in 9-15 seconds. The accuracy for humans isn’t specified, but it’s largely irrelevant. At least, it will be when you see what the AI got. In the same tests, the bots got a 99.8% success rate with a response time of under one second.
The bots’ accuracy ranges from 85-100 per cent, with the majority above 96 per cent. This substantially exceeds the human accuracy range we observed (50-85 per cent)
On the other hand, the CAPTCHA photo tests appear to be a bit of a different story. The AI isn’t terrible. Well, I suppose, depending on your perspective, it might be. But the short version is that humans and bots are pretty much neck-and-neck when it comes to image-based CAPTCHA systems, at least for the moment.
This is very likely to change soon, given the rate at which other AI technologies are advancing. But even if they’re on level pegging with a human, the system as it stands right now is still broken.
What’s also potentially likely is that the web’s most annoying photographs are soon going to disappear forever!
[via The Independent]
FIND THIS INTERESTING? SHARE IT WITH YOUR FRIENDS!