There’s been a lot of buzz lately around AI, “deep learning”, computer vision. It’s all to do with image recognition. Apple and Google have also been implementing it with their mobile operating systems to help categorise your shots. Facebook also does this, too, although it rarely makes itself obvious. It’ll often see faces, and ask if you want to tag somebody. Sometimes it’ll even recognise the person. But that’s about it.
Facebook actually looks at a whole lot more, though. If you want to go digging through page source, you can find this information out yourself. But, it’s a bit of a hassle. Now, a new Chrome extension overlays the information right on top of the image in your browser. While primarily intended for those who use screen readers, it does offer insight into how Facebook automatically reports or censors certain images.
Installing the plugin presents an overlay in the top right hand corner of images you see as you scroll through your feed. Here’s one of mine I posted from my phone while out walking a couple of months ago.
At least it seems to do a better job of detecting birds than Flickr – which failed to recognise birds in a dozen of 15 bird photographs I tested it with a few weeks ago. It also managed to detect that there’s water running alongside the path, which is a pretty neat trick. Especially considering it’s such a small part of the overall image.
It also shows the overlays in many of the thumbnail views, too. Here’s the 9 most recent photos I’ve been tagged in on my Facebook profile.
Some of the tags are fairly simple. Whether it’s indoors or outdoors, how many people are in it, trees, cars, skies, and so on. Although, Facebook’s image recognition does seem to have something of a foot fetish. As well as the two examples above, I’ve also seen it misidentify folded DJI Mavic Pro drones as shoes, too.
For the most part, though, it looks like it’s fairly intelligent. So, if you’re wondering why your images are being reported in “Secret” Facebook groups, or why certain images come with a warning, then this probably has something to do with it.
You can install the Show Facebook Computer Vision Tags Chrome extension here. One slightly annoying thing, is that you don’t seem to be able to toggle it off and on without completely disabling the entire plugin. It’s already getting a little tiresome, especially when I can’t actually see the thumbnails of images people post.
Are you curious about how Facebook identifies things in your images? Do you care? What other benefits could Facebook provide with this technology beyond screen readers? Tell us your thoughts in the comments.