No matter what the platform, the Internet is plagued with a common problem. Whether it’s Facebook, Instagram, YouTube, or a regular old fashioned forum on the web, trolls are everywhere. They have nothing better to do than try to mess up the day of other people. They’re the largest collective source of negativity on the Internet, and countless hours are wasted every day manually trying to thwart them.
This Wired interview with Instagram CEO, Kevin Systrom, they think they’ve found an answer. Instagram believes that the future for fighting trolls lays in artificial intelligence. A piece of software designed to detect the troll comments and remove them before they’re ever seen. The goal is to produce a much more pleasant community in which Instagram users can exist and converse.
It’s an interesting interview. It’s a bit long (nearly 23 minutes), but it talks about the issues really well. Online trolls can have real life consequences. They are a serious concern. So, it’s not just about making online platforms a nice place to visit. It’s not just about making sure people will come back to the platform. It can literally save lives.
Two images are spoken about during the interview from Systrom’s own Instagram feed. One, from a year ago, is filled with hate fuelled comments. The other, more recent image, is nothing but positive comments. Aside from the effect this has on the person who posted the image, it creates a very different mood in those viewing it, too.
Over the last year, Instagram has added quite a few features to help prevent the problem. It’s also given users a lot more control over comments on their images. They’ve introduced some spam protection that seems to work rather well, too.
The first task fro Instagram was to figure out exactly what problems the users were facing. Instagram actually has a “Community team”. These people are hired solely to interface with the users. So identifying the problems was relatively straightforward. Then Facebook released its AI text analysis system, DeepText, and then the idea hit him.
I think if we get a bunch of people to look at comments and rate them good or bad, that’s your training set. Then what you do is you basically feed it to the machine learning system, and you let it go through 80% of it. Then you hold out 20% of the other comments.
And then you say “Ok, machine, go and rate these comments for us based on the training set”, and then we see how well it does, and we tweak it over time.
He goes on to say that now, the system has been fed so much information that it can detect these negative & troll comments with “amazing accuracy”.
There is some nuance in comments on Instagram, which, when taken out of context, can appear to be trolls when they are not. Two friends, giving each other a hard time and joking around in the comments on their images is one thing. A random person with whom there is no history doing it is completely another.
This is a big challenge, and Systrom admits there will be some false positives that get caught up in this system. It’s a difficult balance, and the question is whether it’s worth risking these few false positives for the sake of all the genuine bad that gets blocked. Personally, I think it is. Although, I also think that eventually, there’ll be a way to solve this issue.
One interesting thing mentioned in the interview is the vague description of things Instagram keeps a “running score” of with each user’s account. It might go some way to explain why some users suddenly start to notice a huge dip in traffic and engagement, despite their follower count going up.
There’s no word on when this technology will be implemented into Instagram. Systrom does say, though, that it’s already out there on the phones of Instagram employees right now. So, it sounds inevitable.
If it works as well as hoped, then it’d be nice to see similar applied to other platforms, too. Instagram haven’t confirmed if they’ll open source the system, but they don’t seem to rule out the possibility at this point.
