DIY Photography

Your one stop shop for everything photo-video

  • News
  • Inspiration
  • Reviews
  • Tutorials
  • DIY
  • Gear
Search

Submit A Story

Google flags medical photos of children as “sexual abuse,” sending parents under police investigation

Aug 24, 2022 by Dunja Djudjic 1 Comment

  • Share
  • Tweet
  • Flipboard
  • WhatsApp

google photos children sexual abuse

Two fathers from two US states recently went under police investigation over “sexual abuse.” The reason: Google Photos’ algorithm flagged photos of their children, closed their accounts, and automatically reported them to authorities.

The thing is: none of the dads were involved in such despicable crimes. Both cases took place in the middle of COVID-19 lockdowns. The parents photographed their children in order to send the photos to the doctors, but Google couldn’t tell the context which caused them a lot of trouble.

Both cases happened in 2021 when the entire world was in lockdown. One dad from San Francisco and the other from Houston both tried to get diagnoses for their children from pediatricians. What got Google alarmed is that, in both cases, the photos were of the children’s genitals.

The San Francisco case

When a man named Mark noticed swelling on his toddler’s groin, his wife scheduled an emergency video consultation with a doctor for the following morning. Because, you know, the pandemic. According to The New York Times, the scheduling nurse told them to send photos of the swelling so that the doctor could see it before the call. Mark took photos with his phone, the wife texted them to herself, and then uploaded them to the medical provider’s message system. It all went well: the doctor determined what caused the swelling, prescribed antibiotics, and the kid got well.

But shortly after that, Google notified Mark that his account had been disabled for “harmful content” that was in “severe violation of Google’s policies and might be illegal.” Naturally, he filed an appeal, but Google rejected it. In fact, they even reported the photos to the National Center for Missing and Exploited Children’s CyberTipline, who further reported Mark to the police.

Ten months later, the San Francisco Police Department notified Mark that they’d investigated him based on the photos and Google’s report. They had a search warrant to access everything in Mark’s Google account: messages, photos, videos, internet searches, and location. Eventually, they concluded that Mark didn’t commit the crime he was accused of, and he used the police report to try and reactivate his account. However, Google denied his request once again.

The Houston case

In the case of the Houston dad, it got even more complicated. The course of the events was similar: the doctor asked the father to take photos of his son’s genitals so they could diagnose an infection. Since automatic backup was on, the images were automatically uploaded to Google Photos and sent from the father to his wife via Google messenger. Once again, Google flagged the photos, filed a report, and the couple was investigated by the police who concluded that no crime was commited.

What makes things even worse is that this man didn’t only lose all his emails and a bunch of data on his Google accounts. He and his wife were reportedly in the middle of buying a new home. Since the photos had the man’s email address disabled, they faced additional complications.

Speaking with Gizmodo via email, a Google spokesperson said the following:

 

“Child sexual abuse material (CSAM) is abhorrent and we’re committed to preventing the spread of it on our platforms. We follow US law in defining what constitutes CSAM and use a combination of hash matching technology and artificial intelligence to identify it and remove it from our platforms. Additionally, our team of child safety experts reviews flagged content for accuracy and consults with pediatricians to help ensure we’re able to identify instances where users may be seeking medical advice. Users have the ability to appeal any decision, our team reviews each appeal and we will reinstate an account if an error has been made.”

As Gizmodo reports, the police concluded in both cases that the parents had committed no crime. However, Google still stuck with its decision and permanently deactivated the men’s accounts across all its platforms despite the appeals.

Frankly, you can’t blame AI for flagging the photos, as they for once  actually contain what they were flagged for. In fact, in cases like this, even human screening can go wrong if the context is missing. Still, if the police concluded that both fathers are innocent. The men used the police report as a proof of innocence and appealed Google’s decision, so I don’t see why their accounts remain suspended.

[via Gizmodo]

FIND THIS INTERESTING? SHARE IT WITH YOUR FRIENDS!

  • Share
  • Tweet
  • Flipboard
  • WhatsApp

Related posts:

Terry Richardson under investigation over sexual assault claims Apple to scan iPhones and iCloud for photos of child sexual abuse Man under police investigation for taking a photo of a child at Starbucks Police tells attorney he can’t film them, Police Chief invites citizens to film the police in response

Filed Under: news Tagged With: Abuse, flagging, Google Photos

Dunja Djudjic: from diyphotography.net

About Dunja Djudjic

Dunja Djudjic is a multi-talented artist based in Novi Sad, Serbia. With 15 years of experience as a photographer, she specializes in capturing the beauty of nature, travel, and fine art. In addition to her photography, Dunja also expresses her creativity through writing, embroidery, and jewelry making.

« This “mind-reading” AI turns your thoughts into photos
Tamron 50-400mm f/4.5-6.3 is a new, groundbreaking all-purpose zoom for Sony E »

Submit A Story

Get our FREE Lighting Book

DIYP lighting book cover

* download requires newsletter signup
DIYPhotography

Recent Comments

Free Resources

Advanced lighting book

Recent Posts

  • Here’s a bullet time video booth you can build yourself
  • Ricoh has discontinued the HD PENTAX-DA 21mm F3.2AL Limited silver lens
  • This “stellar flower” unravels the twilight’s evolution in 360 degrees
  • Strobes vs Continuous LEDs – Which is right for you?
  • Wave goodbye to Apple’s My Photo Stream next month

Udi Tirosh: from diyphotography.netUdi Tirosh is an entrepreneur, photography inventor, journalist, educator, and writer based in Israel. With over 25 years of experience in the photo-video industry, Udi has built and sold several photography-related brands. Udi has a double degree in mass media communications and computer science.

Alex Baker: from diyphotography.netAlex Baker is a portrait and lifestyle driven photographer based in Valencia, Spain. She works on a range of projects from commercial to fine art and has had work featured in publications such as The Daily Mail, Conde Nast Traveller and El Mundo, and has exhibited work across Europe

David Williams: from diyphotography.netDave Williams is an accomplished travel photographer, writer, and best-selling author from the UK. He is also a photography educator and published Aurora expert. Dave has traveled extensively in recent years, capturing stunning images from around the world in a modified van. His work has been featured in various publications and he has worked with notable brands such as Skoda, EE, Boeing, Huawei, Microsoft, BMW, Conde Nast, Electronic Arts, Discovery, BBC, The Guardian, ESPN, NBC, and many others.

John Aldred: from diyphotography.netJohn Aldred is a photographer with over 20 years of experience in the portrait and commercial worlds. He is based in Scotland and has been an early adopter - and occasional beta tester - of almost every digital imaging technology in that time. As well as his creative visual work, John uses 3D printing, electronics and programming to create his own photography and filmmaking tools and consults for a number of brands across the industry.

Dunja Djudjic: from diyphotography.netDunja Djudjic is a multi-talented artist based in Novi Sad, Serbia. With 15 years of experience as a photographer, she specializes in capturing the beauty of nature, travel, and fine art. In addition to her photography, Dunja also expresses her creativity through writing, embroidery, and jewelry making.

Copyright © DIYPhotography 2006 - 2023 | About | Contact | Advertise | Write for DIYP | Full Disclosure | Privacy Policy