DIY Photography

Your one stop shop for everything photo-video

  • News
  • Inspiration
  • Reviews
  • Tutorials
  • DIY
  • Gear
Search

Submit A Story

Creepy AI reconstructs your portrait based only on your voice

Apr 5, 2022 by Dunja Djudjic 9 Comments

  • Share
  • Tweet
  • Flipboard
  • WhatsApp

Turning speech into text has become so common that i’s a part of almost every smartphone. But have you ever thought about turning your speech into a portrait? Researchers have, and they’ve even made it possible.

Artificial intelligence scientists at MIT’S Computer Science and Artificial Intelligence Laboratory (CSAIL) have created AI that turns short snippets of audio speech recording into a human face. As if this weren’t both stunning and creepy enough, the results are actually fairly accurate, too!

The CSAIL researchers published a paper about their invention back in 2019. It’s an algorithm called, not surprisingly, Speech2Face, and the name says it all. In the demo, you can take a peek at how it works and what are the results. At the very top of the page, you’ll hear the audio snippets of different people speaking. Their real photo is just for your reference, and Speech2Face recreated their portrait based only on a three-second recording of their voice.

Interestingly enough, the AI seems to be working better when the audio clips are longer. The researchers have shared some examples of faces recreated from three versus six seconds of speech.

Of course, the results are still far from perfect, but they’re still amazing and eerily accurate. Still, the AI sometimes completely misses the point and mixes up the gender, age, and ethnicity of the subject:

Privacy concerns

Even though the algorithm was created for scientific purposes only, the question of privacy has been raised. The team claims that their method “cannot recover the true identity of a person from their voice,” i.e. recreate an exact image of their face.

“This is because our model is trained to capture visual features (related to age, gender, etc.) that are common to many individuals, and only in cases where there is strong enough evidence to connect those visual features with vocal/speech attributes in the data (see “voice-face correlations” below). As such, the model will only produce average-looking faces, with characteristic visual features that are correlated with the input speech. It will not produce images of specific individuals.”

However, if the algorithm becomes so sophisticated that it could recreate super-realistic faces, what impact could it have? The first thought that comes to my mind is that technology like this could be of immense help to police officers and detectives… Or I’m just looking too many crime TV shows. On the other hand, it could have a negative impact on YouTube and TikTok stars who’re trying to save their private life from followers so they only do voiceovers and don’t appear in front of the camera. But like every technology, I guess this one could be super-useful in good hands, and dangerous in bad ones.

[via PetaPixel]

FIND THIS INTERESTING? SHARE IT WITH YOUR FRIENDS!

  • Share
  • Tweet
  • Flipboard
  • WhatsApp

Related posts:

Disney’s face swapping AI is fast, accurate, and creepy as hell Tennessee: Shooting creepy photos of women ruled legal because it was done in public Photographer called out over creepy “Sexual Arousal” clause in model release form Instagram now lets you send voice messages

Filed Under: news Tagged With: AI, Artificial Intelligence, creepy

Dunja Djudjic: from diyphotography.net

About Dunja Djudjic

Dunja Djudjic is a multi-talented artist based in Novi Sad, Serbia. With 15 years of experience as a photographer, she specializes in capturing the beauty of nature, travel, and fine art. In addition to her photography, Dunja also expresses her creativity through writing, embroidery, and jewelry making.

« Fujifilm patents 28mm, 32mm and 35mm f/1.0 lenses – Are they expanding their f/1.0 lineup?
Curious owl photobombing weather camera goes viral »

Submit A Story

Get our FREE Lighting Book

DIYP lighting book cover

* download requires newsletter signup
DIYPhotography

Recent Comments

Free Resources

Advanced lighting book

Recent Posts

  • Comparing iPhone 13 vs iPhone 14 for astrophotography
  • Don’t buy a Z8 directly from Nikon, customers say
  • YouTube is killing off “Stories” to focus on Shorts and Live
  • Four ways to shoot epic stop motion hyperlapse with a smartphone gimbal
  • Canon reported to be working on a “zoomable” teleconverter

Udi Tirosh: from diyphotography.netUdi Tirosh is an entrepreneur, photography inventor, journalist, educator, and writer based in Israel. With over 25 years of experience in the photo-video industry, Udi has built and sold several photography-related brands. Udi has a double degree in mass media communications and computer science.

Alex Baker: from diyphotography.netAlex Baker is a portrait and lifestyle driven photographer based in Valencia, Spain. She works on a range of projects from commercial to fine art and has had work featured in publications such as The Daily Mail, Conde Nast Traveller and El Mundo, and has exhibited work across Europe

David Williams: from diyphotography.netDave Williams is an accomplished travel photographer, writer, and best-selling author from the UK. He is also a photography educator and published Aurora expert. Dave has traveled extensively in recent years, capturing stunning images from around the world in a modified van. His work has been featured in various publications and he has worked with notable brands such as Skoda, EE, Boeing, Huawei, Microsoft, BMW, Conde Nast, Electronic Arts, Discovery, BBC, The Guardian, ESPN, NBC, and many others.

John Aldred: from diyphotography.netJohn Aldred is a photographer with over 20 years of experience in the portrait and commercial worlds. He is based in Scotland and has been an early adopter - and occasional beta tester - of almost every digital imaging technology in that time. As well as his creative visual work, John uses 3D printing, electronics and programming to create his own photography and filmmaking tools and consults for a number of brands across the industry.

Dunja Djudjic: from diyphotography.netDunja Djudjic is a multi-talented artist based in Novi Sad, Serbia. With 15 years of experience as a photographer, she specializes in capturing the beauty of nature, travel, and fine art. In addition to her photography, Dunja also expresses her creativity through writing, embroidery, and jewelry making.

Copyright © DIYPhotography 2006 - 2023 | About | Contact | Advertise | Write for DIYP | Full Disclosure | Privacy Policy