DIY Photography

Your one stop shop for everything photo-video

  • News
  • Inspiration
  • Reviews
  • Tutorials
  • DIY
  • Gear
Search

Submit A Story

Nvidia has developed neural network that could revolutionise video streaming

Oct 7, 2020 by John Aldred 10 Comments

  • Share
  • Tweet
  • Flipboard
  • WhatsApp

One of the big problems with video, especially when watching it online, is the amount of bandwidth it often takes up. This problem of bitrates is particularly a problem when it comes to things like live streaming and video conferencing. Researchers at Nvidia think they’ve found a way around the limitations of existing video streaming codecs, though, with the development of a new neural network engine.

The new engine works by completely ignoring every traditional video codec out there in order to bring the amount of bandwidth required for video streaming down to a fraction of what it might normally use with something like h.264.

The new technology is possible due to Nvidia Maxine, their cloud-AI video streaming platform for developers. Nvidia posted a video explaining the new technology to YouTube, which you can see above.

It essentially works by sending along the usual keyframes like you’d expect with h.264, but instead of generating whole in-between frames and sending those down the pipe, as would happen with h.264, it creates a sort of mask of the subject, focusing on key parts of their face. The movements of these key locations is then sent instead of image data for each frame, and the recipient sees an AI-reconstructed image.

This means that the bandwidth is drastically reduced as only a relatively tiny data set is sent to the recipient instead of entire images or blocky chunks of changed pixel data.

Nvidia says that this technology also offers some other advantages for video conferencing, too, such as being able to reposition the subject’s head to face the camera. For most of us, looking directly at the camera isn’t possible. Our cameras are usually either on top of the monitor that’s displaying the other side of the conversation or they’re off to the side – so we’re not looking directly at the camera. The new Nvidia technology solves that problem, by helping to make sure subjects are looking at each other while they talk.

And, naturally, because it’s able to understand the shape and structure of your face, you’re able to swap yourself out for things like virtual characters – although that isn’t really anything new. My phone’s been able to do that for a couple of years already now.

The technology is still in its early days, and you can see that there are some issues with this technique in the video up top. When you compare the quality vs the amount of bandwidth used, though, there’s really no comparison when it comes to something like video conferencing.

As time progresses and the technology gets more advanced and more reliable and cameras and screens keep increasing in resolution, this or a similar technology could potentially become the key to streaming super high resolution and high frame rate data of all types into our homes without having to go through major hardware upgrades to increase the available bandwidth.

[via DPreview]

FIND THIS INTERESTING? SHARE IT WITH YOUR FRIENDS!

  • Share
  • Tweet
  • Flipboard
  • WhatsApp

Related posts:

Your Next Smart Selection Could Be Powered By A Neural Network Neural network turns your Pixelated, black and white gameboy photos into realistic portraits Neural network creates photo-realistic images of people who aren’t real New neural network repairs damaged and low-quality images

Filed Under: news Tagged With: AI, Neural Networks, Nvidia

John Aldred: from diyphotography.net

About John Aldred

John Aldred is a photographer with over 20 years of experience in the portrait and commercial worlds. He is based in Scotland and has been an early adopter - and occasional beta tester - of almost every digital imaging technology in that time. As well as his creative visual work, John uses 3D printing, electronics and programming to create his own photography and filmmaking tools and consults for a number of brands across the industry.

« This weird dome is a portable light tent for product photography
How to fix the biggest problem with the Godox AD400/AD600 Pro strobes »

Submit A Story

Get our FREE Lighting Book

DIYP lighting book cover

* download requires newsletter signup
DIYPhotography

Recent Comments

Free Resources

Advanced lighting book

Recent Posts

  • Comica CVM-VM30 is the world’s first wireless shotgun mic
  • Everything you need to know about science/technology photography
  • How Fujifilm survived while Kodak didn’t
  • Use these two simple methods to remove skin shine from portraits
  • Improved Sun Moon Expert app gets you perfectly timed sun and moon photos

Udi Tirosh: from diyphotography.netUdi Tirosh is an entrepreneur, photography inventor, journalist, educator, and writer based in Israel. With over 25 years of experience in the photo-video industry, Udi has built and sold several photography-related brands. Udi has a double degree in mass media communications and computer science.

Alex Baker: from diyphotography.netAlex Baker is a portrait and lifestyle driven photographer based in Valencia, Spain. She works on a range of projects from commercial to fine art and has had work featured in publications such as The Daily Mail, Conde Nast Traveller and El Mundo, and has exhibited work across Europe

David Williams: from diyphotography.netDave Williams is an accomplished travel photographer, writer, and best-selling author from the UK. He is also a photography educator and published Aurora expert. Dave has traveled extensively in recent years, capturing stunning images from around the world in a modified van. His work has been featured in various publications and he has worked with notable brands such as Skoda, EE, Boeing, Huawei, Microsoft, BMW, Conde Nast, Electronic Arts, Discovery, BBC, The Guardian, ESPN, NBC, and many others.

John Aldred: from diyphotography.netJohn Aldred is a photographer with over 20 years of experience in the portrait and commercial worlds. He is based in Scotland and has been an early adopter - and occasional beta tester - of almost every digital imaging technology in that time. As well as his creative visual work, John uses 3D printing, electronics and programming to create his own photography and filmmaking tools and consults for a number of brands across the industry.

Dunja Djudjic: from diyphotography.netDunja Djudjic is a multi-talented artist based in Novi Sad, Serbia. With 15 years of experience as a photographer, she specializes in capturing the beauty of nature, travel, and fine art. In addition to her photography, Dunja also expresses her creativity through writing, embroidery, and jewelry making.

Copyright © DIYPhotography 2006 - 2023 | About | Contact | Advertise | Write for DIYP | Full Disclosure | Privacy Policy