DIY Photography

Your one stop shop for everything photo-video

  • News
  • Inspiration
  • Reviews
  • Tutorials
  • DIY
  • Gear
Search

Submit A Story

NVIDIA’s GauGAN AI landscape generator can now create scenes from scratch from written descriptions

Nov 22, 2021 by John Aldred Add Comment

  • Share
  • Tweet
  • Flipboard
  • WhatsApp

NVIDIA has been doing a lot of cool stuff with AI. One of those things is GauGAN AI, something of a predecessor to NVIDIA’s Canvas application, which we checked out here when I tried to recreate some of my landscape photographs with it. Well, GauGAN2 is here now and it’s gotten smarter. Way smarter. You don’t just paint coloured pixels where you want things to be anymore. Oh no, now it actually understands what you say!

And unlike Canvas, which is free to download but limited to those with an NVIDIA RTX GPU, anybody can have a go and play with GauGAN2 through its handy web interface.

The GAN part of GuaGAN stands for generative adversarial networks. Put simply, this is two networks competing against each other. One is generating something that it hopes looks real while the other is looking at that something to see if it’s real or fake when compared to the real somethings it knows exist in the real world. This second half of the equation is also able to go back to the generator and tell it how to improve.

That’s a very very simplified explanation of the process, but that’s essentially how GuaGAN works to generate its AI landscapes. And now you can generate them by entering a simple phrase, like the NVIDIA suggestions “sunset at the beach” or “ocean waves hitting rocks on the beach”. Or, you can do what I did and see how well it handles “misty forest with mountains in the background and a blue sky” and see what happens.

Well, this happened…

It’s not quite what I’d envisioned, but it’s pretty close. I certainly didn’t see that many clouds in my head, but the misty forest mountains are spot on. And this was just from a single line of text without actually drawing anything on the screen manually.

You don’t only have to paint or describe a scene, though. You can do a combination of both, generating a segmentation map of your description and then fine-tuning it by brushing on extra information and details. NVIDIA says that GuaGAN2 is one of the first examples to combine multiple modalities within a single network, making it a powerful image generation tool for creating realistic art with a mix of words and simple sketches.

Ultimately, NVIDIA wants to make it faster and smarter to let artists easily turn their ideas to images. Rather than needing to draw out an entire scene from scratch, they’ll be able to describe a scene and have the AI generate something that users can than use as a starting point for their work. NVIDIA says, “this starting point can then be customized with sketches to make a specific mountain taller or add a couple of trees in the foreground, or clouds in the sky.”

It’s a fascinating piece of technology and I’m looking forward to seeing this improve in the future. Already from my simple tests with GauGAN2, it’s already come a long way from when I tested Canvas just a couple of months ago.

If you want to find out more, head on over to the NVIDIA Blog and be sure to have a play with it for yourself!

FIND THIS INTERESTING? SHARE IT WITH YOUR FRIENDS!

  • Share
  • Tweet
  • Flipboard
  • WhatsApp

Related posts:

Instagram adds photo descriptions to aid visually impaired users Tutorial: How To Create Film Sets From Scratch This AI turns rough doodles into realistic landscape photos Random Photo Marketing Generator promises “old-world craftsmanship” for forum warriors and cat photos

Filed Under: news Tagged With: AI, Artificial Intelligence, doodle, landscape, neural network, Neural Networks, Nvidia, Sketch

John Aldred: from diyphotography.net

About John Aldred

John Aldred is a photographer with over 20 years of experience in the portrait and commercial worlds. He is based in Scotland and has been an early adopter - and occasional beta tester - of almost every digital imaging technology in that time. As well as his creative visual work, John uses 3D printing, electronics and programming to create his own photography and filmmaking tools and consults for a number of brands across the industry.

« On the depiction of Africans in photo contests
These colorized photos pay a tribute to animals of the First and Second World War »

Submit A Story

Get our FREE Lighting Book

DIYP lighting book cover

* download requires newsletter signup

Recent Comments

Free Resources

Advanced lighting book

Learn photography

Recent Posts

  • Nikon announces Zf shortage (Surprise!)
  • Finally! Sennheiser EW-DP SKP plug-on transmitter offers 32-bit float recording
  • Photographing an abandoned Italian villa: A time capsule of history and mystery
  • The Tascam DR-10L Pro is an unclippable 32-bit float personal audio recorder
  • This year’s Nature TTL POTY winner puts global warming right in our faces

Udi Tirosh: from diyphotography.netUdi Tirosh is an entrepreneur, photography inventor, journalist, educator, and writer based in Israel. With over 25 years of experience in the photo-video industry, Udi has built and sold several photography-related brands. Udi has a double degree in mass media communications and computer science.

Alex Baker: from diyphotography.netAlex Baker is a portrait and lifestyle driven photographer based in Valencia, Spain. She works on a range of projects from commercial to fine art and has had work featured in publications such as The Daily Mail, Conde Nast Traveller and El Mundo, and has exhibited work across Europe

David Williams: from diyphotography.netDave Williams is an accomplished travel photographer, writer, and best-selling author from the UK. He is also a photography educator and published Aurora expert. Dave has traveled extensively in recent years, capturing stunning images from around the world in a modified van. His work has been featured in various publications and he has worked with notable brands such as Skoda, EE, Boeing, Huawei, Microsoft, BMW, Conde Nast, Electronic Arts, Discovery, BBC, The Guardian, ESPN, NBC, and many others.

John Aldred: from diyphotography.netJohn Aldred is a photographer with over 20 years of experience in the portrait and commercial worlds. He is based in Scotland and has been an early adopter - and occasional beta tester - of almost every digital imaging technology in that time. As well as his creative visual work, John uses 3D printing, electronics and programming to create his own photography and filmmaking tools and consults for a number of brands across the industry.

Dunja Djudjic: from diyphotography.netDunja Djudjic is a multi-talented artist based in Novi Sad, Serbia. With 15 years of experience as a photographer, she specializes in capturing the beauty of nature, travel, and fine art. In addition to her photography, Dunja also expresses her creativity through writing, embroidery, and jewelry making.

Copyright © DIYPhotography 2006 - 2023 | About | Contact | Advertise | Write for DIYP | Full Disclosure | Privacy Policy