Describe a scene to this AI and it creates a picture

I’ve had the chance to write about some amazing and creepy AI solutions over the past couple of days. And today, I was mind-blown by another one. It’s called DALL-E 2, and yeah, it’s kinda like an AI blend of WALL-E and Salvador Dali.

OpenAI’s cleverly named AI is able to create realistic “photos” based on your textual description. An astronaut riding a bike, a koala dunking a basketball, a bowl of soup that looks like a monster… You name it, DALL-E 2 will turn it into a picture that’s likely never been seen before.

DALL-E 2 is the younger and let’s say, the more handsome brother of DALL-E. It gives you higher-resolution and more accurate image results. It relies on your textual input created in natural language. It combines concepts, attributes, and styles to create the image you describe, and basically, the sky is the limit.

In addition, you can use DALL-E 2 for image editing. After all, AI-based editing seems to be a thing nowadays. You can modify existing images by giving the AI a natural language input. It can add and remove elements while taking shadows, reflections, and textures into account. In addition, you can create different variations of a photo (e.g. at different angles and in different styles) inspired by the original image.

My first thought when I started reading about DALL-E 2 was Dream by WOMBO app. It also uses text cues you give it to produce an AI-generated picture. However, the Dream app creates surreal artwork, while DALL-E 2 generates quite realistic pictures. For example, here’s the “monster plasticine bowl of soup” I got with the Dream app:

And here are a few examples of how DALL-E 2 imagines it:

“DALL·E 2 has learned the relationship between images and the text used to describe them,” writes OpenAI. “It uses a process called ‘diffusion,’ which starts with a pattern of random dots and gradually alters that pattern towards an image when it recognizes specific aspects of that image.”

This kind of technology could be used for a number of purposes by artists, illustrators, photographers, comic book creators… Or just by us regular folks out of pure fun. I’ve even signed up for the waitlist hoping to be able to test it out (you can do it here). However, we can’t disregard potential misuses such as fake news or fake revenge porn, to name a few. As always, even the most harmless technology can become a weapon in the wrong hands.

But OpenAI promises that they’ve taken all necessary steps to prevent the misuse of DALL-E 2. First, they’ve limited its ability to generate violent, hateful, or adult images. “By removing the most explicit content from the training data, we minimized DALL·E 2’s exposure to these concepts,” OpenAI writes. “We also used advanced techniques to prevent photorealistic generations of real individuals’ faces, including those of public figures.” Furthermore, OpenAI has both automated and human monitoring systems to guard against misuse. After all, DALL·E 2 is still a research project so it’s not available in OpenAI’s API. “As part of our effort to develop and deploy AI responsibly, we are studying DALL·E’s limitations and capabilities with a select group of users,” the company writes. So, as they improve it and protect it further, I hope that it will become available to everyone to test out. It sure seems fun!

[via PetaPixel]