DIY Photography

Hacking Photography - one Picture at a time

  • News
  • Inspiration
  • Reviews
  • Tutorials
  • DIY
  • Gear
Search

Submit A Story

This insane smartphone app lets you copy and paste elements of the real world into Photoshop

May 6, 2020 by John Aldred 3 Comments

  • Share
  • Tweet
  • Flipboard
  • WhatsApp

This is one of the craziest things I’ve ever seen, and I had to check my calendar to make sure it wasn’t April 1st, but French artist, designer and programmer, Cyril Diagne has figured out how to use augmented reality to literally copy and paste the world around you straight into a Photoshop document. Yeah, for real. And he’s even released the source code on GitHub!

Diagne announced his creation to the world via Twitter this week, and it really looks like some kind of After Effects fakery. If he hadn’t released the source code, then you probably wouldn’t even believe it was real at all. But it shows him clearly snapping objects around him, which are then cut out from their backgrounds and then pasted right into the current Photoshop document.

4/10 – Cut & paste your surroundings to Photoshop

Code: https://t.co/cVddH3u3ik

Book: @HOLOmagazine
Garment: SS17 by @thekarentopacio
Type: Sainte Colombe by @MinetYoann @ProductionType
Technical Insights: ↓#ML #AR #AI #AIUX #Adobe #Photoshop pic.twitter.com/LkTBe0t0rF

— Cyril Diagne (@cyrildiagne) May 3, 2020

According to Diagne, the “secret sauce” top the augmented reality is BASNet for object detection and background removal – which he describes as “stunning”. OpenCV SIFT  is used to determine where on the computer screen the phone is pointing in order to paste. Then it sends it over to Photoshop via a Python plugin and up it pops in your document.

The latency for picking out the object in the real world, Diagne says, is about 2.5 seconds and it takes about a further 4 seconds in order to paste it into Photoshop. but to even do it that quickly is absolutely incredible.

It works so well, that even Adobe were impressed, with the official Photoshop Twitter account sharing it out.

And again, the OpenCV SIFT trick to find where the phone is pointing at the screen.

I also packaged it as a small python library: https://t.co/en0EyGSklp

Send a camera image + a screenshot and you get accurate x, y screen coordinates! pic.twitter.com/OOCFrrZseZ

— Cyril Diagne (@cyrildiagne) May 3, 2020

It is a fascinating technology. It’s one that will make a lot of people extremely happy, and quite a few people afraid for their job’s future! But it’s not ready for prime time yet. It’s a “research prototype” – essentially a proof of concept. But with the source code freely available (and knowing that Adobe has seen it), maybe this is something we’ll see coming to Lightroom Mobile at some point?

[via PetaPixel]

FIND THIS INTERESTING? SHARE IT WITH YOUR FRIENDS!

  • Share
  • Tweet
  • Flipboard
  • WhatsApp

Related posts:

This augmented reality app lets you plan your shot in real time from your phone Augmented Reality Photo Studio app lets you photograph virtual models in real locations Real-Time Digital Make Up Alters Real Faces In… Real-Time This app lets you use your smartphone as an external Lightroom control panel

Filed Under: news Tagged With: Adobe Photoshop, ar, augmented reality, Cyril Diagne

About John Aldred

John Aldred is a photographer with over 20 years of experience in the portrait and commercial worlds. He is based in Scotland and has been an early adopter - and occasional beta tester - of almost every digital imaging technology in that time. As well as his creative visual work, John uses 3D printing, electronics and programming to create his own photography and filmmaking tools and consults for a number of brands across the industry.

« Five ways to master manual mode
Coronavirus allegedly forces B&H to furlough 20% of its employees »

Submit A Story

Get our FREE Lighting Book

DIYP lighting book cover

* download requires newsletter signup
DIYPhotography

Recent Comments

Free Resources

Advanced lighting book

Recent Posts

  • Sony teases upcoming ZV-E1 full-frame vlogging camera coming on March 29
  • The Xencelabs Pen Display 24 is silent, glare-free retouching tablet
  • Fall in love with astrophotography with these 10 space objects
  • Hipstamatic app relaunches as a social network, but only for iOS
  • Instagram now has ads even in search results. Sigh

Alex Baker is a portrait and lifestyle driven photographer based in Valencia, Spain. She works on a range of projects from commercial to fine art and has had work featured in publications such as The Daily Mail, Conde Nast Traveller and El Mundo, and has exhibited work across Europe

Dave Williams is an accomplished travel photographer, writer, and best-selling author from the UK. He is also a photography educator and published Aurora expert. Dave has traveled extensively in recent years, capturing stunning images from around the world in a modified van. His work has been featured in various publications and he has worked with notable brands such as Skoda, EE, Boeing, Huawei, Microsoft, BMW, Conde Nast, Electronic Arts, Discovery, BBC, The Guardian, ESPN, NBC, and many others.

John Aldred is a photographer with over 20 years of experience in the portrait and commercial worlds. He is based in Scotland and has been an early adopter - and occasional beta tester - of almost every digital imaging technology in that time. As well as his creative visual work, John uses 3D printing, electronics and programming to create his own photography and filmmaking tools and consults for a number of brands across the industry.

Dunja Djudjic is a multi-talented artist based in Novi Sad, Serbia. With 15 years of experience as a photographer, she specializes in capturing the beauty of nature, travel, and fine art. In addition to her photography, Dunja also expresses her creativity through writing, embroidery, and jewelry making.

Copyright © DIYPhotography 2006 - 2023 | About | Contact | Advertise | Write for DIYP | Full Disclosure | Privacy Policy