Google Lens now lets you search using videos

Google Lens is one of the most useful apps there is for all sorts of stuff: from copying and pasting your handwritten scribbles to examining skin changes. You can use your camera lens to learn about the world around you, and it now doesn’t only support still images but videos as well! This is the latest upgrade to the app, allowing it to answer near-real-time questions about your surroundings using video analysis.

Real-time video analysis

English-speaking Android and iOS users with the Google app installed can now start capturing a video via Lens and ask questions about objects of interest in the video. The feature leverages Google’s Gemini AI model to understand the video content and provide relevant answers.

Lou Wang, director of product management for Lens, explained in an interview with TechCrunch:

“Let’s say you want to learn more about some interesting fish. [Lens will] produce an overview that explains why they’re swimming in a circle, along with more resources and helpful information.”

To access this new video analysis feature, you’ll need to sign up for Google’s Search Labs program and opt in to the “AI Overviews and more” experimental features. Once enabled, holding your smartphone’s shutter button activates Lens’ video-capturing mode. Simply ask a question while recording and Lens will provide an answer based on the information gathered from the video and AI Overviews.

Other new features in Google Lens

As I mentioned, Google Lens has had more improvements. They include Image and Text Search and enhanced E-Commerce functionality, and Lou Wang reflects on both of them.

Image and Text Search

Lens now supports searching with images and text simultaneously. You can take a photo using the Google app and then ask a question verbally. Then, Lens will analyze the image and provide relevant results based on the text query. It’s worth noting that this is for English-speaking users only, and you don’t have to be signed up for Labs to use it.

Enhanced E-commerce functionality

Lens is also getting new features specifically designed for e-commerce. “Let’s say you saw a backpack, and you like it,” Wang said. “You can use Lens to identify that product and you’ll be able to instantly see details you might be wondering about.” However, this feature is still not available in all countries and, for now, only applies to certain shopping categories: electronics, toys, and beauty.

[via TechCrunch]

---