Google’s new Pixel 9’s AI is capable of searching

through your screenshots.

 

 

 

Your screenshots may occasionally contain crucial information, but it’s not always easy to find it.

Right present, my phone’s photo library contains 4,811 screenshots, many of which contain data I intended to keep. But actually finding them when I needed them was the real challenge taking those photos was the easy part.

Google says it knows the answer.

Along with other new AI tricks, Google’s new Pixel 9 smartphones, which were unveiled on Tuesday at the company’s Bay View campus in Mountain View, California, may help you find vital information hidden away in screenshots you’ve previously taken, such as WiFi passwords, event details, and more.

The company says you only have to ask, using the recently released Pixel Screenshots app or by chatting with its AI-enabled Gemini assistant.

That suggests that the days of wasting hours scrolling through photos in quest of that one crucial photo are almost over, provided you’re ready to switch to a Pixel.

The software (ideally) responds to your request by giving the information extracted from the screenshot; if not, it further shows the screenshots from which the information was extracted.

You must consciously choose to use the functionality; it is not enabled by default.

Thankfully, Google claims that all of its picture analysis takes place immediately on its Pixel phones, mostly because of their new Tensor G4 processors. No screenshots are sent to distant servers to be evaluated.

This endeavor to imbue phones with a form of visual memory may prove surprisingly beneficial in everyday life, especially when contrasted with the current crop of AI tools integrated into smartphones, such as text rewriters, image manipulators, and voice transcribers.

After all, who among us hasn’t screenshotted something they wanted to save forever?

Naturally, there are restrictions. By default, the function is limited to extracting meaningful information from images that were taken lately; older screenshots stored in services such as Google Photos are not visible to it. (If you really wanted to, you could manually add older photographs to the Pixel Screenshots app.) While the app acknowledges that its answers “may be inaccurate,” it allows you to verify them.

For better or worse, Google has decided to keep this functionality proprietary to its Pixel 9 phones and will not be making it available to other Android phone manufacturers with whom it has previously collaborated on AI technology.

This is bad news for owners of Android smartphones, which will be in high demand given IDC data, as many of them already have the technology required to run AI models without connecting to the cloud.

According to the company, shipments of these “generative AI smartphones” should more than treble this year. However, as it looks to take a larger chunk of the smartphone market, Google may find it necessary to maintain some AI functions exclusive to its own products.

According to industry research firm Canalys, Google only held 0.8 percent of the worldwide smartphone market share in the first half of 2024, despite selling more Pixel phones annually since 2021.

A more conversational variant of Google’s Gemini AI, known as Gemini Live, is integrated into the company’s latest smartphones. It is capable of having lengthy, meandering talks with its owner, much how users may speak naturally to programs like OpenAI‘s ChatGPT. According to the business, Gemini can now respond to what’s shown on the phone’s screen and provide more details, outwitting Apple’s enhanced Siri and reaching users’ pockets simultaneously.

Runar Bjorhovde, an analyst at Canalys, stated, “Google had a first-mover advantage in the market that helped it differentiate, but the competition is catching up.”