Finding what you're looking for on Facebook is about to get a whole lot easier.
The social media company has rolled out a new feature that lets users search for photos by describing their contents. The words searched don't have to appear in the caption or tag--so, in theory, typing "John Smith in front of a tree" should pull up just the photo you're thinking of.
According to a blog post published to the site Thursday, the feature uses Facebook's computer vision software, called Lumos. Engineers trained the platform's artificial intelligence by feeding it tens of millions of photos from Facebook's massive, user-uploaded collection.
Facebook tweaked the algorithm so the results are diverse, instead of pulling up several angles of the same subject or event. The results are separated into photos from your friends and ones posted publicly posted to Facebook.
The feature can return results for objects as well as concepts, like "travel photos" or "weird hat." A few test searches seem to indicate that it relies pretty heavily on caption language for those. (It also, apparently, thinks my curly-haired little godson is wearing a weird hat.)
In theory, the A.I. will become smarter, and thus more accurate, over time.
Facebook's computer vision system is what powers the "automatic alternative text" feature it rolled out last year, which describes photos to visually impaired people. It's also why you so rarely see offensive photos on the site--the A.I. is able to recognize the contents of a photo with a high degree of accuracy and pull any images that contain nudity, violence, and so on. That's not without its pitfalls, though: In September, the company received flak when its censoring algorithm removed the powerful photo of a naked child running from napalm bombs during the Vietnam War. (Facebook soon restored the photo, citing its historical significance.) Such pitfalls are likely to befall Lumos, especially in its earliest days.
Like Facebook, Google uses computer vision technology in its Photos feature, allowing users to easily search their collection of images. It recently released a platform called PlaNet, which can with some accuracy identify where a photo was taken by studying landmarks, vegetation, and languages on signs in the background.
The new search feature is available in the U.S. to both mobile and web users. In the blog post, Facebook says it's in the process of applying its Lumos technology to videos.