Artrendex has launched ArtPI, which it described as: “a new interface or API driven by artificial intelligence that’s poised to transform the way art gets discovered, displayed, and sold.” The company said it has the potential to transform art discovery in the same way that Shazam transformed music discovery.

ArtPI detects features and patterns using overarching visual style characteristics based in art theory and history. Built from the ground up specifically for art, and drawing on centuries of artworks, ArtPI can find visually similar works, label styles and eras, recognize subject matter and artist, and find connections within a large collection.

For museums and other institutions, this means users can search their collection visually, exploring artworks in a more organic and intuitive way by looking for particular features. Curators can have the entire collection at their fingertips, finding unexpected ties and similarities that may inspire new ways to display or interpret works. For galleries, dealers, and auction houses, ArtPI can be used in conjunction with social media to uncover trends or predict market conditions.

“AI is often used to label images and objects in them, and ArtPI can also find works by subject matter,” said Ahmed Elgammal, Artrendex founder, Professor of Computer Science at Rutgers University, and the creator of ArtPI. “But you don’t always just want to find painting of dogs or putti. Sometimes you want a particular style of lighting or line. We’ve trained our algorithms to identify these elements and principles of art, based on foundational art historians’ approaches. By working with the visual elements instead of content labels or other metadata like artists or era, you can find art you didn’t know existed and see its connections with other works.”

ArtPI works even when images of an artwork are distorted, allowing for less-than-ideal source images or shots from a variety of angles. The AI model has been trained to correct for distortion, distilling a slanted or poorly-lit image into information that can be compared to other works. This removes the need for photographic documentation, enabling ArtPI to be used in a variety of less formal circumstances. Now, for example, you can see what work from a major exhibit won the hearts of Instagramers, even if they only took a quick snapshot of the piece.

AI can spot visual patterns, often better than humans. ArtPI employs neural networks to train its algorithms on images, using more than 250,000 artworks for its initial training phase. The more works encountered by the algorithm, the more nuanced the results. “Humans grasp context quickly, but AI is great at homing in on details and doing so for millions of examples in seconds or fraction of second,” said Elgammal. “Every search and every image added to ArtPI will contribute to its refinement, increasing its value over time.”

Philadelphia’s Barnes Foundation is one of the first institutions to deploy ArtPI. The Barnes first analyzed its extensive collection, which includes a large number of impressionist works. The first round of AI-powered analysis by different computer vision platforms generated some unexpected results: in one instance, the content of museum’s Renoirs were labelled as stuffed animals. However, once the museum’s works were inputted to ArtPI, some remarkable connections emerged.

“We thought this part of our process would be the hardest, yet it proved the easiest,” said Shelley Bernstein, the digital consultant on the Barnes project. “The visual similarities were so good, they were almost too good.” The problem: the Barnes collection had a wonderfully idiosyncratic way of displaying works together that originated with the collection’s founder. To recreate this approach online, the Barnes and ArtPI crafted a custom way to display results that better mirrors the real-life museum experience.

“Exploring art, be it for educational or valuation purposes, can be done in many ways, and AI is just another tool, another frame that helps us find new insights into the art we’re working with,” said Elgammal. “Being aware and optimized for the concepts of art, ArtPI is designed to be the infrastructure recognition engine for museums and the art market, in other words, the ‘Shazam for Art.’ I believe visual search and suggested connection can both democratize the discovery process for laypeople and renew the excitement curators, galleries, and auction houses feel as they consider the works in their domain.”