gems Wang

More Than Met the Eye

(loading…)

More Than Met the Eye

Novel deep learning embedding-clustering analysis of The Metropolitan Museum of Art's photography collection

Is it possible to analyze a museum’s collection by the aesthetic and semantic content of objects within, beyond historical metadata?

In this class project, I propose and implement a novel methodology to analyze a collection of historic photographs. Each image is mapped to an embedding vector in a latent space by a Resnet-18 neural network model trained on ImageNet image classification. Then, the embeddings are clustered via a Ward hierarchical clustering model into "conceptual clusters."

We apply this methodology to the Metropolitan Museum of Art’s photograph collection, which primarily consists of 19th-century photos. Qualitatively, it performs surprisingly well in extracting clusters representing specific aesthetic ideas, semantic subject types, and photographic technologies, especially since neither the embedding network nor the clustering model has prior knowledge about the specific subject matter.

With these clusters, we analyze their contents, measure how they change over time, and speculate on further applications of the methodology.

In other words:

View the full paper here.

More Projects