Detecting Artworks in Installation Photographs

Web-based Prototype for Artwork Detection

The interface was used to study installation photographs and in particular to search for specific artworks. This is of great value to art historians to answer questions regarding exhibition practices or assist with provenance research. In what context is the same artwork presented over time? In relation to which other works and artists? In which exhibitions did it appear? This might help to reconstruct its ownership? In the fall of 2016, the Museum of Modern Art (MoMA) in New York released a large set of archival documents, including exhibition catalogues, press releases and installation views. The availability of these photographs has triggered the following study: Can we evaluate these images with the help of computer vision? The following examples illustrate a search initiated on the web-interface, which demonstrates how we can study installation photographs with computer-based methods.

The search algorithm was applied to installation photographs of the MoMA. The present image shows a photograph from the Museum's Collection exhibition held in 1940 and shows Balthus' portrait of Joan Miró and his daughter Dolores (1937, center). The image was selected as a query.

Copyright: Painting belongs to the Museum of Modern Art (New York), © 2018 Artists Rights Society (ARS), New York / ADAGP, Paris

The above image shows results of the search query; our algorithm was able to detect Balthus' portrait; a look at the corresponding metadata reveals that the painting was included in the exhibitions 'Painting and Sculpture from the Museum Collection' (1940), 'Portraits from the Museum Collection' (1960) and 'Modern Masters: Manet to Matisse' (1975). Positive detections were marked in green and negative ones in red by the user. The feedback is used to retrain the model, which then leads to better results.