Peter Bell,
Masato Takami,
Lisa Dieckmann &
Björn Ommer
The search algorithm used in the "Passion Search" is designed to eliminate the flaws of the bag-of-words model and should therefore be able to work with arbitrary search requests. With the help of a user-friendly web interface, the user can select particular sections of the image in order to search for them within the dataset. The image can be uploaded by the user or it can be a section of an existing image of the dataset. During the search, positive results can be taken as training examples in order to improve the search results. The algorithm not only enables the detection of identical or very similar image sections but also detects larger variations. The results are then sorted according to the degree of similarity.
The image section that was initially searched for appears on the top left of the page and is followed by photographs of the same art work. After these results, the less similar variations of the image section appear as well as related objects. The search results show the more similar results before the less similar ones. This enables the user to understand the search process and determine for him/herself at which point the results become less relevant.
Although the program has not been set up with an object semantic, some search results still show semantic relationships through the image sections selected by the user.
Indeed not only objects from the core of the image can be searched for but also objects from the outer areas of the image.
The search in Fig 7. is meaningful especially because of the halo and the veil. In comparison to the images of Christ, there is a noticeably larger variety of images. There is also an increase in the variety of image results for the other figures from the Passion.
In other cases, the selection of geometrical forms can lead to a reliable detection of relatively rare objects such as in Fig 8.
In some cases, semantically meaningful compositions are easily detected and at the same time provide an indication of the epoch, as seen in Fig.9.
The combined search boxes (white) can be assigned with a logical "and" or "or" link (Fig 10). If the user is unsure whether two objects appear together in a single art work, it is useful to use the "or" function.
For further information and prototype testing please contact: sabine.lang@iwr.uni-heidelberg.de
Objective
To search within 3620 crucifixion images from the Prometheus Image Archive (Cologne)
The search algorithm used in the "Passion Search" is designed to eliminate the flaws of the bag-of-words model and should therefore be able to work with arbitrary search requests. With the help of a user-friendly web interface, the user can select particular sections of the image in order to search for them within the dataset. The image can be uploaded by the user or it can be a section of an existing image of the dataset. During the search, positive results can be taken as training examples in order to improve the search results. The algorithm not only enables the detection of identical or very similar image sections but also detects larger variations. The results are then sorted according to the degree of similarity.
 |
| Fig 1. The motif itself. A semantically meaningful area of the image is selected through two search boxes on the beams of the cross. The initial search image selected by the user always appears on the top left (as well as in the bar at the bottom of the page). The results decrease in similarity from left to right and from the first row to the last. |
 |
| Fig 2. Detection of objects with the same style, composition and medium |
 |
| Fig 3. Detection of objects from serial artistic production |
The image section that was initially searched for appears on the top left of the page and is followed by photographs of the same art work. After these results, the less similar variations of the image section appear as well as related objects. The search results show the more similar results before the less similar ones. This enables the user to understand the search process and determine for him/herself at which point the results become less relevant.
 |
| Fig 4. Canonical image sections: The legs of Christ |
Although the program has not been set up with an object semantic, some search results still show semantic relationships through the image sections selected by the user.
 |
| Fig 5. Canonical motifs such as Christ's face allow for a comparison that spans a variety of different art epochs. However, it is interesting to note that most of the results here are from the Middle Ages. (This could be to some extent because of the form of the halo.) |
Indeed not only objects from the core of the image can be searched for but also objects from the outer areas of the image.
 |
| Fig 6. Form and materiality of art objects |
The search in Fig 7. is meaningful especially because of the halo and the veil. In comparison to the images of Christ, there is a noticeably larger variety of images. There is also an increase in the variety of image results for the other figures from the Passion.
 |
| Fig 7. Depictions of Mary |
In other cases, the selection of geometrical forms can lead to a reliable detection of relatively rare objects such as in Fig 8.
 |
| Fig 8. Selection of an image with geometrical forms |
In some cases, semantically meaningful compositions are easily detected and at the same time provide an indication of the epoch, as seen in Fig.9.
 |
| Fig 9. Selection of a semantically meaningful composition |
The combined search boxes (white) can be assigned with a logical "and" or "or" link (Fig 10). If the user is unsure whether two objects appear together in a single art work, it is useful to use the "or" function.
 |
| Fig 10. Usage of two combined search boxes. |
For further information and prototype testing please contact: sabine.lang@iwr.uni-heidelberg.de