6.2 Human-labelled Images

Another important type of ground-truth data used for evaluating visual attention models is the human-labelled images. Usually, the salient areas to the human eyes in visual scenes correspond to the salient objects. Many studies have used the saliency map to detect objects for natural images [3, 4, 7, 9]. The quantitative evaluation can be performed if an appropriate database with ground-truth is available. One widely used database of this type is the one including 5000 images with ground-truth salient objects marked with bounding boxes by nine subjects [7]. Some sample images from the database are shown in Figure 6.2(a). The human-labelled ground-truth data and the saliency maps of these images from the visual attention model in [15] are shown in Figure 6.2(b) and (c), respectively. Of course, it is possible for subjects to mark the salient objects more precisely (rather than just using bounding boxes as in Figure 6.2(b)) [10]. Additional human-labelled databases can be found in [7, 10].

Figure 6.2 (a) Sample images (from [7]); (b) the ground-truth (human-labeled) images; (c) the corresponding saliency maps (from the model in [15]). Figure 6.2(b) Reproduced from T. Liu, J. Sun, N. Zheng, X. Tang and H. Y. Shum, ‘Learning to detect a salient object,’ Microsoft Research Asia, http://research.microsoft.com/en-us/um/people/jiansun/salientobject/salient_object.htm (accessed November 25, 2012); Figure 6.2(c) © 2012 IEEE. Reprinted, with permission, from Y. Fang, W. Lin, B. Lee, C. Lau, Z. Chen, C. Lin, ‘Bottom-up Saliency Detection Model Based on Human Visual Sensitivity and Amplitude Spectrum’, IEEE Transactions on Multimedia, February 2012

Human- (manually-) labelled images can be obtained consistently only when the scene is relatively simple and contains not more than one salient object (or a cluster of salient objects), since it is not easy for subjects to tell which is the second fixation point, the third fixation point and so on, in a complex scenario. This is the reason why eye-tracking has been used for collecting ground-truth data, and we will deal with this next.