110 5. MULTIMODAL TRANSFER LEARNING
0.4
0.3
0.2
0.1
0.0
4.5
4.0
3.5
3.0
2.5
2.0
1.5
Percentages
Log (Video#)
1
(not at all)
2
(slightly)
4
(very)
5
(extremely)
3
(moderately)
0.22
0.41
0.21
0.1
0.06
Level of Importance Venue Category ID
0 20 40 60 80 100 120 140 160 180
(a) Affect Study (b) Dataset Unbalance
Figure 5.1: (a) Affect study regarding acoustic modality of micro-videos and (b) micro-video
unbalance distribution over the dataset [191].
sounds are hence in two heterogeneous domains. How to integrate them within a unified model
is largely untapped. Furthermore, according to our statistics over the benchmark dataset in [191],
we observe a very severe problem of unbalanced training data, as shown in Figure 5.1b. is
phenomenon tends to guide a bias winner-takes-all model, i.e., unseen samples are probably
classified into the venue category with the most training micro-videos.
5.3 RELATED WORK
Our work is closely related to multimedia location estimation, dictionary learning, and acoustic
concept detection. We have introduced multimedia location estimation and dictionary learning
in Sections 4.3.1 and 4.3.3, respectively. We hence only detail the acoustic concept detection in
this part.
Acoustic concept detection on the user-generated videos is a relatively new field in mul-
timedia community [135], composed of the data-driven [17, 20] and task-driven [2, 128] ap-
proaches from the perspective of modeling acoustic concepts. e main motivation of acoustic
concept detection is that audio analysis provides a complementary information to detect the
specific events that are hardly identified with visual cues. Recent studies [167] have shown that
detecting sound events to bridge the gap between the low-level features and the high-level se-
mantics outperforms the pure feature-based approaches. Different from acoustic concept detec-
tion, we target at constructing a knowledge base of acoustic concepts and leveraging such base
to strengthen the representation learning of micro-videos.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.