In Chapter 4, we introduce a series of multimodal cooperative learning methods toward micro-

video understanding, especially on the task of estimating their venue categories. It utilizes the

consensus information on visual, acoustic, and textual modalities of micro-videos to recognize

the venue information. is multi-modal method, however, overlooks that the description of

each modality may differ dramatically. According to the experiments [191], the acoustic modal-

ity has the weakest capability of indicating venue information. It will cause the substantial cask

effect, which is crucial to venue estimation and further content understanding.

To gain deep insights, we performed a user study to verify the acoustic influence on esti-

mating venue category. Given 100 micro-videos randomly selected from Vine, 5 volunteers were

(2) the acoustic modality in 84% of micro-videos is insufficiently descriptive (below 4) to sup-

port the venue category estimation, pertaining to their noise and low-quality. is study shows

that detecting acoustic concepts is useful yet needs further enhancement.

Leveraging external rich sound knowledge to compensate the internal acoustic modality is an

intuitive thought. It is, however, non-trivial owing to the following challenges: (1) as micro-

videos often record events, it is desired to detect high-level acoustic concepts, which are more

discriminative for event description [141]. We thus have to learn the conceptual representation

of micro-videos; (2) micro-videos are about users’ daily activities, which restricts us to harness the

external real-life sound clips. However, to the best of our knowledge, there is no such sound data

collection available. Moreover, (3) the external sounds are mono-modal data, whereas the micro-

videos are unifying of textual, visual, and acoustic modalities. e micro-videos and external