116 5. MULTIMODAL TRANSFER LEARNING
textual concepts. More formally, we define these fully connected layers as
8
ˆ
ˆ
ˆ
ˆ
ˆ
<
ˆ
ˆ
ˆ
ˆ
ˆ
:
e
1
D
1
.
W
1
a C b
1
/
;
e
2
D
2
.
W
2
e
1
C b
2
/
;
e
L
D
L
.
W
L
e
L1
C b
L
/
;
(5.7)
where W
l
, b
l
,
l
, and e
l
denote the weight matrix, bias vector, activation function, and output
vector in the l-th hidden layers, respectively. As for activation function in each hidden layer, we
choose Rectifier (ReLU) to learn higher-order concept interactions in a nonlinear way. Regard-
ing the size of hidden layers, common solutions are following the tower, constant, and diamond
patterns.
e output of the penultimate hidden layer is flattened to a dense vector e
L
, which is
passed to a fully connected softmax layer. It computes the probability distributions over the
venue category labels as
p
.
by
k
je
L
/
D
exp
e
>
L
w
k
P
K
k
0
D1
exp
e
>
L
w
k
0
; (5.8)
where w
k
is a weight vector of the k-th venue category and e
L
can be viewed as the fi-
nal abstract representation of the input x. ereafter, we obtain the probabilistic label vector
by D Œby
1
; : : : ;by
K
over the K venue categories.
ereafter, we adopt the regression-based function to minimize the loss between the es-
timated label vector and its target values, as
J
3
D
1
2
X
x2X
k
y by
k
2
; (5.9)
where an ideal model should predict the venue category correctly for each micro-video.
We ultimately obtain our objective function of the proposed deep transfer model by jointly
regularizing the sound knowledge transfer, multi-modal fusion, and DNN for venue estimation
as
J D J
1
C J
2
C J
3
: (5.10)
5.5.4 TRAINING
We adopted the stochastic gradient descent to train our model in a mini-batch mode and up-
dated the corresponding model parameters using back propagation. In particular, we first sam-
pled a batch of instances and took a gradient step to optimize the loss function of external sound
transfer. We then sampled a batch of .x
i
; x
j
;
ij
/ and took another gradient step to optimize the
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.