4.4. MULTIMODAL CONSISTENT LEARNING 65
4.4.1 OPTIMIZATION
We adopt the alternating optimization strategy to solve the three variables A
s
, B, and W in
Eq. (4.3). To be more specific, we optimize one variable while fixing the others in each iteration.
We keep this iterative procedure until the objective function converges.
Computing A
s
with B and W fixed: we first fix B and W, and take derivative of with
respect to A
s
. We have
@
@A
s
D
1
X
s
A
s
B
X
s
C
2
A
s
: (4.4)
By setting Eq. (4.4) to zero, it can be derived that
A
s
D
1
X
sT
C
2
I
1
1
X
sT
B
; (4.5)
where I 2 R
D
s
D
s
is an identity matrix. e first term in Eq. (4.5) can be easily proven to be
positive definite and hence invertible according to the definition of positive-definite matrix.
Computing B with A
s
and W fixed: with A
s
and W fixed, we compute the derivative of
regarding B as follows:
@
@B
D
1
S
X
sD1
B X
s
A
s
C
BWW
T
YW
T
: (4.6)
By setting Eq. (4.6) to zero, we have
B D
YW
T
C
1
S
X
sD1
X
s
A
s
!
1
SI C WW
T
1
; (4.7)
where
1
SI C WW
T
can be easily proven to be invertible according to the definition of positive-
definite matrix.
Computing W with A
s
and B fixed: Considering that the last term in Eq. (4.3) is not
differentiable, we use an equivalent formulation of it, which has been proven by [3], to facilitate
the optimization as follows:
3
2
X
v2V
k
W
G
v
k
!
2
: (4.8)
Still, it is intractable. We thus further resort to another variational formulation of Eq. (
4.8).
According to the Cauchy-Schwarz inequality, given an arbitrary vector b 2 R
M
such that b ¤ 0,
we have
M
X
iD1
jb
i
j D
M
X
iD1
1
2
i
1
2
i
jb
i
j
M
X
iD1
i
!
1
2
M
X
iD1
1
i
b
2
i
!
1
2
M
X
iD1
1
i
b
2
i
!
1
2
; (4.9)
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.