8.2. VIRTUAL TRY-ON WITH ARBITRARY POSE 99
integrate the auxiliary template generation into the primary item-item compatibility modeling
and boost the performance; (2) how to accurately generate a compatible bottom template for the
given top to guide the item-template compatibility modeling arises the second challenge; and
(3) as each fashion item may involve multiple modalities (i.e., visual and contextual modalities),
both of which can convey important information toward the compatibility modeling, how to
effectively fuse the multi-modal factors poses the last challenge.
In the future, we plan to enhance the compatibility measurement between fashion items by
incorporating the auxiliary item-template link apart from the primary item-item correlation. On
the one hand, the multi-modality information of fashion items will be considered into the item-
item compatibility modeling network to comprehensively measure the compatibility between
different fashion items. On the other hand, an auxiliary complementary template generation
network is expected to transfer the given top to a compatible bottom template.
8.2 VIRTUAL TRY-ON WITH ARBITRARY POSE
As aforementioned that all the clothing items are served for people and the user factor, like
the user body shape, plays an pivotal role in the personalized fashion compatibility modeling,
the try-on effect is a major concern for people. Toward this end, owing to the recent advances
in computer graphics, several efforts have been made to tackle the online virtual fitting prob-
lem with 3D modeling, such as TriMirror.
2
Despite that 3D-based methods are able to make
the realistic clothing try-on, the huge labor costs for 3D annotated data and expensive scanner
equipment limit their real-world applications.
Fortunately, to demonstrate fashion products more intuitively, e-commerce websites, such
as Zalando,
3
usually display images of fashion models wearing their products apart from the pure
product images. e tremendous publicly available images open the door to the virtual try-on
with economical 2D modeling. Although several pioneer researches have achieved promising
performance in the virtual try-on, existing efforts can only generate single-view results, that
is, keeping the personal posture unchanged while changing the clothing item on the person.
However, in reality, people may prefer to have different views wearing the outfit and then assess
the compatibility. In light of this, we define a new virtual try-on task, where given a person
image with the old clothing item, a personal desired pose, and a target clothing item, we aim to
automatically generate a person image with the target clothing item in his/her desired pose, as
illustrated in Figure 8.2.
Indeed, advanced image generation and synthesis models such as Generative Adversarial
Networks (GANs) and Variational Autoencoders (VAEs) have demonstrated remarkable per-
formance over various image generation tasks. However, it is suboptimal to directly apply these
methods to solving the complex try-on task due to the following reasons. (1) In the context of
virtual try-on, the body shape and desired pose of the target person highly affect the final look of
2
https://www.trimirror.com/
3
https://zalando.com/
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.