What is your first thought when you hear about computer vision (CV) in fashion? Or, what is the first thing that pops into your head when you hear about deep learning fashion? Let us guess – online clothing shopping or virtual try-on applications?
Well, this might be surprising but deep fashion is not a far future anymore. What’s more, fashionably speaking, the usage of deep learning in the fashion industry seems to be already old-fashioned rather than pioneering or innovative. Many famous brands like Dior, Macy’s, Nike, Zara are already using artificial intelligence (AI) in e-commerce, and this is not only about market segments for retail clothing. There is far more than this within intelligent fashion. Most crucially, fashion is all about visuals. And where there are visuals, there is computer vision.
Let’s see how exactly data analytics and AI approaches entered the fashion industry and what happens when so seemingly different fields come together.
As mentioned above, AI-powered tools have been already deeply embedded in many creative fields such as art, film, music, graphic design, advertising, and fashion. Being a multibillion-dollar global industry, fashion is what creates, sets, and sells style and image, and quite often dictates canons of beauty.
Technically, making fashion truly intelligent is a very difficult task due to a huge variability of fashion items in style and design. Current trends on intelligent fashion are aimed at the tasks not only to detect clothing in an image but also analyze and synthesize new ones, and, hence, offer tailored recommendations. Within deep learning in the fashion industry, three main aspects appear definable: low-level pixel computation, mid-level fashion understanding, and high-level fashion analysis. The former is intended to label certain items on a picture and deals with human and cloth segmentation, landmark detection, and human pose estimation. Mid-level tasks aim to distinguish fashion images like items and styles. And finally, high-level analysis is recommendation-oriented, it includes synthesis and fashion trend forecast.
Here are the use-cases of how CV and deep learning are deepening fashion.
Try-before-You-Buy Solutions
An excellent example of CV-enabled fashion technologies is virtual fitting room applications. These allow potential customers to try on a garment or accessory using various software applications. You must admit it is great! Whether you choose glasses, watches, or hats, you are able to try on a model in real-time easily changing its color and shape.
Gap Dressing Room AR APP By Avametric – source
Such solutions are based on the pose estimation models used for landmarks detection. The deep fashion datasets might be taken from open-source libraries.
A yet harder task is to implement virtual try-on clothes. Because clothing alters its form when taking the shape of a person’s body, for proper augmented reality (AR) experience, a deep learning model should identify not only basic key points on the body’s joints but also the three-dimensional body shape.
Fashion Item Retrieval
Another benefit of using deep learning-based models is the fashion image retrieval task. For some, shopping can be enjoyable things to do, for others, could be absolutely frustrating. If you are not a shopping fan, when buying online can be even more challenging. You are just scrolling and scrolling, browsing gazillions of items, and could nohow find what you’re looking for. Or another case, imagine you saw a gorgeous Jennifer Lopez’s dress/purse/waistband (underline whichever is appropriate :)) and took fire to find something like that. Although many online retailer websites support keyword-based searches, it would be much handier if a mechanism existed which could help us to find the desired apparel based on a visual query rather than a text description alone.
The great news is that CV may perfectly cope with this issue by finding a similar or alternative product you requested. And, most importantly, much faster than you would be searching by yourself. Still, clothing retrieval tasks based on queries by the customer’s picture is highly challenging. This is due to a significant discrepancy between the real-world photos and those captured by retailers. Another problem is that clothing items are highly deformable, and, thus, their appearance may differ dramatically.
To solve the clothing retrieval task there is a trend to create attribute-aware deep neural network architectures that may include both semantic attributes and visual similarity constraints into the feature learning stage. Some of them may exploit over-segmentation algorithms with human pose estimation to get query clothing items and to retrieve similar images from the existing galleries.
And here you can bring up a question: how does CV “know” what exactly should be retrieved?
How Computer Vision Understands Fashion
We know that CV systems are trained to “look” at the picture and generate a list of features for each detected item. This is mainly accomplished by such technology as landmark detection. The fashion landmark detection means recognition of clothing in an image and categorization of fashion items. Fashion landmarks are to define the precise location of such functional clothing regions as a neckline, hemline, sleeves and cuff. However, detecting fashion landmarks is a challenging task due to such constraints as background noise, human poses, and scales. For achieving more accurate landmark prediction, CV algorithms should be more context-aware. Besides that the landmarks indicate the key points on clothes, they also capture their bounding boxes, which helps better discriminate the design, pattern, and class of apparel.
An example of fashion landmark detection – source
To be able to solve the above-mentioned tasks a number of clothes datasets come to the aid. One of the most widespread of them is Deepfashion2. It is a large-scale benchmark with comprehensive tasks and annotations, created by researchers from the Chinese University of Hong Kong. The dataset includes over 800K labeled into categories images with comprehensive descriptive attributes, bounding boxes, and clothing landmarks.
The Big Bang Theory series frame exemplifies the use-case of DeepFashion2 – source
DeepFashion2 allows performing a wide spectrum of tasks such as clothes detection, pose estimation, human and clothes segmentation, and clothing retrieval.
Fashion Recommendation with AI
A popular application of AI in clothing fashion is the deep learning recommendation engine. For e-commerce, it is all about categorization fashion items, clothing analysis, and help in certain style matching. Recommendation for fitting works on the concept of visual compatibility, which performs how favorable different fashion and apparel units can be matched to create a fashionable look. Also, it refers to the personalized recommendations considering such factors as preferable color, print, fabric, and outfit style. And since fashion is not only about what people are wearing but also reveals personality, fashion recommendation technology could help not only in certain cloth matching but in makeup or hairstyle suggestions. In other words, a customer benefits from the intelligent fashion-image consultant.
Another deep fashion application is a virtual assistant or chatbot. This kind of AI-powered software solution is an important part of business communication. Being an effective tool in user request analysis, it responds instantly and assists in keeping in touch with a customer throughout the whole purchase cycle.
Fashion Trends Forecasting Using Deep Learning
Given the frenetic pace in refreshment of fashion and design, retail businesses need to consistently keep up within the forefront and predict consumer preferences for the next season. Traditionally, such estimates are made based on the data from previous years. However, AI-based methods can reduce forecasting errors significantly.
Besides obvious business interests in sales forecasting for the retail clothing market, it is also important for consumers to choose appropriate fashion goods. Deep learning models for fashion are impressively helpful in analyzing current trends and customers’ behavior. So, knowing what is and what expected to be on-trend, businesses can deliver a better brand experience and, thus, provide exactly what shoppers look for.
