Add Computer Vision Features to E-commerce App

Learn how to create and train your own custom image categorization model powered by visual artificial intelligence with Ximilar.

Michal Lukáč June 27, 2017

5 minutes of reading

Table of Contents

How to Plan Computer Vision Features and Choose the Right Provider

Earlier, I wrote a post about the difference between general and custom computer vision platforms. Today, I would like to focus on real-world use cases. Let’s dive into image recognition features planning.

An Imaginary Use Case: Sock Recognition With AI

We are running a small business to sell colourful socks. We want to add a “Socks Matching Engine” feature to our app. Customers will upload a picture of two different socks and apps tell them the visual alternatives, which will be filtered on basic features such as colour or pattern. This is sometimes called a visual search. This way we are going to gain respect for Hollywood’s fashion policy and increase sales!

Plan the Image Recognition Task

Before we start, let’s think about this.

Vision Model — “Parameter Extractor”

What we want is a model designed to extract information from images. We will focus on several parameters:

Pattern (dotted, striped, winter, summer)
Colour
Sock type (ankle length, quarter length, crew length)

Each customer’s image is evaluated and labelled. This provides us with information about what are the favourite colours, patterns, and types of socks of each customer. That’s great because we can now customise the next newsletter to fit your customer’s style.

At this point, matching can be as simple as adding a few rules saying that blue and orange socks go together, striped goes with dotted and so on. This, of course, is a hack that does not bring much value into the fashion field, but it will work at the beginning.

We can also align categories with our e-shop categories and recommend customers similar socks to those they already have. When you have enough images collected, we are ready to build a “Fashion advisor” model. We will also keep data extraction models to help us understand the customers and make clever suggestions.

Finding the Right Providers

Now we know what functions we are looking for:

Extract colour
Extract pattern
Extract sock type
Custom fashion recommender

The most important is model accuracy. There is no solution that can provide 100% accuracy because your customer’s images are going to be so much different. Reaching 80–90% accuracy is great!

Extract Colour From Image via API

It is easy to define a colour using the general categorization model. Our Dominant colours service should work in this case. We can extract dominant colours with drag and drop via demo or extract them via API. You can ignore the background and analyse the colours of the product. We can use these colours for filtering socks from our shop site for a specific colour.

The top 3 dominant colours of the sock image were analysed via the Ximilar Dominant Colors service.

Extract Patterns With AI

This might be the hardest part to recognize. I recommend training custom models for patterns because we want every image to have a pattern label. General models can detect strong patterns but do not provide patterns for every option. Having 20 pattern categories means we need only about 400 images of socks for custom model training. More information about the custom vision dataset is mentioned in this post. A custom image recognition model can be trained online via a browser, just log in to app.ximilar.com and build your own models.

Trained AI image categorization model via Ximilar platform for identifying patterns.

Identify the Type of Sock

General vision can also detect the type of sock. I tested a few socks and got mostly “outdoor shoe” results, which are not very accurate. I prefer to spend one more hour on getting images from my e-shop database and sorting them into classes, rather than having blank spaces in my image recognition engine. Using a custom categorization model also leads to higher accuracy in classification.

Having three parameters extracted from an image and saved in the database, we are now able to create the matching logic.

Identifying subcategories of products with machine learning model.

Searching and Recommending Socks

Building your own image-recommending engine can be hard. Luckily, Ximilar offers a solution that can easily find products from your database of socks. We can use previous models for filtering the result on Sock type, pattern and colour.

Showing visually similar socks with Ximilar Visual Search services.

Summary

Most computer vision tasks (models) are more complex than what one provider can deliver. It often needs some business insight, so it is necessary to take time and think about the path to our goals. Most of the solutions like Google Vision or Azure AI services are very expensive for training and deploying your custom models.

Ximilar provides easy to use and powerful solution for training and deploying your custom machine learning models for images. The training and deployment of the image recognition models are free, which saves your expenses for the development of your business idea. If you have some business ideas that require a customized computer vision solution, then contact us. We have experts and tools that can help you grow your project.